nvme: update error handling and ana_state to work well with device-mapper-multipath

BZ: 1948690 Upstream Status: RHEL-only

Forward-port RHEL8 NVMe changes to RHEL9. These changes were rejected by Christoph Hellwig (hch is one of upstream's 4 NVMe maintainers). The other (3) upstream NVMe maintainers don't have a problem with these changes, but Christoph is hostile toward any changes that make NVMe work better with dm-multipath (because he wants everyone to use NVMe's native multipathing, doesn't care that others have reasons why they want to use dm-multipath).

So unfortunately we need to just keep carrying these changes in RHEL. Can revisit getting these changes to land upstream in time for RHEL10, but I don't care to engage upstream (specifically hch) on this topic at this point in time.

Thankfully these NVMe changes are not intrusive, have proven stable/useful in RHEL8 and were easily forward-ported from RHEL8 to RHEL9.

Signed-off-by: Mike Snitzer snitzer@redhat.com