when the code was switched over to dk_ioctl() - countrary to disk_ioctl(),
dk_ioctl() returns ENOTTY for ioctls it doesn't support, so must be called
as last resort, not first
bug was introduced in rev 1.83 (2015-05-02) for ld(4), and 1.335 (2016-01-03)
for raid(4)
number of ioq from 128 to 1024; tested with VirtualBox and QEMU
* remove NVME_INTMC/NVME_INTMS writes in hw intr handler as this is not MPSAFE,
fortunately they don't seem to be necessary; shaves two register writes
* need to use full mutex_enter() in nvme_q_complete(), to avoid small
race between one handler exiting the loop and another entering
* for MSI, handover the command result processing to softintr; unfortunately
can't easily do that for INTx interrupts as they require doorbell write
to deassert
* unlock/relock q->q_cq_mtx before calling ccb_done to avoid potential deadlocks
* make sure to destroy queue mutexes when destroying the queue (LOCKDEBUG)
* make ns ctx pool per-device, so that it's deallocated properly on module
unload
* handle ctx allocation failure in ld_nvme_dobio()
* remove splbio() calls in ld_nvme_dobio() and sync, the paths are exercised
only for dump/shutdown, and that already disables interrupts
* free the ns ctx in ld_nvme_biodone() before calling lddone() to avoid
memory starvation, as lddone() can trigger another i/o request
* be more careful with using PR_WAITOK, the paths are called from interrupt
context and there we can't wait
possible to load the driver as module on those platforms (the weak symbols
are not found by module load, claiming they don't exist); this makes it possible
to load the driver on e.g. sparc64, which is supposed to be working
under OpenBSD
unfortunately QEMU sparc64 emulator starts causing data access errors
on first device register read in nvme_attach(), so can't confirm the driver
is actually working on sparc64; same happens in QEMU when booting OpenBSD
image, so it seems to be emulator bug
