The major changes are:
+ 4Gb (24XX) card support
+ Rewritten fabric and loop evaluation code
+ New f/w sets
The 4Gb changes required major rototilling, which caused a rewrite of
fabric and loop eval code. The latter can now be set up to tune for
dynamic device arrival/departure if the framework is set up for it,
or to be firm about waiting for devices.
Testing has been principally on amd64, i386 and sparc64 and seems to
not have broken things for me.
status from 10 seconds to 250 milli seconds for the second and subsequent
attempts to determine the link status. This prevents unnecessary long
stalls after the device attach routines have completed.
Tested on a dual port QLogic 3212. Approved by mjacob@feral.com.
it should be there, but in practice, on some systems an ugly race condition
rears its head: SCSI commands are issued before the FC thread ever runs, and
fail from then on forever. Yuck.
Fix the bug where the thread worker constantly looped. It was a race as
to whether or not the thread fired up or a polled/probed command happened
first, or maybe it's due to other scsipi changes, but we never set the
value that caused the fc thread worker to actually go to sleep until an
initial loop up.
This was rather ugly as it consumed nearly all CPU time available from thence
on. Bad.
to be trying to wriggle out of supporting this well. Instead, use
GID_FT to get a list of Port IDs and then use GPN_ID/GNN_ID to find the
port and node wwn. This should make working on fabrics a bit cleaner and
more stable.
This also caused some cleanup of SNS subcommand canonicalization so that
we can actually check for FS_ACC and FS_RJT, and if we get an FS_RJT,
print out the reason and explanation codes.
We'll keep the old GA_NXT method around if people want to uncomment a
controlling definition in ispvar.h.
This also had us clean up ISPASYNC_FABRICDEV to use a local lportdb argument
and to have the caller explicitly say that a device is at the end of the
fabric list.
Distinguish between 2312 and 2300 cards (they *are* different). Enable
RIO (Reduced Interrupt Operation) for the LVD cards (hey- I've seen
batched completions of the 30 commands at a time with this,....)...
If we get a Port Logout on local loop topologies, we have to force the
f/w to log back in. The easiest way (for us) to do this is to force
a LIP. This also will wake up the disk that probably just had a f/w crash.
Implement mailbox 'continuations'- this allows interrupts to re-drive
a mailbox command if it's one that just essentially repeats the previous
mailbox command (e.g., f/w download). This saves a boatload of sleep/wakeup
twitches.
If we're not a 2300 and we're about to return with a 'bogus interrupt'- check
the semaphore register to be non-zero at all and outgoing mailbox 0- this
seems to be where some of the lost ISP1080 commands came from.
the response queue. Instead of the ad hoc ISP_SWIZZLE_REQUEST, we now have
a complete set of inline functions in isp_inline.h. Each platform is
responsible for providing just one of a set of ISP_IOX_{GET,PUT}{8,16,32}
macros.
The reason this needs to be done is that we need to have a single set of
functions that will work correctly on multiple architectures for both little
and big endian machines. It also needs to work correctly in the case that
we have the request or response queues in memory that has to be treated
specially (e.g., have ddi_dma_sync called on it for Solaris after we update
it or before we read from it).
One thing that falls out of this is that we no longer build requests in the
request queue itself. Instead, we build the request locally (e.g., on the
stack) and then as part of the swizzling operation, copy it to the request
queue entry we've allocated. I thought long and hard about whether this was
too expensive a change to make as it in a lot of cases requires an extra
copy. On balance, the flexbility is worth it. With any luck, the entry that
we build locally stays in a processor writeback cache (after all, it's only
64 bytes) so that the cost of actually flushing it to the memory area that is
the shared queue with the PCI device is not all that expensive. We may examine
this again and try to get clever in the future to try and avoid copies.
Another change that falls out of this is that MEMORYBARRIER should be taken
a lot more seriously. The macro ISP_ADD_REQUEST does a MEMORYBARRIER on the
entry being added. But there had been many other places this had been missing.
It's now very important that it be done.
For NetBSD, it does a ddi_dmamap_sync as appropriate. This gets us out of
the explicit ddi_dmamap_sync on the whole response queue that we did for SBus
cards at each interrupt.
Set things up so that platforms that cannot have an SBus don't get a lot of
the SBus code checks (dead coded out).
Additional changes:
Fix a longstanding buglet of sorts. When we get an entry via isp_getrqentry,
the iptr value that gets returned is the value we intend to eventually plug
into the ISP registers as the entry *one past* the last one we've written-
*not* the current entry we're updating. All along we've been calling sync
functions on the wrong index value. Argh. The 'fix' here is to rename all
'iptr' variables as 'nxti' to remember that this is the 'next' pointer-
not the current pointer.
Devote a single bit to mboxbsy- and set aside bits for output mbox registers
that we need to pick up- we can have at least one command which does not
have any defined output registers (MBOX_EXECUTE_FIRMWARE).
Explicitly decode GetAllNext SNS Response back *as* a GetAllNext response.
Otherwise, we won't unswizzle it correctly.
Nuke some additional __P macros.
Temp work around problems where if we allow for non-polled mailbox commands
we got nailed by hardclock calling is for a timed thaw. Basically, this
means we only enable non-polled mailbox commands in the FC kthread when it
calls isp_fc_runstate.
in how interrupts are down- the 23XX has not only a different place to check
for an interrupt, but unlike all other QLogic cards, you have to read the
status as a 32 bit word- not 16 bit words. Rather than have device specific
functions as called from the core module (in isp_intr), it makes more sense
to have the platform/bus modules do the gruntwork of splitting out the
isr, semaphore register and the first outgoing mailbox register (if needed)
*prior* to calling isp_intr (if calling isp_intr is necessary at all).
way of doing business- modulo some startup spasms and peculiarities
of the way kthreads are started (*after* configuration, weird) and
some strangeness with the freeze/thaw code, what now happens is
that any of Loop Down, LIP, Loop Reset or Port Datbase or Name
Server Database Changed ASYNC events cause the queues to freeze
for this channel. The arrival of a Loop UP is not relevant.
What *is* relevant is that the Port Datbase or Name Server Changed
async event indicate that it's okay to go and (re)evaluate the
state of the FC link and (re)probe local loop and fabric membership.
We have a kthread do this because it's *sooooo* much nicer to be
able to sleep while doing the 130-250 mailbox commands it'll take
to re-evaluate things.
When the state is well known again, we can unfreeze the channel
queues. Then, as commands start arriving, we simply can start them
or bounce them with XS_SELTIMEOUT (if the device in question has
gone away). Previously, we did lazy evaluation, which meant that
if a change occurred, we would wait until the very *next* command
to go rebuild stuff.
The reason this is not sensible is:
a) Even with sleeping, you can hang up your system because you might be
making some poor stat(2) call pay the price of re-evaluating the whole
fabric.
b) If we ever really want to get to dynamic attachment/detachment, we
should find out sooner, rather than later, where things get to.
Split off ispminphys_1020 from ispminphys- a 1020 has a 24 bit limit-
not anything newer.
Re-enable LIPs and Loop Resets as async events- this allows the outer
layer to set policy about them.
Roll platform major && minor. Remove bogus waitq (no longer used).
Remove callout entry in softc (no longer used). Define some shorthands
for channels. Clean up a variety of cruft left over from the
thorpej_scsipi changeover.
This is a completely rewritten scsipi_xfer execution engine, and the
associated changes to HBA drivers. Overview of changes & features:
- All xfers are queued in the mid-layer, rather than doing so in an
ad-hoc fashion in individual adapter drivers.
- Adapter/channel resource management in the mid-layer, avoids even trying
to start running an xfer if the adapter/channel doesn't have the resources.
- Better communication between the mid-layer and the adapters.
- Asynchronous event notification mechanism from adapter to mid-layer and
peripherals.
- Better peripheral queue management: freeze/thaw, sorted requeueing during
recovery, etc.
- Clean separation of peripherals, adapters, and adapter channels (no more
scsipi_link).
- Kernel thread for each scsipi_channel makes error recovery much easier
(no more dealing with interrupt context when recovering from an error).
- Mid-layer support for tagged queueing: commands can have the tag type
set explicitly, tag IDs are allocated in the mid-layer (thus eliminating
the need to use buggy tag ID allocation schemes in many adapter drivers).
- support for QUEUE FULL and CHECK CONDITION status in mid-layer; the command
will be requeued, or a REQUEST SENSE will be sent as appropriate.
Just before the merge syssrc has been tagged with thorpej_scsipi_beforemerge
change I didn't take is the %llu format- I can't have a common
across multiple platform module assume a %ll argument capability-
which really pointed out that I shouldn't be trying to *print*
something which could long long.
quite simply a question of the Qlogic being little endian and having
to have stuff swapped on big endian machines- it also has to do with the
fact that the SBus and PCI DMA layouts are wierd with respect to this.
At any rate, now finally fixed- works on Mac G4, tested it on a SS10
for sparc, checked on alpha to see if I've broken anything, and as
soon as I get another spare afternoon I'll finally install a sparc64
version which should just work (as it'll be like the Mac).
interace cleanups, some new common functions. The major impact that
will be noticeable right away is that if you boot with not Fibre connected
to the FC cards, you no longer hang indefinitely.
