hcds which queue up more then one packet at once (uhci, ehci and xhci),
must clear the queue after an error which has caused the queue to halt.
Currently this is handled as a special case inside the hcd code, this
patch instead adds an USB_RET_REMOVE_FROM_QUEUE packet result code, teaches
the 3 hcds about this and moves the clearing of the queue on a halt into
the USB core.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This can be used by usb-device code which wishes to process an entire endpoint
queue at once, to do this the usb-device code returns USB_RET_ADD_TO_QUEUE
from its handle_data class method and defines a flush_ep_queue class method
to call when the hcd is done queuing up packets.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
If the guest is using multiple transfers to try and keep the usb bus busy /
used at maximum efficiency, currently we would see / do the following:
1) submit transfer 1 to the device
2) submit transfer 2 to the device
3) report transfer 1 completion to guest
4) report transfer 2 completion to guest
5) submit transfer 1 to the device
6) report transfer 1 completion to guest
7) submit transfer 2 to the device
8) report transfer 2 completion to guest
etc.
So after the initial submission we would effectively only have 1 transfer
in flight, rather then 2. This is caused by us not checking the queue for
addition of new transfers by the guest (ie the resubmission of a recently
finished transfer), while waiting for a pending transfer to complete.
This patch does add a check for this, changing the sequence to:
1) submit transfer 1 to the device
2) submit transfer 2 to the device
3) report transfer 1 completion to guest
4) submit transfer 1 to the device
5) report transfer 2 completion to guest
6) submit transfer 2 to the device
etc.
Thus keeping 2 transfers in flight (most of the time, and always 1),
as intended by the guest.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
For ctrl endpoints Windows (atleast Win7) creates circular td lists, so far
these were not a problem because we would stop filling the queue if altnext
was set. Since further patches in this patchset remove the altnext check this
does become a problem and we need detection for going in circles.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Often the guest will queue up new packets in response to a packet, in the
async schedule with its IOC flag set, completing. By speeding up the
frame-timer, we notice these new packets earlier. This increases the
speed (MB/s) of a Linux guest reading from a USB mass storage device by a
factor of 1.15 on top of the "Improve latency of interrupt delivery"
speed-ups, both with and without input pipelining enabled.
I've not tested the speed-up of this patch without the
"Improve latency of interrupt delivery" patch.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
While doing various performance tests of reading from USB mass storage devices
I noticed the following::
1) When an async handled packet completes, we don't immediately report an
interrupt to the guest, instead we wait for the frame-timer to run and
report it from there
2) If 1) has been fixed and an async handled packet takes a while to complete,
then async_stepdown will become a high value, which means that there
will be a large latency before any new packets queued by the guest in
response to the interrupt get seen
1) was done deliberately as part of commit f0ad01f92:
http://www.kraxel.org/cgit/qemu/commit/?h=usb.57&id=f0ad01f92ca02eee7cadbfd225c5de753ebd5fce
Since setting the interrupt immediately on async packet completion was causing
issues with Linux guests, I believe this recently fixed Linux bug explains
why this is happening:
http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=361aabf395e4a23cf554cf4ec0c0c6963b8beb01
Note that we can *not* count on this fix being present in all Linux guests!
I was hoping that the recently added support for Interrupt Threshold Control
would fix the issues with Linux guests, but adding a simple ehci_commit_irq()
call to ehci_async_bh() still caused problems with Linux guests.
The problem is, that when doing ehci_commit_irq() from ehci_async_bh(),
the "old" frindex value is used to calculate usbsts_frindex, and when
the frame-timer then runs possibly very shortly after ehci_async_bh(),
it increases the frame-timer, and thus any interrupts raised from that
frame-timer run, will also get reported to the guest immediately, rather
then being delayed to the next frame-timer run.
Luckily the solution for this is simple, this means that we need to
increase frindex before calling ehci_commit_irq() from ehci_async_bh(),
which in the end boils down to simple calling ehci_frame_timer() instead
of ehci_async_bh() from the bh.
This may seem like it causes a lot of extra work to be done, but this
is not true. Any work done from the frame-timer processing the periodic
schedule is work which then does not need to be done the next time the
frame timer runs, also the frame-timer will re-arm itself at (possibly)
a later time then it was armed for saving a vmexit at that time.
As an additional advantage moving to simply calling the frame-timer also
fixes 2) as the packet completion will set async_stepdown to 0, and the
re-arming of the timer with an async_stepdown of 0 ensures that any
newly queued up packets get seen in a reasonable amount of time.
This improves the speed (MB/s) of a Linux guest reading from a USB mass
storage device by a factor of 1.5 - 1.7 with input pipelining disabled,
and by a factor of 1.8 with input pipelining enabled.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
According to 4.15.1.2 an interrupt must be raised when a short packet
is received. If we don't do this it may take a significant time for
the guest to notice a short trasnfer has completed, since only the last td
will have its IOC flag set, and a short transfer may complete in an earlier
packet.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This field is used in some places to track the tbytes field of the token, but
in other places the field is used directly, use it directly everywhere for
consistency.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Rather then having a special check to start queuing after the first packet,
and then another check for the other packets in uhci_fill_queue(), simply
check the previous packet beforehand in uhci_fill_queue()
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Packets with an invalid pid, or which were cancelled have
usb_packet_map() called on them on init, but not usb_packet_unmap()
before being freed.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
target_phys_addr_t is unwieldly, violates the C standard (_t suffixes are
reserved) and its purpose doesn't match the name (most target_phys_addr_t
addresses are not target specific). Replace it with a finger-friendly,
standards conformant hwaddr.
Outstanding patchsets can be fixed up with the command
git rebase -i --exec 'find -name "*.[ch]"
| xargs s/target_phys_addr_t/hwaddr/g' origin
Signed-off-by: Avi Kivity <avi@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
According to the spec we must raise an interrupt when one is requested
even for non active tds.
Linux depends on this, for bulk transfers it runs an inactivity timer
to work around a bug in early uhci revisions, when we take longer then
200 ms to process a packet, this timer goes of, and as part of the
handling Linux then unlinks the qh, and relinks it after the frindex
has increased by atleast 1, the problem is Linux only checks for the
frindex increases on an interrupt, and we don't send that, causing
the qh to go inactive for more then 32 frames, at which point we
consider the packet cancelled.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Instead simple disconnect the device like host redirection does on
migration.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
* 'trivial-patches' of git://github.com/stefanha/qemu:
versatilepb: Use symbolic indices for ARM PIC
qdev: kill bogus comment
qemu-barrier: Fix compiler version check for future gcc versions
hw: Add missing 'static' attribute for QEMUMachine
cleanup useless return sentence
qemu-sockets: Fix compiler warning (regression for MinGW)
vnc: Fix spelling (hellmen -> hellman) in comment
slirp: Fix spelling in comment (enought -> enough, insure -> ensure)
tcg/arm: Use tcg_out_mov_reg rather than inline equivalent code
cpu: Add missing 'static' attribute to qemu_global_mutex
configure: Support empty target list (--target-list=)
hw: Fix return value check for bdrv_read, bdrv_write
The entries for libhw* are no longer needed in .gitignore.
There is also no longer a difference between common-obj-y and
hw-obj-y, so one of those two macros is sufficient.
Signed-off-by: Stefan Weil <sw@weilnetz.de>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
This patch cleans up return sentences in the end of void functions.
Reported-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Amos Kong <akong@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@gmail.com>
With the IOMMU infrastructure introduced before 1.2, we need to use
dma_memory_map() to obtain a qemu pointer to memory from an IO bus address.
However, dma_memory_map() alters the given length to reflect the length
over which the used DMA translation is valid - which could be either more
or less than the requested length.
usb_packet_map() does not correctly handle these cases, simply failing if
dma_memory_map() alters the requested length. If dma_memory_map()
increased the length, we just need to use the requested length for the
qemu_iovec_add(). However, if it decreased the length, it means that a
single DMA translation is not valid for the whole sglist element, and so
we need to loop, splitting it up into multiple iovec entries for each
piece with a DMA translation (in practice >2 pieces is unlikely).
This patch implements the correct behaviour
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
There are several issues with our handling of the MULT epcap field
of interrupt qhs, which this patch fixes.
1) When we don't execute a transaction because of the transaction counter
being 0, p->async stays EHCI_ASYNC_NONE, and the next time we process the
same qtd we hit an assert in ehci_state_fetchqtd because of this. Even though
I believe that this is caused by 3 below, this patch still removes the assert,
as that can still happen without 3, when multiple packets are queued for the
same interrupt ep.
2) We only *check* the transaction counter from ehci_state_execute, any
packets queued up by fill_queue bypass this check. This is fixed by not calling
fill_queue for interrupt packets.
3) Some versions of Windows set the MULT field of the qh to 0, which is a
clear violation of the EHCI spec, but still they do it. This means that we
will never execute a qtd for these, making interrupt ep-s on USB-2 devices
not work, and after recent changes, triggering 1).
So far we've stored the transaction counter in our copy of the mult field,
but with this beginnig at 0 already when dealing with these version of windows
this won't work. So this patch adds a transact_ctr field to our qh struct,
and sets this to the MULT field value on fetchqh. When the MULT field value
is 0, we set it to 4. Assuming that windows gets way with setting it to 0,
by the actual hardware going horizontal on a 1 -> 0 transition, which will
give it 4 transactions (MULT goes from 0 - 3).
Note that we cannot stop on detecting the 1 -> 0 transition, as our decrement
of the transaction counter, and checking for it are done in 2 different places.
Reported-by: Shawn Starr <shawn.starr@rogers.com>
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Set maxports to 15. This is what the usb3 route string can handle.
Set maxslots to 64. This is more than the number of root ports we
can have, but with additional hubs you can end up with more devices.
Set maxintrs (aka msi vectors) to 16. Should be enougth, especially
considering that vectors are a limited ressource. Linux guests use
only three at the moment.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
* 'usb.65' of git://git.kraxel.org/qemu:
uhci: Don't queue up packets after one with the SPD flag set
usb-redir: Revert usb-redir part of commit 93bfef4c
usb-redir: Add chardev open / close debug logging
usb-redir: Add support for migration
usb-redir: Store max_packet_size in endp_data
usb-redir: Add an already_in_flight packet-id queue
usb-redir: Change cancelled packet code into a generic packet-id queue
ehci: Walk async schedule before and after migration
ehci: Don't set seen to 0 when removing unseen queue-heads
configure: usbredir fixes
ehci: Don't process too much frames in 1 timer tick (v2)
ehci: Fix interrupts stopping when Interrupt Threshold Control is 8
ehci: switch to new-style memory ops
usb-host: allow emulated (non-async) control requests without USBPacket
The USB network interface has a single buffer which the guest reads
from. This patch prevents multiple calls to usbnet_receive() from
clobbering the input buffer. Instead we queue packets until buffer
space becomes available again.
This is inspired by virtio-net and e1000 rxbuf handling.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
The USB network interface has two code paths depending on whether or not
RNDIS mode is enabled. Refactor usbnet_receive() so that there is a
common path throughout the function instead of duplicating everything
across if (is_rndis(s)) ... else ... code paths.
Clean up coding style and 80 character line wrap along the way.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Don't queue up packets after a packet with the SPD (short packet detect)
flag set. Since we won't know if the packet will actually be short until it
has completed, and if it is short we should stop the queue.
This fixes a miniature photoframe emulating a USB cdrom with the windows
software for it not working.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit 93bfef4c6e makes qemu-devices
which report the qemu version string to the guest in some way use a
qemu_get_version function which reports a machine-specific version string.
However usb-redir does not expose the qemu version to the guest, only to
the usbredir-host as part of the initial handshake. This can then be logged
on the usbredir-host side for debugging purposes and is otherwise completely
unused! For debugging purposes it is important to have the real qemu version
in there, rather then the machine-specific version.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
So that we've a place to migrate it to / from to allow restoring it after
migration.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
After a live migration, the usb-hcd will re-queue all packets by
walking over the schedule in the guest memory again, but requests which
were encountered on the migration source before will already be in flight,
so these should *not* be re-send to the usbredir-host.
This patch adds an already in flight packet ud queue, which will be filled by
the source before migration and then moved over to the migration dest, any
async handled packets are then checked against this queue to avoid sending
the same packet to the usbredir-host twice.
Signed-off-by: Hans de Goede <hdegoede@redhat,com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
When removing unseen queue-heads from the async queue list, we should not
set the seen flag to 0, as this may cause them to be removed by
ehci_queues_rip_unused() during the next call to ehci_advance_async_state()
if the timer is late or running at a low frequency.
Note:
1) This *may* have caused the instant unlink / relinks described in commit
9bc3a3a216
2) Rather then putting more if-s inside ehci_queues_rip_unused, this patch
instead introduces a new ehci_queues_rip_unseen function.
3) This patch also makes it save to call ehci_queues_rip_unseen() multiple
times, which gets used in the folluw up patch titled:
"ehci: Walk async schedule before and after migration"
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
The Linux ehci isoc scheduling code fills the entire schedule ahead of
time minus 80 frames. If we make a large jump in where we are in the
schedule, ie 40 frames, then the scheduler all of a sudden will only have
40 frames left to work in, causing it to fail packet submissions
with error -27 (-EFBIG).
Changes in v2:
-Don't hardcode a maximum number of frames to process in one tick, instead:
-Process a minimum number of frames to ensure we do eventually catch up
-Stop (after the minimum number) when the guest has requested an irq
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
If Interrupt Threshold Control is 8 or a multiple of 8, then
s->usbsts_frindex can become exactly 0x4000, at which point
(s->usbsts_frindex > s->frindex) will never become true, as
s->usbsts_frindex will not be lowered / reset in this case.
This patch fixes this.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Also register different memory regions for capabilities,
operational registers and port status registers. Create
separate tracepoints for operational regs and port status
regs. Ditch a bunch of sanity checks because the memory
core will do this for us now.
Offloading the byte, word and dword access handling to the
memory core also has the side effect of fixing ehci register
access on bigendian hosts.
Cc: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
xhci needs this for USB_REQ_SET_ADDRESS due to the way
usb addressing is handled by the xhci hardware.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
... and register subregions instead, so we offload the dispatching
to the the memory subsystem which is designed to handle it.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Move all state belonging to the (single) interrupter into a separate
struct. First step in adding support for multiple interrupters.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Split xhci_irq_update into a function which handles intx updates
(including lowering the irq line once the guests acks the interrupt)
and one which is used for raising an irq only.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Drop custom write_config function which isn't needed any more.
Make the msi property a bit property so it accepts 'on' & 'off'.
Enable MSI by default.
TODO: add compat property to disable on old machine types.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Add support for building superspeed endpoint companion descriptors,
create them for superspeed usb devices.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This patch changes the way xhci ports are linked to USBPorts. The fixed
1:1 relationship between xhci ports and USBPorts is gone. Now each
USBPort represents a physical plug which has usually two xhci ports
assigned: one usb2 and ond usb3 port. usb devices show up at one or the
other, depending on whenever they support superspeed or not.
This patch also makes the number of usb2 and usb3 ports runtime
configurable by adding 'p2' and 'p3' properties. It is allowed to
have different numbers of usb2 and usb3 ports. Specifying p2=4,p3=2
will give you an xhci adapter which supports all speeds on physical
ports 1+2 and usb2 only on ports 3+4.
Change the register layout to be a bit more sparse and also not depend
on the number of ports. Useful when for making the number of ports
runtime-configurable.
This patch splits the xhci_xfer_data function into three.
The xhci_xfer_data function used to do does two things:
(1) copy transfer data between guest memory and a temporary buffer.
(2) report transfer results to the guest using events.
Now we three functions to handle this:
(1) xhci_xfer_map creates a scatter list for the transfer and
uses that (instead of the temporary buffer) to build a
USBPacket.
(2) xhci_xfer_unmap undoes the mapping.
(3) xhci_xfer_report sends out events.
The patch also fixes reporting of transaction errors which must be
reported unconditinally, not only in case the guest asks for it
using the ISP flag.
[ v2: fix warning ]
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
original xhci code (the one which used libusb directly) used to use
'background transfers' for iso streams. In upstream qemu the iso
stream buffering is handled by usb-host & usb-redir, so we will
never ever need this. It has been left in as reference, but is dead
code anyway. Rip it out.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Windows users need to know that they have to use the Baum driver to make
the qemu braille device work.
Signed-off-by: Samuel Thibault <samuel.thibault@ens-lyon.org>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
In order for redirection to work properly when redirecting to an emulated
XHCI controller, the usb-redir-host must support both
usb_redir_cap_ep_info_max_packet_size and usb_redir_cap_64bits_ids,
reject any devices redirected to an XHCI controller when these are not
supported.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This is needed for usb-redir to work properly with the xhci emulation.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This gives us support for 64 bit ids which is needed for using XHCI with
the new hcd generated ids.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Babble is the appropriate error in this case (rather then signalling a stall).
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This is a preparation patch for completely getting rid of the async-packet
struct in usb-redir, instead relying on the (new) per ep queues in the
qemu usb core.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
The shadow copy only serves as an extra check (besides the packet-id) to
ensure the packet we get back is a reply to the packet we think it is.
This check has never triggered in all the time usb-redir is in use now,
and since the verified data in the returned packet-header is not used
otherwise, removing the check does not open any possibilities for the
usbredirhost to confuse us.
This is a preparation patch for completely getting rid of the async-packet
struct in usb-redir, instead relying on the (new) per ep queues in the
qemu usb core.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This is a preparation patch for completely getting rid of the async-packet
struct in usb-redir, instead relying on the (new) per ep queues in the
qemu usb core.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
There is no need for this, and doing so means that a backend trying to
write immediately after an open event will see qemu_chr_be_can_write
returning 0, which not all backends handle well as there is no wakeup
mechanism to detect when the frontend does become writable.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
USB_RET_NAK is not a valid response for async handled packets (and will
trigger an assert as such).
Also drop the warning when receiving a status of cancelled for packets not
cancelled by qemu itself, this can happen when a device gets unredirected
by the usbredir-host while transfers are pending.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Since my previous comment said "Should never happen", I tried changing the
next line to an assert(0), which did not go well, which as the new comments
explains is logical if you think about it for a moment.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
USB_RET_PROCERR can be triggered by the guest (by for example requesting more
then BUFFSIZE bytes), so don't assert on it.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Currently each time we try to execute a NAK-ed packet we redo
ehci_init_transfer, and usb_packet_map, re-allocing (without freeing) the
sg list every time.
This patch fixes this, it does this by introducing another async state, so
that we also properly cleanup a NAK-ed packet on cancel.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
make qemu_queue_{cancel,reset} return the number of packets released,
so the caller can figure whenever there have been active packets even
though there shouldn't have been any. Add tracepoint to log this.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Reported packets which have completed before being cancelled as such to the
host. Note that the new code path this patch adds is untested since it I've
been unable to actually trigger the race which needs this code path.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
-combine the qh check with the check for devaddr changes
-also ensure that p gets set to NULL when the queue gets cancelled on
devaddr change, which was not done properly before this patch
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
This reverts commit 9bc3a3a216, which got
added to fix an issue where the real, underlying cause was not stopping
the ep queue on an error.
Now that the underlying cause is fixed by the "usb: Halt ep queue and
cancel pending packets on a packet error" patch, the "don't flush" fix
is no longer needed.
Not only is it not needed, it causes us to see cancellations (unlinks)
done by the Linux EHCI driver too late, which in combination with the new
usb-core packet-id generation where qtd addresses are used as ids, causes
duplicate ids for in flight packets.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
This can happen with usb-redir live-migration when the packet gets re-queued
after the migration and the original queuing from the migration source side
has already finished.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This way the hcd can re-use the same packet to retry without needing
to re-init it.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
If an (emulated) usb-device tries to write more data to a packet then
its iov len, this will trigger an assert in usb_packet_copy(), and if
a driver somehow circumvents that check and writes more data to the
iov then there is space, we have a much bigger problem then not correctly
reporting babble to the guest.
In practice babble will only happen with (real) redirected devices, and there
both the usb-host os and the qemu usb-device code already check for it.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Kick next scsi transfer from request release callback instead of command
completion callback, otherwise we might get stuck in case scsi_req_unref()
doesn't release the request instantly due to someone else holding a
reference too.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
One of the recent changes (likely the addition of queuing support) has broken
interrupt endpoints, this patch fixes this.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
ehci_state_executing does not need to check for p->usb_status == USB_RET_ASYNC
or USB_RET_PROCERR, since ehci_execute_complete already does a similar check
and will trigger an assert if either value is encountered.
USB_RET_ASYNC should never be the packet status when execute_complete runs
for obvious reasons, and USB_RET_PROCERR is only used by ehci_state_execute /
ehci_execute not by ehci_state_executing / ehci_execute_complete.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
ehci_qh_do_overlay() already calls ehci_flush_qh() before it returns, calling
it twice is useless.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
After the "ehci: Print a warning when a queue unexpectedly contains packets
on cancel" commit. Under certain reproducable conditions I was getting the
following message: "EHCI: Warning queue not empty on queue reset".
After aprox. 8 hours of debugging I've finally found the cause. The Linux EHCI
driver has an IAAD watchdog, to work around certain EHCI hardware sometimes
not acknowledging the doorbell at all. This watchdog has a timeout of 10 ms,
which is less then the time between 2 runs through the async schedule when
async_stepdown is at its highest value.
Thus the watchdog can trigger, after which Linux clears the IAAD bit and
re-uses the QH. IOW we were not properly detecting the unlink of the qh, due
to us missing (ignoring for more then 10 ms) the IAAD command, which triggered
the warning.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
This patch adds IDs to usb packets. Those IDs are (a) supposed to be
unique for the lifecycle of a packet (from packet setup until the packet
is either completed or canceled) and (b) stable across migration.
uhci, ohci, ehci and xhci use the guest physical address of the transfer
descriptor for this.
musb needs a different approach because there is no transfer descriptor.
But musb also doesn't support pipelining, so we have never more than one
packet per endpoint in flight. So we go create an ID based on endpoint
and device address.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
For controllers which queue up more then 1 packet at a time, we must halt the
ep queue, and inside the controller code cancel all pending packets on an
error.
There are multiple reasons for this:
1) Guests expect the controllers to halt ep queues on error, so that they
get the opportunity to cancel transfers which the scheduled after the failing
one, before processing continues
2) Not cancelling queued up packets after a failed transfer also messes up
the controller state machine, in the case of EHCI causing the following
assert to trigger: "assert(p->qtdaddr == q->qtdaddr)" at hcd-ehci.c:2075
3) For bulk endpoints with pipelining enabled (redirection to a real USB
device), we must cancel all the transfers after this a failed one so that:
a) If they've completed already, they are not processed further causing more
stalls to be reported, originating from the same failed transfer
b) If still in flight, they are cancelled before the guest does
a clear stall, otherwise the guest and device can loose sync!
Note this patch only touches the ehci and uhci controller changes, since AFAIK
no other controllers actually queue up multiple transfer. If I'm wrong on this
other controllers need to be updated too!
Also note that this patch was heavily tested with the ehci code, where I had
a reproducer for a device causing a transfer to fail. The uhci code is not
tested with actually failing transfers and could do with a thorough review!
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
This fixes linux guests started without any USB devices not seeing newly
plugged devices until "lsusb" is done inside the guest.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
First, not all interrupts are subject to Interrupt Threshold Control,
some of them must be delivered without delay.
Second, Interrupt Threshold Control state must be part of vmstate,
otherwise we might loose IRQs on migration.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Pick other product id to fix clash with audio.
Current usage list (after applying this patch):
46f4:0001 -- usb-storage
46f4:0002 -- usb-audio
46f4:0003 -- usb-uas
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Need to clear p->result after copying setup data using usb_packet_copy()
because we'll reuse the USBPacket for the data transfer.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit 5931065907 is incomplete,
we'll arrive in the scsi command complete callback in CSW state
and must handle that case correctly.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Another step in moving the vlan feature out of net core. Users only
deal with NetClientState and therefore qemu_del_vlan_client() should be
named qemu_del_net_client().
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
The vlan feature is no longer part of net core. Rename VLANClientState
to NetClientState because net clients are not explicitly associated with
a vlan at all, instead they have a peer net client to which they are
connected.
This patch is a mechanical search-and-replace except for a few
whitespace fixups where changing VLANClientState to NetClientState
misaligned whitespace.
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
Signed-off-by: Zhi Yong Wu <wuzhy@linux.vnet.ibm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
* kwolf/for-anthony: (41 commits)
fdc-test: Clean up a bit
fdc-test: introduce test_relative_seek
fdc: fix relative seek
qemu-iotests: Valgrind support
coroutine-ucontext: Help valgrind understand coroutines
qemu-io: Fix memory leaks
hw/block-common: Factor out fall back to legacy -drive cyls=...
blockdev: Don't limit DriveInfo serial to 20 characters
hw/block-common: Factor out fall back to legacy -drive serial=...
hw/block-common: Move BlockConf & friends from block.h
Relax IDE CHS limits from 16383,16,63 to 65535,16,255
blockdev: Drop redundant CHS validation for if=ide
hd-geometry: Compute BIOS CHS translation in one place
qtest: Test we don't put hard disk info into CMOS for a CD-ROM
ide pc: Put hard disk info into CMOS only for hard disks
block: Geometry and translation hints are now useless, purge them
qtest: Cover qdev property for BIOS CHS translation
ide: qdev property for BIOS CHS translation
qdev: New property type chs-translation
qdev: Collect private helpers in one place
...
Move down the expire time calculation down in the frame timer, to the
point where the timer is actually reloaded. This way we'll notice any
async_stepdown changes (especially resetting to 0 due to usb activity).
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
With the async schedule being kicked from other places than the frame
timer (commit 0f588df8b3) it may happen
that we call ehci_commit_interrupt() more than once per frame.
Move the call from the async schedule handler to the frame timer to
restore old irq behavior, which is more correct. Fixes regressions
with some linux kernel versions.
TODO: implement full Interrupt Threshold Control support.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
$subject says all: when loading old (v1) vmstate which doesn't contain
expire_time initialize it with a reasonable default (current time).
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
$subject says all. First cut.
It's a pure UAS (usb attached scsi) emulation, without BOT (bulk-only
transport) compatibility. If your guest can't handle it use usb-storage
instead.
The emulation works like any other scsi hba emulation (eps, lsi, virtio,
megasas, ...). It provides just the HBA where you can attach scsi
devices as you like using '-device'. A single scsi target with up to
256 luns is supported.
For now only usb 2.0 transport is supported. This will change in the
future though as I plan to use this as playground when codeing up &
testing usb 3.0 transport and streams support in the qemu usb core and
the xhci emulation.
No migration support yet. I'm planning to add usb 3.0 support first as
this probably requires saving additional state.
Special thanks go to Paolo for bringing the qemu scsi emulation into
shape, so this can be added nicely without having to touch a single line
of scsi code.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
* mjt/mjt-iov2:
rewrite iov_send_recv() and move it to iov.c
cleanup qemu_co_sendv(), qemu_co_recvv() and friends
export iov_send_recv() and use it in iov_send() and iov_recv()
rename qemu_sendv to iov_send, change proto and move declarations to iov.h
change qemu_iovec_to_buf() to match other to,from_buf functions
consolidate qemu_iovec_copy() and qemu_iovec_concat() and make them consistent
allow qemu_iovec_from_buffer() to specify offset from which to start copying
consolidate qemu_iovec_memset{,_skip}() into single function and use existing iov_memset()
rewrite iov_* functions
change iov_* function prototypes to be more appropriate
virtio-serial-bus: use correct lengths in control_out() message
Conflicts:
tests/Makefile
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Replace iso transfer fprintf's with trace points. Also rename existing
tracepoints so they all match usb_host_iso_*.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Create a new usb_ep_reset() function to reset endpoint state, without
re-initialiting the queues, so we don't unlink in-flight packets just
because usb-host has to re-parse the descriptor tables.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit 0f588df8b3, added code
to ehci_wakeup to kick the async schedule on wakeup, but the else
was positioned wrong making it trigger for devices which are routed
to the companion rather then to the ehci controller itself.
This patch fixes this. Note that the "programming style" with using the
return at the end of the companion block matches how the companion case
is handled in the other ports ops, and is done this way for consistency.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
hcd-ehci.c is missing an usb_packet_init() call for the ipacket UsbPacket
it uses for isoc transfers, triggering an assert (taking the entire vm down)
in usb_packet_setup as soon as any isoc transfers are done by a high speed
USB device.
Signed-off-by: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Commit 4be23939ab makes ehci instantly
zap any unlinked queue heads when the guest rings the doorbell.
While hacking up uas support this turned out to be a problem. The linux
kernel can unlink and instantly relink the very same queue head, thereby
killing any async packets in flight. That alone isn't an issue yet, the
packet will canceled and resubmitted and everything is fine. We'll run
into trouble though in case the async packet is completed already, so we
can't cancel it any more. The transaction is simply lost then.
usb_ehci_qh_ptrs q (nil) - QH @ 39c4f000: next 39c4f122 qtds 00000000,00000001,39c50000
usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0
usb_ehci_qh_ptrs q 0x7f95feba90a0 - QH @ 39c4f000: next 39c4f122 qtds 00000000,00000001,39c50000
usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0
usb_ehci_qh_ptrs q 0x7f95fe515210 - QH @ 39c4f120: next 39c4f0c2 qtds 29dbce40,29dbc4e0,00000009
usb_ehci_qh_fields QH @ 39c4f120 - rl 4, mplen 512, eps 2, ep 1, dev 2
usb_ehci_packet_action q 0x7f95fe515210 p 0x7f95fdec32a0: alloc
usb_packet_state_change bus 0, port 2, ep 1, packet 0x7f95fdec32e0, state undef -> setup
usb_ehci_packet_action q 0x7f95fe515210 p 0x7f95fdec32a0: process
usb_uas_command dev 2, tag 0x2, lun 0, lun64 00000000-00000000
scsi_req_parsed target 0 lun 0 tag 2 command 42 dir 2 length 16384
scsi_req_parsed_lba target 0 lun 0 tag 2 command 42 lba 5933312
scsi_req_alloc target 0 lun 0 tag 2
scsi_req_continue target 0 lun 0 tag 2
scsi_req_data target 0 lun 0 tag 2 len 16384
usb_uas_scsi_data dev 2, tag 0x2, bytes 16384
usb_uas_write_ready dev 2, tag 0x2
usb_packet_state_change bus 0, port 2, ep 1, packet 0x7f95fdec32e0, state setup -> complete
usb_ehci_packet_action q 0x7f95fe515210 p 0x7f95fdec32a0: free
usb_ehci_qh_ptrs q 0x7f95fdec3210 - QH @ 39c4f0c0: next 39c4f002 qtds 29dbce40,00000001,00000009
usb_ehci_qh_fields QH @ 39c4f0c0 - rl 4, mplen 512, eps 2, ep 2, dev 2
usb_ehci_queue_action q 0x7f95fe5152a0: free
usb_packet_state_change bus 0, port 2, ep 2, packet 0x7f95feba9170, state async -> complete
^^^ async packets completes.
usb_ehci_packet_action q 0x7f95fdec3210 p 0x7f95feba9130: wakeup
usb_ehci_qh_ptrs q (nil) - QH @ 39c4f000: next 39c4f122 qtds 00000000,00000001,39c50000
usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0
usb_ehci_qh_ptrs q 0x7f95feba90a0 - QH @ 39c4f000: next 39c4f122 qtds 00000000,00000001,39c50000
usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0
usb_ehci_qh_ptrs q 0x7f95fe515210 - QH @ 39c4f120: next 39c4f002 qtds 29dbc4e0,29dbc8a0,00000009
usb_ehci_qh_fields QH @ 39c4f120 - rl 4, mplen 512, eps 2, ep 1, dev 2
usb_ehci_queue_action q 0x7f95fdec3210: free
usb_ehci_packet_action q 0x7f95fdec3210 p 0x7f95feba9130: free
^^^ endpoint #2 queue head removed from schedule, doorbell makes ehci zap the queue,
the (completed) usb packet is freed too and gets lost.
usb_ehci_qh_ptrs q (nil) - QH @ 39c4f000: next 39c4f0c2 qtds 00000000,00000001,39c50000
usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0
usb_ehci_qh_ptrs q 0x7f95feba90a0 - QH @ 39c4f000: next 39c4f0c2 qtds 00000000,00000001,39c50000
usb_ehci_qh_fields QH @ 39c4f000 - rl 0, mplen 0, eps 0, ep 0, dev 0
usb_ehci_queue_action q 0x7f9600dff570: alloc
usb_ehci_qh_ptrs q 0x7f9600dff570 - QH @ 39c4f0c0: next 39c4f122 qtds 29dbce40,00000001,00000009
usb_ehci_qh_fields QH @ 39c4f0c0 - rl 4, mplen 512, eps 2, ep 2, dev 2
usb_ehci_packet_action q 0x7f9600dff570 p 0x7f95feba9130: alloc
usb_packet_state_change bus 0, port 2, ep 2, packet 0x7f95feba9170, state undef -> setup
usb_ehci_packet_action q 0x7f9600dff570 p 0x7f95feba9130: process
usb_packet_state_change bus 0, port 2, ep 2, packet 0x7f95feba9170, state setup -> async
usb_ehci_packet_action q 0x7f9600dff570 p 0x7f95feba9130: async
^^^ linux kernel relinked the queue head, ehci creates a new usb packet,
but we should have delivered the completed one instead.
usb_ehci_qh_ptrs q 0x7f95fe515210 - QH @ 39c4f120: next 39c4f002 qtds 29dbc4e0,29dbc8a0,00000009
usb_ehci_qh_fields QH @ 39c4f120 - rl 4, mplen 512, eps 2, ep 1, dev 2
So instead of instantly zapping the queue we'll set a flag that the
queue needs revalidation in case we'll see it again in the schedule.
ehci then checks that the queue head fields addressing / describing the
endpoint and the qtd pointer match the cached content before reusing it.
Cc: Hans de Goede <hdegoede@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Only write back the dwords the hc is supposed to update. Should not
make a difference in theory as the guest must not touch the td while
it is active to avoid races. But it is still more correct.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
Not a single driver has any possibility of failure on their
exit function, let's keep it that way.
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
The USB UHCI and EHCI drivers were converted some time ago to use the
pci_dma_*() helper functions. However, this conversion was not complete
because in some places both these drivers do DMA via the usb_packet_map()
function in usb-libhw.c. That function directly used
cpu_physical_memory_map().
Now that the sglist code uses DMA wrappers properly, we can convert the
functions in usb-libhw.c, thus conpleting the conversion of UHCI and EHCI
to use the DMA wrappers.
Note that usb_packet_map() invokes dma_memory_map() with a NULL invalidate
callback function. When IOMMU support is added, this will mean that
usb_packet_map() and the corresponding usb_packet_unmap() must be called in
close proximity without dropping the qemu device lock - otherwise the guest
might invalidate IOMMU mappings while they are still in use by the device
code.
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
The OHCI device emulation can provide both PCI and SysBus OHCI
implementations. Because of this, it was not previously converted to
use the PCI DMA helper functions.
This patch converts it to use the new universal DMA helper functions.
In the PCI case, it obtains its DMAContext from pci_dma_context(), in
the SysBus case, it uses NULL - i.e. assumes for now that there will
be no IOMMU translation for a SysBus OHCI.
Cc: Gerd Hoffmann <kraxel@redhat.com>
Cc: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: David Gibson <david@gibson.dropbear.id.au>
Signed-off-by: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
We update the QTAILQ in the loop, thus we must use the SAFE version
to make sure we don't touch the queue struct after freeing it.
https://bugzilla.novell.com/show_bug.cgi?id=766310
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
QEMU exposes its version to the guest's hardware and in some cases that is wrong
(e.g. Windows prints messages about driver updates when you switch
the QEMU version).
There is a new field now on the struct QEmuMachine, hw_version, which may
contain the version that the specific machine should report. If that field is
set, then that machine will report that version to the guest.
Signed-off-by: Crístian Viana <vianac@linux.vnet.ibm.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
This is far less interesting than it sounds. We simply add an Object to each
BusState and then register the types appropriately. Most of the interesting
refactoring will follow in the next patches.
Since we're changing fundamental type names (BusInfo -> BusClass), it all needs
to convert at once. Fortunately, not a lot of code is affected.
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
[AF: Made all new bus TypeInfos static const.]
[AF: Made qbus_free() call object_delete(), required {qom,glib}_allocated]
Signed-off-by: Andreas Färber <afaerber@suse.de>
This makes it easier to remove it from BusInfo.
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
[AF: Drop now unnecessary NULL initialization in scsibus_get_dev_path()]
Signed-off-by: Andreas Färber <afaerber@suse.de>
In qdev, each bus in practice identified an abstract superclass, but
this was mostly hidden. In QOM, instead, these abstract classes are
explicit so we can move bus properties there.
All bus property walks are removed, and all device property walks
are changed to look along the class hierarchy instead.
We would have duplicates if class A defines some properties and its
subclass B does not define any, because class_b->props will be
left equal to class_a->props.
The solution here is to reintroduce the class_base_init TypeInfo
callback, that was present in one of the early QOM versions but
removed (on my request...) before committing.
This breaks global bus properties, an obscure feature when used
with the command-line which is actually useful and used when used by
backwards-compatible machine types. So this patch also adjusts the
global bus properties in hw/pc_piix.c to refer to the abstract class.
Globals and other properties must be modified in the same patch to
avoid complications related to initialization ordering.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andreas Färber <afaerber@suse.de>
Simple code movement in order to simplify future refactoring.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Andreas Färber <afaerber@suse.de>
Reorder arguments to be more natural, readable and
consistent with other iov_* functions, and change
argument names, from:
iov_from_buf(iov, iov_cnt, buf, iov_off, size)
to
iov_from_buf(iov, iov_cnt, offset, buf, bytes)
The result becomes natural English:
copy data to this `iov' vector with `iov_cnt'
elements starting at byte offset `offset'
from memory buffer `buf', processing `bytes'
bytes max.
(Try to read the original prototype this way).
Also change iov_clear() to more general iov_memset()
(it uses memset() internally anyway).
While at it, add comments to the header file
describing what the routines actually does.
The patch only renames argumens in the header, but
keeps old names in the implementation. The next
patch will touch actual code to match.
Now, it might look wrong to pay so much attention
to so small things. But we've so many badly designed
interfaces already so the whole thing becomes rather
confusing or error prone. One example of this is
previous commit and small discussion which emerged
from it, with an outcome that the utility functions
like these aren't well-understdandable, leading to
strange usage cases. That's why I paid quite some
attention to this set of functions and a few
others in subsequent patches.
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
