The signatures need to match.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
A lot of fiddling around to get us below 80 columns.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210607200649.1840382-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Add a little forwarder shim until we are sure that everyone is
comfortable with how to use the tools in their new packaged location.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Remove the shebang, and add a package-defined entry point instead. Now,
it can be accessed using 'qemu-ga-client' from the command line after
installing the package.
The next commit adds a forwarder shim that allows the running of this
script without needing to install the package again.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
The script itself will be unavailable for a few commits before being
restored, with no way to run it right after this commit. This helps move
git history into the new file. To prevent linter regressions, though, we
do need to immediately touch up the filename to remove dashes (to make
the module importable), and remove the executable bit.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
This script is in slightly rough shape, but it still works. A lot of
care went into its initial development. In good faith, I'm updating it
to the latest Python coding standards. If there is in interest in this
script, though, I'll be asking for a contributor to take care of it
further.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
It's only a Dict[str, Any] most of the time. It's not actually
guaranteed to be anything in particular. Fix this type to be
more accurate to the reality we live in.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
I'm only doing a very quick best-effort to preserve this script, to help
keep it from breaking further. I think there are pending ideas swirling
on the right way to implement better SDKs and better clients, and this
script might be a handy reference for those discussions. It presents
some interesting design problems, like static type safety when using a
dynamic RPC mechanism.
I believe it's worth preserving the effort and care that went into
making this script by updating it to work with our current
infrastructure. However, I am disabling the requirement for docstrings
in this file.
If you would like to help improve this script, please add docstrings
alongside any refactors or rejuvenations you might apply at that time.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Turn that nice usage comment into a docstring.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
optparse isn't supported anymore, it's from the python2 days. Replace it
with the mostly similar argparse.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Fixes: 50d189c
These error classes aren't available anymore. Fix the bitrot.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
- Line length should be < 80
- You shouldn't perform unscoped imports except at the top of the module
Notably, the sys.path hack creates problems with the import rule. This
will be fixed later.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210604155532.1499282-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
The 'fuse' command will be unavailable if 'fusepy' is not installed. It
will simply not load and subsequently be unavailable as a subcommand.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-20-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
By leaving the script absent for a commit, git-blame travels to the new
file instead of staying on the shim.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-19-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Move qom-fuse over to the python package now that it passes the
linter. Update the import paradigms so that it continues to pass in the
context of the Python package.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-18-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
In preparation for moving qom-fuse over to the python package, we need
some new dependencies to support it.
Add an optional 'fusepy' dependency that users of the package can opt
into with e.g. "pip install qemu[fuse]" which installs the requirements
necessary to obtain the additional functionality.
Add the same fusepy dependency to the 'devel' extras group --
unfortunately I do not see a way for optional groups to imply other
optional groups at present, so the dependency is repeated. The
development group needs to include the full set of dependencies for the
purpose of static analysis of all features offered by this library.
Lastly, add the [fuse] extras group to tox's configuration as a
workaround so that if a stale tox environment is found when running
`make check-tox`, tox will know to rebuild its environments.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-17-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Because fusepy does not have type hints, add some targeted warning
suppressions.
Namely, we need to allow subclassing something of an unknown type (in
qom_fuse.py), and we need to allow missing imports (recorded against
fuse itself) because mypy will be unable to import fusepy (even when
installed) as it has no types nor type stubs available.
Note: Until now, it was possible to run invocations like 'mypy qemu/'
from ./python and have that work. However, these targeted suppressions
require that you run 'mypy -p qemu/' instead. The correct, canonical
invocation is recorded in ./python/tests/mypy.sh and all of the various
CI invocations always use this correct form.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-16-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
- Use FuseOSError to signal ENOENT instead of returning it
- Wrap qom-get in str(), as we don't always know its type
- The empty return should be b'', not ''.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-15-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
the qom_list method provides a type-safe object that's easier to type
check, so switch to using it.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-14-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Move qom-fuse onto the QOMCommand base established in
python/qemu/qmp/qom_common.py. The interface doesn't change
incompatibly, "qom-fuse mountpoint" still works as an invocation, and
QMP_SOCKET is still used as the environment variable.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-13-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
The methods inherited from fuse don't need docstrings; that's up to
fusepy to handle.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-12-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
- Catch specific exceptions from QMP
- Reraise errors with explicit context
- method parameters should match parent's names
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-11-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
fd and fh are fine: we often use these for "file descriptor" or "file
handle" accordingly. It is rarely the case that you need to enforce a
more semantically meaningful name beyond "This is the file we are using
right now."
While we're here: add comments for all of the non-standard pylint
names. (And the underscore.)
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-10-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
flake8 still has one warning because of the sys.path hack, but that will
be going away by the end of this patch series.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-9-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Hint: you can use symlinks to create qom_fuse.py in python/qemu/qmp/ and
point to scripts/qom-fuse to apply the standard linting rules to this
script.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-8-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Redirect to the new qom scripts. These forwarders can be deleted
eventually when there has been more time for the dust on the Python
packaging to settle and people understand how to find these commands.
Note: You can run these by setting $PYTHONPATH in your shell and then
running "python3 -m qemu.qmp.qom", or you can install the qemu namespace
package and use the "qom" or "qom-set" scripts.
I've written how to install the package elsewhere, but for the sake of
git-blame, cd to ./python, and then do:
- pip3 install [--user] [-e] .
--user will install to your local user install (will not work inside of
a venv), omitting this flag installs to your system-wide packages
(outside of a venv) or to your current virtual environment (inside the
venv).
When installing to a venv or to your system-wide packages, "qom"
should be in your $PATH already. If you do a user install, you may
need to add ~/.local/bin to your $PATH if you haven't already.
-e installs in editable mode: the installed package is effectively just
a symlink to this folder; so changes to your git working tree are
reflected in the installed package.
Note: installing these packages to an environment outside a venv can be
dangerous: Many QEMU scripts will begin to prefer the installed version
instead of the version directly in the tree. Use with caution. editable
mode is recommended when working outside of a venv.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-7-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Add the 'qom', 'qom-set', 'qom-get', 'qom-list', and 'qom-tree' scripts
to the qemu.qmp package. When you install this package, these scripts
will become available on your command line.
(e.g. when inside of a venv, `cd python && pip install .` will add
'qom', 'qom-set', etc to your $PATH.)
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-6-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
Inspired by qom-set, qom-get, qom-tree and qom-list; combine all four of
those scripts into a single script.
A later addition of qom-fuse as an 'extension' necessitates that some
common features are split out and shared between them.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-5-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
This takes the place of qmp-shell's __get_address function. It also
allows other utilities to share the same parser and syntax for
specifying QMP locations.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-4-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
In porting the qom tools, qmp-shell, etc; it becomes evident that this
type is wrong.
This is an integer, not a string. We didn't catch this before because
none of QEMUMonitorProtocol's *users* happen to be checked, and the
internal logic of this class is otherwise self-consistent. Additionally,
mypy was not introspecting into the socket() interface to realize we
were passing a bad type for AF_INET. Fixed now.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-id: 20210603003719.1321369-3-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
In a previous commit, I added tox to the development requirements of the
Python library. I never bothered to add them to the Pipfile, because
they aren't needed there. Here, I sync it anyway in its own commit so
that when we add new packages later that the diffstats will not
confusingly appear to pull in lots of extra packages.
Ideally I could tell Pipenv simply not to install these, but it doesn't
seem to support that, exactly. The alternative is removing Tox from the
development requires, which I'd rather not do.
The other alternative is re-specifying all of the dependencies of
setup.cfg in the Pipfile, which I'd also rather not do.
Picking what feels least-worst here.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 20210603003719.1321369-2-jsnow@redhat.com
Signed-off-by: John Snow <jsnow@redhat.com>
req->receiving is a flag of request being in one concrete yield point
in nbd_co_do_receive_one_chunk().
Such kind of boolean flag is always better to unset before scheduling
the coroutine, to avoid double scheduling. So, let's be more careful.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-33-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
We already have two similar helpers for other state. Let's add another
one for convenience.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-32-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
The only last step we need to reuse the function is coroutine-wrapper.
nbd_open() may be called from non-coroutine context. So, generate the
wrapper and use it.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-31-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
We'll need a possibility of non-blocking nbd_co_establish_connection(),
so that it returns immediately, and it returns success only if a
connections was previously established in background.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-30-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Split out the part that we want to reuse for nbd_open().
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210610100802.5888-29-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
We are going to reuse the script to generate a nbd_ function in
further commit. Prepare the script now.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-28-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
block/nbd doesn't need underlying sioc channel anymore. So, we can
update nbd/client-connection interface to return only one top-most io
channel, which is more straight forward.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-27-vsementsov@virtuozzo.com>
[eblake: squash in Vladimir's fixes for uninit usage caught by clang]
Signed-off-by: Eric Blake <eblake@redhat.com>
Currently sioc pointer is used just to pass from socket-connection to
nbd negotiation. Drop the field, and use local variables instead. With
next commit we'll update nbd/client-connection.c to behave
appropriately (return only top-most ioc, not two channels).
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-26-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Negotiation during reconnect is now done in a thread, and s->sioc is
not available during negotiation. Negotiation in thread will be
cancelled by nbd_client_connection_release() called from
nbd_clear_bdrvstate(). So, we don't need this code chunk anymore.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-25-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Now that we can opt in to negotiation as part of the client connection
thread, use that to simplify connection_co. This is another step on
the way to moving all reconnect code into NBDClientConnection.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-24-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
To be reused in the following patch.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Roman Kagan <rvkagan@yandex-team.ru>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-23-vsementsov@virtuozzo.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Now, when a thread can do negotiation and retry, it may run relatively
long. We need a mechanism to stop it, when the user is not interested
in a result any more. So, on nbd_client_connection_release() let's
shutdown the socket, and do not retry connection if thread is detached.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210610100802.5888-22-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Add an option for a thread to retry connecting until it succeeds. We'll
use nbd/client-connection both for reconnect and for initial connection
in nbd_open(), so we need a possibility to use same NBDClientConnection
instance to connect once in nbd_open() and then use retry semantics for
reconnect.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210610100802.5888-21-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweak]
Signed-off-by: Eric Blake <eblake@redhat.com>
Add arguments and logic to support nbd negotiation in the same thread
after successful connection.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210610100802.5888-20-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
We don't update connect_thread_func() to use QEMU_LOCK_GUARD, as it
will get more complex critical sections logic in further commit, where
QEMU_LOCK_GUARD doesn't help.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20210610100802.5888-19-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
We now have bs-independent connection API, which consists of four
functions:
nbd_client_connection_new()
nbd_client_connection_release()
nbd_co_establish_connection()
nbd_co_establish_connection_cancel()
Move them to a separate file together with NBDClientConnection
structure which becomes private to the new API.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210610100802.5888-18-vsementsov@virtuozzo.com>
[eblake: comment tweaks]
Signed-off-by: Eric Blake <eblake@redhat.com>
