Commit Graph

366 Commits

Author SHA1 Message Date
Markus Armbruster
a0c7b99bf7 qapi: Fix crash on missing enum member name
New test case enum-dict-no-name.json crashes:

    $ python3 scripts/qapi-gen.py tests/qapi-schema/enum-dict-no-name.json
    Traceback (most recent call last):
    [...]
      File "/work/armbru/qemu/scripts/qapi/expr.py", line 458, in check_enum
	member_name = member['name']
    KeyError: 'name'

Root cause: we try to retrieve member 'name' before we check for
missing members.  With that fixed, we get the expected error "'data'
member misses key 'name'".

Fixes: 0825f62c84
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210616072121.626431-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-07-15 12:56:41 +02:00
John Snow
d4092ffa26 qapi/parser: add docstrings
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string spacing tweaked slightly]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 17:10:09 +02:00
John Snow
9b91e76b3a qapi/parser: allow 'ch' variable name
We can have a two-letter variable name, as a treat.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-15-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 17:09:44 +02:00
John Snow
013a3aceb5 qapi/parser: Remove superfluous list comprehension
A generator suffices (and quiets a pylint warning).

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:29 +02:00
John Snow
810aff8f29 qapi/parser: add type hint annotations
Annotations do not change runtime behavior.
This commit *only* adds annotations.

(Annotations for QAPIDoc are in a forthcoming commit.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
03386200b9 qapi/parser: Rework _check_pragma_list_of_str as a TypeGuard
TypeGuards wont exist in Python proper until 3.10. Ah well. We can hack
up our own by declaring this function to return the type we claim it
checks for and using this to safely downcast object -> List[str].

In so doing, I bring this function under _pragma so it can use the
'info' object in its closure. Having done this, _pragma also now no
longer needs to take a 'self' parameter, so drop it.

To help with line-length, and with the context evident from its new
scope, rename the function to the shorter check_list_str().

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210519183951.3946870-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
c256263f3d qapi/parser: Fix token membership tests when token can be None
When the token can be None (EOF), we can't use 'x in "abc"' style
membership tests to group types of tokens together, because 'None in
"abc"' is a TypeError.

Easy enough to fix. (Use a tuple: It's neither a static typing error nor
a runtime error to check for None in Tuple[str, ...])

Add tests to prevent a regression. (Note: they cannot be added prior to
this fix, as the unhandled stack trace will not match test output in the
CI system.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
e0e8a0ac2e qapi: add must_match helper
Mypy cannot generally understand that these regex functions cannot
possibly fail. Add a "must_match" helper that makes this clear for
mypy.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
43b1be65f0 qapi/parser: Use @staticmethod where appropriate
No self, no thank you!

(Quiets pylint warnings.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
234dce2c2d qapi/parser: assert object keys are strings
The single quote token implies the value is a string. Assert this to be
the case, to allow us to write an accurate return type for get_members.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
9cd0205d55 qapi/parser: enforce all top-level expressions must be dict in _parse()
Instead of using get_expr nested=False, allow get_expr to always return
any expression. In exchange, add a new error message to the top-level
parser that explains the semantic error: Top-level expressions must
always be JSON objects.

This helps mypy understand the rest of this function which assumes that
get_expr did indeed return a dict.

The exception type changes from QAPIParseError to QAPISemError as a
result, and the error message in two tests now changes.

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210519183951.3946870-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:28 +02:00
John Snow
7c610ce6a9 qapi/parser: Assert lexer value is a string
The type checker can't narrow the type of the token value to string,
because it's only loosely correlated with the return token.

We know that a token of '#' should always have a "str" value.
Add an assertion.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
John Snow
16ff40acc9 qapi/parser: factor parsing routine into method
For the sake of keeping __init__ smaller (and treating it more like a
gallery of what state variables we can expect to see), put the actual
parsing action into a parse method. It remains invoked from the init
method to reduce churn.

To accomplish this, @previously_included becomes the private data
member ._included, and the filename is stashed as ._fname.

Add any missing declarations to the init method, and group them by
function so they can be understood quickly at a glance.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
John Snow
b2b31fdf9b qapi/source: Remove line number from QAPISourceInfo initializer
With the QAPISourceInfo(None, None, None) construct gone, there's no
longer any reason to have to specify that a file starts on the first
line. Remove it from the initializer and default it to 1.

Remove the last vestiges where we check for 'line' being unset, that
can't happen, now.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
John Snow
3404e57410 qapi/parser: Don't try to handle file errors
Fixes: f5d4361cda
Fixes: 52a474180a
Fixes: 46f49468c6

Remove the try/except block that handles file-opening errors in
QAPISchemaParser.__init__() and add one each to
QAPISchemaParser._include() and QAPISchema.__init__() respectively.

This simultaneously fixes the typing of info.fname (f5d4361cda), A
static typing violation in test-qapi (46f49468c6), and a regression of
an error message (52a474180a).

The short-ish version of what motivates this patch is:

- It's hard to write a good error message in the init method,
  because we need to determine the context of our caller to do so.
  It's easier to just let the caller write the message.
- We don't want to allow QAPISourceInfo(None, None, None) to exist. The
  typing introduced by commit f5d4361cda types the 'fname' field as
  (non-optional) str, which was premature until the removal of this
  construct.
- Errors made using such an object are currently incorrect (since
  52a474180a)
- It's not technically a semantic error if we cannot open the schema.
- There are various typing constraints that make mixing these two cases
  undesirable for a single special case.
- test-qapi's code handling an fname of 'None' is now dead, drop it.
  Additionally, Not all QAPIError objects have an 'info' field (since
  46f49468), so deleting this stanza corrects a typing oversight in
  test-qapi introduced by that commit.

Other considerations:

- open() is moved to a 'with' block to ensure file pointers are
  cleaned up deterministically.
- Python 3.3 deprecated IOError and made it a synonym for OSError.
  Avoid the misleading perception these exception handlers are
  narrower than they really are.

The long version:

The error message here is incorrect (since commit 52a474180a):

> python3 qapi-gen.py 'fake.json'
qapi-gen.py: qapi-gen.py: can't read schema file 'fake.json': No such file or directory

In pursuing it, we find that QAPISourceInfo has a special accommodation
for when there's no filename. Meanwhile, the intent when QAPISourceInfo
was typed (f5d4361cda) was non-optional 'str'. This usage was
overlooked.

To remove this, I'd want to avoid having a "fake" QAPISourceInfo
object. I also don't want to explicitly begin accommodating
QAPISourceInfo itself being None, because we actually want to eventually
prove that this can never happen -- We don't want to confuse "The file
isn't open yet" with "This error stems from a definition that wasn't
defined in any file".

(An earlier series tried to create a dummy info object, but it was tough
to prove in review that it worked correctly without creating new
regressions. This patch avoids that distraction. We would like to first
prove that we never raise QAPISemError for any built-in object before we
add "special" info objects. We aren't ready to do that yet.)

So, which way out of the labyrinth?

Here's one way: Don't try to handle errors at a level with "mixed"
semantic contexts; i.e. don't mix inclusion errors (should report a
source line where the include was triggered) and command line errors
(where we specified a file we couldn't read).

Remove the error handling from the initializer of the parser. Pythonic!
Now it's the caller's job to figure out what to do about it. Handle the
error in QAPISchemaParser._include() instead, where we can write a
targeted error message where we are guaranteed to have an 'info' context
to report with.

The root level error can similarly move to QAPISchema.__init__(), where
we know we'll never have an info context to report with, so we use a
more abstract error type.

Now the error looks sensible again:

> python3 qapi-gen.py 'fake.json'
qapi-gen.py: can't read schema file 'fake.json': No such file or directory

With these error cases separated, QAPISourceInfo can be solidified as
never having placeholder arguments that violate our desired types. Clean
up test-qapi along similar lines.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210519183951.3946870-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-05-20 11:28:27 +02:00
John Snow
b54626e0b8 qapi/error.py: enable mypy checks
Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210421192233.3542904-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
30d0a016e9 qapi/error: Add type hints
No functional change.

Note: QAPISourceError's info parameter is Optional[] because schema.py
treats the info property of its various classes as Optional to
accommodate built-in types, which have no source. See prior commit
'qapi/error: assert QAPISourceInfo is not None'.

Signed-off-by: John Snow <jsnow@redhat.com>

Message-Id: <20210421192233.3542904-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
92870cf3af qapi/error.py: enable pylint checks
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
ac6a7d8884 qapi/error.py: move QAPIParseError to parser.py
Keeping it in error.py will create some cyclic import problems when we
add types to the QAPISchemaParser. Callers don't need to know the
details of QAPIParseError unless they are parsing or dealing directly
with the parser, so this won't create any harsh new requirements for
callers in the general case.

Update error.py with a little docstring that gives a nod to where the
error may now be found.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
ac89761179 qapi/error: assert QAPISourceInfo is not None
Built-in stuff is not parsed from a source file, and therefore have no
QAPISourceInfo. If such None info was used for reporting an error,
built-in stuff would be broken. Programming error. Instead of reporting
a confusing error with bogus source location then, we better crash.

We currently crash only if self.col was set. Assert that self.info is
not None in order to crash reliably.

We can not yet change the type of the initializer to prove this cannot
happen at static analysis time before the remainder of the code is fully
typed.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
86cc2ff65a qapi/error: Make QAPISourceError 'col' parameter optional
It's already treated as optional, with one direct caller and some
subclass callers passing 'None'. Make it officially optional, which
requires moving the position of the argument to come after all required
parameters.

QAPISemError becomes functionally identical to QAPISourceError. Keep the
name to preserve its semantic meaning and avoid code churn, but remove
the now-useless __init__ wrapper.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
b54e07cc46 qapi/error: Use Python3-style super()
Missed in commit 2cae67bcb5 "qapi: Use super() now we have Python 3".

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210421192233.3542904-3-jsnow@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
46f49468c6 qapi/error: Repurpose QAPIError as an abstract base exception class
Rename QAPIError to QAPISourceError, and then create a new QAPIError
class that serves as the basis for all of our other custom exceptions,
without specifying any class properties.

This leaves QAPIError as a package-wide error class that's suitable for
any current or future errors.

(Right now, we don't have any errors that DON'T also want to specify a
Source location, but this MAY change. In these cases, a common abstract
ancestor would be desired.)

Add docstrings to explain the intended function of each error class.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421192233.3542904-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
e81718c698 qapi/expr: Update authorship and copyright information
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-18-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
eab99939a7 qapi/expr.py: Use tuples instead of lists for static data
It is -- maybe -- possibly -- three nanoseconds faster.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
a48653638f qapi/expr.py: Add docstrings
Now with more :words:!

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
79e4fd14fb qapi/expr: Only explicitly prohibit 'Kind' nor 'List' for type names
Per list review: qapi-code-gen.txt reserves suffixes Kind and
List only for type names, but the code rejects them for events and
commands, too.

It turns out we reject them earlier anyway: In check_name_upper() for
event names, and in check_name_lower() for command names.

Still, adjust the code for clarity over what precisely we are guarding
against.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-15-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
328e8ca71a qapi/expr.py: enable pylint checks
Signed-off-by: John Snow <jsnow@redhat.com>
Tested-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
e42648dccd qapi/expr.py: Remove single-letter variable
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
210fd63104 qapi/expr.py: Consolidate check_if_str calls in check_if
This is a small rewrite to address some minor style nits.

Don't compare against the empty list to check for the empty condition, and
move the normalization forward to unify the check on the now-normalized
structure.

With the check unified, the local nested function isn't needed anymore
and can be brought down into the normal flow of the function. With the
nesting level changed, shuffle the error strings around a bit to get
them to fit in 79 columns.

Note: although ifcond is typed as Sequence[str] elsewhere, we *know* that
the parser will produce real, bona-fide lists. It's okay to check
isinstance(ifcond, list) here.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
b9ad358aa0 qapi/expr.py: add type hint annotations
Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
538cd41065 qapi/expr.py: Modify check_keys to accept any Collection
This is a minor adjustment that lets parameters @required and
@optional take tuple arguments, in particular ().  Later patches will
make use of that.

(Iterable would also have worked, but Iterable also includes things like
generator expressions which are consumed upon iteration, which would
require a rewrite to make sure that each input was only traversed
once. Collection implies the "can re-iterate" property.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
7a783ce5b5 qapi/expr.py: Add casts in a few select cases
Casts are instructions to the type checker only, they aren't "safe" and
should probably be avoided in general. In this case, when we perform
type checking on a nested structure, the type of each field does not
"stick".

(See PEP 647 for an example of "type narrowing" that does "stick".
 It is available in Python 3.10, so we can't use it yet.)

We don't need to assert that something is a str if we've already checked
or asserted that it is -- use a cast instead for these cases.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
4918bb7def qapi/expr.py: Check type of union and alternate 'data' member
Prior to this commit, specifying a non-object value here causes the QAPI
parser to crash in expr.py with a stack trace with (likely) an
AttributeError when we attempt to call that value's items() method.

This member needs to be an object (Dict), and not anything else. Add a
check for this with a nicer error message, and formalize that check with
new test cases that exercise that error.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
926bb8add7 qapi/expr.py: move string check upwards in check_type
For readability purposes only, shimmy the early return upwards to the
top of the function, so cases proceed in order from least to most
complex.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
b66c62a2d3 qapi/expr.py: Add assertion for union type 'check_dict'
mypy isn't fond of allowing you to check for bool membership in a
collection of str elements. Guard this lookup for precisely when we were
given a name.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
59b5556ce8 qapi/expr.py: constrain incoming expression types
mypy does not know the types of values stored in Dicts that masquerade
as objects. Help the type checker out by constraining the type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
0f231dcf29 qapi/expr.py: Check for dict instead of OrderedDict
OrderedDict is a subtype of dict, so we can check for a more general
form. These functions do not themselves depend on it being any
particular type.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
b7341b89c9 qapi/expr.py: Remove 'info' argument from nested check_if_str
The function can just use the argument from the scope above. Otherwise,
we get shadowed argument errors because the parameter name clashes with
the name of a variable already in-scope.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210421182032.3521476-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
John Snow
d0a263cdd0 qapi/expr: Comment cleanup
The linter yaps after 0825f62c84. Fix this trivial issue to restore the
linter baseline.

(Yes, ideally -- and soon -- the linter will be part of CI so we don't
clutter up the log with fixups. For now, though, the baseline is useful
for testing intermediate commits as types are added to the QAPI
library.)

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210421182032.3521476-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-04-30 12:59:54 +02:00
Markus Armbruster
d83b47646e qapi: Enforce union and alternate branch naming rules
Union branch names should use '-', not '_'.  Enforce this.  The only
offenders are in tests/.  Fix them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-29-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message typo fixed]
2021-03-23 22:31:53 +01:00
Markus Armbruster
407efbf9e7 qapi: Enforce enum member naming rules
Enum members should use '-', not '_'.  Enforce this.  Fix the fixable
offenders (all in tests/), and add the remainder to pragma
member-name-exceptions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-28-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:43 +01:00
Markus Armbruster
5aceeac04d qapi: Enforce struct member naming rules
Struct members, including command arguments, event data, and union
inline base members, should use '-', not '_'.  Enforce this.  Fix the
fixable offenders (all in tests/), and add the remainder to pragma
member-name-exceptions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-27-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:43 +01:00
Markus Armbruster
05ebf841ef qapi: Enforce command naming rules
Command names should be lower-case.  Enforce this.  Fix the fixable
offenders (all in tests/), and add the remainder to pragma
command-name-exceptions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-25-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
e744708a77 qapi: Enforce feature naming rules
Feature names should use '-', not '_'.  Enforce this.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-24-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
9af4b6b9e8 qapi: Prepare for rejecting underscore in command and member names
Command names and member names within a type should be all lower case
with words separated by a hyphen.  We also accept underscore.  Rework
check_name_lower() to optionally reject underscores, but don't use
that option, yet.

Update expected test output for the changed error message.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-23-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
b86df37478 qapi: Rename pragma *-whitelist to *-exceptions
Rename pragma returns-whitelist to command-returns-exceptions, and
name-case-whitelist to member-name-case-exceptions.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-20-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
4a67bd31a4 qapi: Factor out QAPISchemaParser._check_pragma_list_of_str()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-17-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
3e6c8a6331 qapi: Enforce type naming rules
Type names should be CamelCase.  Enforce this.  The only offenders are
in tests/.  Fix them.  Add test type-case to cover the new error.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-15-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Regexp simplified, new test made more robust]
2021-03-23 22:31:05 +01:00
Markus Armbruster
d4f4cae8de qapi: Enforce event naming rules
Event names should be ALL_CAPS with words separated by underscore.
Enforce this.  The only offenders are in tests/.  Fix them.  Existing
test event-case covers the new error.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-14-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
00ffe242d6 qapi: Consistently permit any case in downstream prefixes
We require lowercase __RFQDN_ downstream prefixes only where we
require the prefixed name to be lowercase.  Don't; permit any case in
__RFQDN_ prefixes anywhere.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-13-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
d224e0c092 qapi: Move uppercase rejection to check_name_lower()
check_name_lower() is the only user of check_name_str() using
permit_upper=False.  Move the associated code from check_name_str() to
check_name_lower(), and drop the parameter.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-12-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:31:05 +01:00
Markus Armbruster
eaab06faa5 qapi: Rework name checking in preparation of stricter checking
Naming rules differ for the various kinds of names.  To prepare
enforcing them, define functions to check them: check_name_upper(),
check_name_lower(), and check_name_camel().  For now, these merely
wrap around check_name_str(), but that will change shortly.  Replace
the other uses of check_name_str() by appropriate uses of the
wrappers.  No change in behavior just yet.

check_name_str() now returns the name without downstream and x-
prefix, for use by the wrappers in later patches.  Requires tweaking
regexp @valid_name.  It accepts the same strings as before.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-11-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message improved]
2021-03-23 22:30:51 +01:00
Markus Armbruster
0825f62c84 qapi: Lift enum-specific code out of check_name_str()
check_name_str() masks leading digits when passed enum_member=True.
Only check_enum() does.  Lift the masking into check_enum().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-10-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2021-03-23 22:30:32 +01:00
Markus Armbruster
5fbc78dd36 qapi: Permit flat union members for any tag value
Flat union branch names match the tag enum's member names.  Omitted
branches default to "no members for this tag value".

Branch names starting with a digit get rejected like "'data' member
'0' has an invalid name".  However, omitting the branch works.

This is because flat union tag values get checked twice: as enum
member name, and as union branch name.  The former accepts leading
digits, the latter doesn't.

Branches whose names start with a digit therefore cannot have members.
Feels wrong.  Get rid of the restriction by skipping the latter check.

This can expose c_name() to input it can't handle: a name starting
with a digit.  Improve it to return a valid C identifier for any
input.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-9-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Commit message rewritten]
2021-03-23 22:30:08 +01:00
Markus Armbruster
dbfe3c7c28 qapi: Fix to reject optional members with reserved names
check_type() fails to reject optional members with reserved names,
because it neglects to strip off the leading '*'.  Fix that.

The stripping in check_name_str() is now useless.  Drop.

Also drop the "no leading '*'" assertion, because valid_name.match()
ensures it can't fail.

Fixes: 9fb081e0b9
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210323094025.3569441-8-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-03-23 22:29:37 +01:00
Markus Armbruster
db29164103 qapi: Implement deprecated-input=reject for QMP command arguments
This policy rejects deprecated input, and thus permits "testing the
future".  Implement it for QMP command arguments: reject commands with
deprecated ones.  Example: when QEMU is run with -compat
deprecated-input=reject, then

    {"execute": "eject", "arguments": {"device": "cd"}}

fails like this

    {"error": {"class": "GenericError", "desc": "Deprecated parameter 'device' disabled by policy"}}

When the deprecated parameter is removed, the error will change to

    {"error": {"class": "GenericError", "desc": "Parameter 'device' is unexpected"}}

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-11-armbru@redhat.com>
2021-03-19 16:05:11 +01:00
Markus Armbruster
d2032598c4 qapi: Implement deprecated-input=reject for QMP commands
This policy rejects deprecated input, and thus permits "testing the
future".  Implement it for QMP commands: make deprecated ones fail.
Example: when QEMU is run with -compat deprecated-input=reject, then

    {"execute": "query-cpus"}

fails like this

    {"error": {"class": "CommandNotFound", "desc": "Deprecated command query-cpus disabled by policy"}}

When the deprecated command is removed, the error will change to

    {"error": {"class": "CommandNotFound", "desc": "The command query-cpus has not been found"}}

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-10-armbru@redhat.com>
2021-03-19 16:05:11 +01:00
Markus Armbruster
a291a38fa1 qapi: Implement deprecated-output=hide for QMP event data
This policy suppresses deprecated bits in output, and thus permits
"testing the future".  Implement it for QMP event data: suppress
deprecated members.

No QMP event data is deprecated right now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-6-armbru@redhat.com>
2021-03-19 15:43:33 +01:00
Markus Armbruster
278fc2f7d3 qapi: Implement deprecated-output=hide for QMP events
This policy suppresses deprecated bits in output, and thus permits
"testing the future".  Implement it for QMP events: suppress
deprecated ones.

No QMP event is deprecated right now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-5-armbru@redhat.com>
2021-03-19 15:43:33 +01:00
Markus Armbruster
91fa93e516 qapi: Implement deprecated-output=hide for QMP command results
This policy suppresses deprecated bits in output, and thus permits
"testing the future".  Implement it for QMP command results.  Example:
when QEMU is run with -compat deprecated-output=hide, then

    {"execute": "query-cpus-fast"}

yields

    {"return": [{"thread-id": 9805, "props": {"core-id": 0, "thread-id": 0, "socket-id": 0}, "qom-path": "/machine/unattached/device[0]", "cpu-index": 0, "target": "x86_64"}]}

instead of

    {"return": [{"arch": "x86", "thread-id": 22436, "props": {"core-id": 0, "thread-id": 0, "socket-id": 0}, "qom-path": "/machine/unattached/device[0]", "cpu-index": 0, "target": "x86_64"}]}

Note the suppression of deprecated member "arch".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20210318155519.1224118-4-armbru@redhat.com>
2021-03-19 15:43:33 +01:00
Markus Armbruster
0e92a19b8c qapi: Fix parse errors for removal of null from schema language
Commit 9d55380b5a "qapi: Remove null from schema language" (v4.2.0)
neglected to update two error messages.  Do that now.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20210224101442.1837475-1-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2021-03-05 15:40:50 +01:00
John Snow
9b77d94699 qapi/introspect.py: set _gen_tree's default ifcond argument to ()
We don't need to create an empty, mutable list to pass to _gen_tree;
since it is now typed as a Sequence, we can use the empty tuple as a
default and omit the argument.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-19-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:51:14 +01:00
John Snow
cea53c31b7 qapi/introspect.py: Type _gen_tree variants as Sequence[str]
Optional[List] is clunky; an empty sequence can more elegantly convey
"no variants". By downgrading "List" to "Sequence", we can also accept
tuples; this is useful for the empty tuple specifically, which we may
use as a default parameter because it is immutable.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-18-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string touched up]
Reviewed-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:51:08 +01:00
John Snow
cf26906c4a qapi/introspect.py: Update copyright and authors list
To reflect the work that went into strictly typing introspect.py,
punish myself by claiming credit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:50:26 +01:00
John Snow
5444dedfc7 qapi/introspect.py: Add docstrings to _gen_tree and _tree_to_qlit
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Doc string improvements squashed in]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:50:23 +01:00
John Snow
82b52f6b84 qapi/introspect.py: add type hint annotations
NB: The type aliases (SchemaInfo et al) declare intent for some of the
"dictly-typed" objects we pass around in introspect.py. They do not
enforce the shape of those objects, and cannot, until Python 3.7 or
later. (And even then, it may not be "worth it".)

Annotations are also added to the QAPISchemaEntity __init__ method in
schema.py to allow mypy to statically prove the type of typ.name,
needed to prove the return type of
QAPISchemaGenIntrospectVisitor._use_type().

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-15-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Note on QAPISchemaEntity.__init__() squashed into commit message,
Comment wrapped to conform to PEP 8]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
cf5db2142b qapi/introspect.py: remove _gen_variants helper
It is easier to give a name to all of the dictly-typed objects we pass
around in introspect.py by removing this helper, as it does not return
an object that has any knowable type by itself.

Inline it into its only caller instead.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-14-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
c0e8d9f3c1 qapi/introspect.py: improve readability of _tree_to_qlit
Subjective, but I find getting rid of the comprehensions helps. Also,
divide the sections into scalar and non-scalar sections, and remove
old-style string formatting.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-13-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
2a6c161bbf qapi/introspect.py: improve _tree_to_qlit error message
Trivial; make the error message just a pinch more explicit in case we
trip this by accident in the future.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:20 +01:00
John Snow
4f7f97a7b3 qapi/introspect.py: create a typed 'Annotated' data strutcure
Presently, we use a tuple to attach a dict containing annotations
(comments and compile-time conditionals) to a tree node. This is
undesirable because dicts are difficult to strongly type; promoting it
to a real class allows us to name the values and types of the
annotations we are expecting.

In terms of typing, the Annotated<T> type serves as a generic container
where the annotated node's type is preserved, allowing for greater
specificity than we'd be able to provide without a generic.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:45:17 +01:00
John Snow
9db2734613 qapi/introspect.py: Introduce preliminary tree typing
The types will be used in forthcoming patches to add typing. These types
describe the layout and structure of the objects passed to
_tree_to_qlit, but lack the power to describe annotations until the next
commit.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-10-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:38:00 +01:00
John Snow
d4c5b429fd qapi/introspect.py: Always define all 'extra' dict keys
This mimics how a typed object works, where 'if' and 'comment' are
always set, regardless of if they have a value set or not.

It is safe to do this because of the way that _tree_to_qlit processes
these values (using dict.get with a default of None), resulting in no
change of output from _tree_to_qlit. There are no other users of this
data.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:54 +01:00
John Snow
5f50cede4a qapi/introspect.py: replace 'extra' dict with 'comment' argument
This is only used to pass in a dictionary with a comment already set, so
skip the runaround and just accept the (optional) comment.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:49 +01:00
John Snow
84cf09915f qapi/introspect.py: Unify return type of _make_tree()
Returning two different types conditionally can be complicated to
type. Return one type for consistency.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-7-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:41 +01:00
John Snow
055569603d qapi/introspect.py: guard against ifcond/comment misuse
_tree_to_qlit is called recursively on dict values (isolated from their
keys); at such a point in generating output it is too late to apply an
ifcond. Similarly, comments do not necessarily have a "tidy" place they
can be printed in such a circumstance.

Forbid this usage by renaming "suppress_first_indent" to "dict_value" to
emphasize that indents are suppressed only for the benefit of dict
values; then add an assertion assuring we do not pass ifcond/comments
in this case.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Comment wrapped to conform to PEP 8]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:28 +01:00
John Snow
84bece7dd4 qapi/introspect.py: add _gen_features helper
_make_tree might receive a dict (a SchemaInfo object) or some other type
(usually, a string) for its obj parameter. Adding features information
should arguably be performed by the caller at such a time when we know
the type of the object and don't have to re-interrogate it.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:20 +01:00
John Snow
d70f5130f6 qapi/introspect.py: use _make_tree for features nodes
At present, we open-code this in _make_tree itself; but if the structure
of the tree changes, this is brittle. Use an explicit recursive call to
_make_tree when appropriate to help keep the interior node typing
consistent.

A consequence of doing this is that the 'ifcond' key of the features
dict will be omitted when ifcond is false-ish, just like it is omitted
in top-level calls to _make_tree. This also increases consistency in our
handling of this property.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:37:13 +01:00
John Snow
6b67bcac0f qapi/introspect.py: assert schema is not None
The introspect visitor is stateful, but expects that it will have a
schema to refer to. Add assertions that state this.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 19:35:11 +01:00
John Snow
2184bca7b1 qapi: Replace List[str] with Sequence[str] for ifcond
It does happen to be a list (as of now), but we can describe it in more
general terms with no loss in accuracy to allow tuples and other
constructs.

In the future, we can write "ifcond: Sequence[str] = ()" as a default
parameter, which we could not do safely with a Mutable type like a List.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210216021809.134886-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message tweaked]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-18 17:10:29 +01:00
John Snow
c51172667b qapi: enable strict-optional checks
In the modules that we are checking so far, we can be stricter about the
difference between Optional[T] and T types. Enable that check.

Enabling it now will assist review on further typing and cleanup work.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-17-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
4a82e468e7 qapi: type 'info' as Optional[QAPISourceInfo]
For everything typed so far, type this parameter as
Optional[QAPISourceInfo].

In the most generic case, QAPISchemaEntity's info field may be None to
represent types that come from built-in definitions. Although some
Entity types may not currently have any built-in definitions, it is not
easily possible to constrain the type except on an ad-hoc basis using
assertions.

It's easier and simpler, then, to just say it's always an Optional type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-16-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
cc0747f6b7 qapi/gen: Drop support for QAPIGen without a file name
The previous commit removed the only user of QAPIGen(None).  Tighten
the type hint.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-15-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
c6cd7e4151 qapi/commands: Simplify command registry generation
QAPISchemaGenCommandVisitor.visit_command() needs to generate the
marshalling function into the current module, and also generate its
registration into the ./init system module.  The latter is done
somewhat awkwardly: .__init__() creates a QAPIGenCCode that will not
be written out, each .visit_command() adds its registration to it, and
.visit_end() copies its contents into the ./init module it creates.

Instead provide the means to temporarily switch to another module.
Create the ./init module in .visit_begin(), and generate its initial
part.  Add registrations to it in .visit_command().  Finish it in
.visit_end().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-14-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
d921d27c1b qapi/gen: Support switching to another module temporarily
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-13-jsnow@redhat.com>
[Commit message tweaked]
2021-02-08 14:15:58 +01:00
John Snow
fd9b160384 qapi/gen: write _genc/_genh access shims
Many places assume they can access these fields without checking them
first to ensure they are defined. Eliminating the _genc and _genh fields
and replacing them with functional properties that check for correct
state can ease the typing overhead by eliminating the Optional[T] return
type.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-12-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
39b2d838f1 qapi: centralize the built-in module name definition
Use a constant to make it obvious we're referring to a very specific thing.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-11-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
4ab0ff6da0 qapi/gen: Combine ._add_[user|system]_module
With callers to _add_system_module now explicitly using the './' prefix
to indicate a system module, there is no longer any reason to have
separate interfaces for adding system vs user modules; use a unified
interface that differentiates based on the name.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-10-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
e2bbc4eaa7 qapi: use './builtin' as the built-in module name
Use './builtin' as the built-in module name instead of
None. Clarify the typing that this is now always a string.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-9-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
12893a8ea7 qapi: use explicitly internal module names
QAPISchemaModularCVisitor._add_system_module() prefixes './' to its name
argument to make it a module name.  Pass the module name instead.  This
will allow us to coalesce the methods to add modules later on.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-8-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
[Commit message reworded]
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
Markus Armbruster
f3a705928a qapi/gen: Replace ._begin_system_module()
QAPISchemaModularCVisitor._begin_system_module() is actually just for
the builtin module.  Rename it to ._begin_builtin_module() and drop
its useless @name parameter.

Clarify conditionals in visit_module to make this clear.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-7-jsnow@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
98967c248c qapi: centralize is_[user|system|builtin]_module methods
Define what a module is and define what kind of a module it is once and
for all, in one place.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-6-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
a253b3eb9a qapi/gen: inline _wrap_ifcond into end_if()
We assert _start_if is not None in end_if, but that's opaque to mypy.
By inlining _wrap_ifcond, that constraint becomes provable to mypy.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-5-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
ad1218086e qapi/main: handle theoretical None-return from re.match()
Mypy cannot understand that this match can never be None, so help it
along.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-4-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
3cc01c546b qapi/events: fix visit_event typing
Actually, the arg_type can indeed be Optional.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-3-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
ec9697ab3f qapi/commands: assert arg_type is not None
When boxed is True, expr.py asserts that we must have
arguments. Ultimately, this should mean that if boxed is True that
arg_type should be defined. Mypy cannot infer this, and does not support
'stateful' type inference, e.g.:

```
if x:
    assert y is not None

...

if x:
    y.etc()
```

does not work, because mypy does not statefully remember the conditional
assertion in the second block. Help mypy out by creating a new local
that it can track more easily.

Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <20210201193747.2169670-2-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2021-02-08 14:15:58 +01:00
John Snow
b4c0aa59af qapi/visit.py: add type hint annotations
Annotations do not change runtime behavior.
This commit *only* adds annotations.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-37-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
2cc1eefb84 qapi/visit.py: remove unused parameters from gen_visit_object
And this fixes the pylint report for this file, so make sure we check
this in the future, too.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20201009161558.107041-36-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
554df4f33f qapi/visit.py: assert tag_member contains a QAPISchemaEnumType
This is true by design, but not presently able to be expressed in the
type system. An assertion helps mypy understand our constraints.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-35-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00
John Snow
dec44d3d65 qapi/types.py: remove one-letter variables
"John, if pylint told you to jump off a bridge, would you?"
Hey, if it looked like fun, I might.

Now that this file is clean, enable pylint checks on this file.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eduardo Habkost <ehabkost@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201009161558.107041-34-jsnow@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2020-10-10 11:37:49 +02:00