Commit Graph

263 Commits

Author SHA1 Message Date
Markus Armbruster
f965e8fea6 qapi: New special feature flag "deprecated"
Unlike regular feature flags, the new special feature flag
"deprecated" is recognized by the QAPI generator.  For now, it's only
permitted with commands, events, and struct members.  It will be put
to use shortly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200317115459.31821-26-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Doc typo fixed]
2020-03-17 21:42:47 +01:00
Markus Armbruster
84ab008687 qapi: Add feature flags to struct members
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20200317115459.31821-21-armbru@redhat.com>
2020-03-17 21:25:47 +01:00
Markus Armbruster
7b3bc9e28f qapi: Consistently put @features parameter right after @ifcond
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200317115459.31821-14-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Markus Armbruster
013b4efc9b qapi: Add feature flags to remaining definitions
In v4.1.0, we added feature flags just to struct types (commit
6a8c0b5102^..f3ed93d545), to satisfy an immediate need (commit
c9d4070991 "file-posix: Add dynamic-auto-read-only QAPI feature").  In
v4.2.0, we added them to commands (commit 23394b4c39 "qapi: Add
feature flags to commands") to satisfy another immediate need (commit
d76744e65e "qapi: Allow introspecting fix for savevm's cooperation
with blockdev").

Add them to the remaining definitions: enumeration types, union types,
alternate types, and events.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20200317115459.31821-13-armbru@redhat.com>
2020-03-17 19:58:34 +01:00
Markus Armbruster
ed39c03e2f qapi: Drop conditionals for Python 2
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200304155932.20452-3-armbru@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
2020-03-05 09:24:11 +01:00
Paolo Bonzini
423edd9a31 drop "from __future__ import print_function"
This is only needed for Python 2, which we do not support anymore.

Cc: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20200204160604.19883-1-pbonzini@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:15:16 +01:00
Philippe Mathieu-Daudé
c88ee46cdb tests: Explicit usage of Python 3
Use the program search path to find the Python 3 interpreter.

Patch created mechanically by running:

  $ sed -i "s,^#\!/usr/bin/\(env\ \)\?python$,#\!/usr/bin/env python3," \
       $(git grep -l 'if __name__.*__main__')

Reported-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Suggested-by: Daniel P. Berrangé <berrange@redhat.com>
Suggested-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Message-Id: <20200130163232.10446-5-philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-02-07 15:12:48 +01:00
Markus Armbruster
3e7fb5811b qapi: Fix code generation for empty modules
When a sub-module doesn't contain any definitions, we don't generate
code for it, but we do generate the #include.

We generate code only for modules that get visited.
QAPISchema.visit() visits only modules that have definitions.  It can
visit modules multiple times.

Clean this up as follows.  Collect entities in their QAPISchemaModule.
Have QAPISchema.visit() call QAPISchemaModule.visit() for each module.
Have QAPISchemaModule.visit() call .visit_module() for itself, and
QAPISchemaEntity.visit() for each of its entities.  This way, we visit
each module exactly once.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191120182551.23795-6-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2020-01-14 11:01:58 +01:00
Markus Armbruster
e151941d1b qapi: Check feature documentation against the schema
Commit f3ed93d545 "qapi: Allow documentation for features" neglected
to check documentation against the schema.  Fix that: check them the
same way we check arguments.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-20-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
e4def78755 qapi: Polish reporting of bogus member documentation
Improve error messages from

    the following documented members are not in the declaration: a
    the following documented members are not in the declaration: aa, bb

to the more concise

    documented member 'a' does not exist
    documented members 'aa', 'bb' do not exist

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-19-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
bf83f04e13 qapi: Fix doc comment checking for commands and events
When a command's 'data' is an object, its doc comment describes the
arguments defined there.  When 'data' names a type, the doc comment
does not describe arguments.  Instead, the doc generator inserts a
pointer to the named type.

An event's doc comment works the same.

We don't actually check doc comments for commands and events.
Instead, QAPISchema._def_command() forwards the doc comment to the
implicit argument type, where it gets checked.  Works because the
check only cares for the implicit argument type's members.

Not only is this needlessly hard to understand, it actually falls
apart in two cases:

* When 'data' is empty, there is nothing to forward to, and the doc
  comment remains unchecked.  Demonstrated by test doc-bad-event-arg.

* When 'data' names a type, we can't forward, as the type has its own
  doc comment.  The command or event's doc comment remains unchecked.
  Demonstrated by test doc-bad-boxed-command-arg.

The forwarding goes back to commit 069fb5b250 "qapi: Prepare for
requiring more complete documentation", put to use in commit
816a57cd6e "qapi: Fix detection of bogus member documentation".  That
fix was incomplete.

To fix this, make QAPISchemaCommand and QAPISchemaEvent check doc
comments, and drop the forwarding of doc comments to implicit argument
types.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-12-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
36a43905ff qapi: Fix enum doc comment checking
Enumeration type documentation comments are not checked, as
demonstrated by test doc-bad-enum-member.  This is because we neglect
to call self.doc.check() for enumeration types.  Messed up in
816a57cd6e "qapi: Fix detection of bogus member documentation".  Fix
it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-10-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
b621a26040 qapi: Implement boxed event argument documentation
Generate a reference "Arguments: the members of ...", just like we do
for commands since commit c2dd311cb7 "qapi2texi: Implement boxed
argument documentation".

No change to generated QMP documentation; we don't yet use boxed
events outside tests/.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-7-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
a0418a4a6b tests/qapi-schema: Fix feature documentation testing
Commit 8aa3a33e44 "tests/qapi-schema: Test for good feature lists in
structs" made test-qapi.py show features, but neglected to show their
documentation.  Fix that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-5-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
f6401deb06 tests/qapi-schema: Cover alternate documentation comments
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-4-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
717cfcfae6 tests/qapi-schema: Demonstrate command and event doc comment bugs
Add negative tests doc-bad-boxed-command-arg and doc-bad-event-arg to
cover boxed and no arguments.  They demonstrate insufficient doc
comment checking.

Update positive test doc-good to cover boxed event arguments.  It
demonstrates the generated doc comment misses arguments.

These bugs will be fixed later in this series.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-3-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
f035b47e3f tests/qapi-schema: Demonstrate feature and enum doc comment bugs
Add negative tests doc-bad-enum-member and doc-bad-feature to cover
documentation for nonexistent enum members and features, and test
doc-undoc-feature to cover features lacking documentation.  None of
them works.  To be fixed later in this series.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191024110237.30963-2-armbru@redhat.com>
2019-10-29 07:35:16 +01:00
Markus Armbruster
79598c8a63 tests/qapi-schema: Cover feature documentation comments
Commit 8aa3a33e44 "tests/qapi-schema: Test for good feature lists in
structs" neglected to cover documentation comments, and the previous
commit followed its example.  Make up for them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191018081454.21369-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-10-22 13:54:13 +02:00
Peter Krempa
2e2e0df270 tests: qapi: Test 'features' of commands
Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191018081454.21369-4-armbru@redhat.com>
2019-10-22 13:54:13 +02:00
Peter Krempa
23394b4c39 qapi: Add feature flags to commands
Similarly to features for struct types introduce the feature flags also
for commands. This will allow notifying management layers of fixes and
compatible changes in the behaviour of a command which may not be
detectable any other way.

The changes were heavily inspired by commit 6a8c0b5102.

Signed-off-by: Peter Krempa <pkrempa@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20191018081454.21369-3-armbru@redhat.com>
2019-10-22 13:54:13 +02:00
Markus Armbruster
758f272b6d tests/qapi-schema: Tidy up test output indentation
Command and event details are indented three spaces, everything else
four.  Messed up in commit 156402e504.  Use four spaces consistently.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Peter Krempa <pkrempa@redhat.com>
Message-Id: <20191018081454.21369-2-armbru@redhat.com>
2019-10-22 13:54:13 +02:00
Markus Armbruster
e6c42b96b9 qapi: Split up scripts/qapi/common.py
The QAPI code generator clocks in at some 3100 SLOC in 8 source files.
Almost 60% of the code is in qapi/common.py.  Split it into more
focused modules:

* Move QAPISchemaPragma and QAPISourceInfo to qapi/source.py.

* Move QAPIError and its sub-classes to qapi/error.py.

* Move QAPISchemaParser and QAPIDoc to parser.py.  Use the opportunity
  to put QAPISchemaParser first.

* Move check_expr() & friends to qapi/expr.py.  Use the opportunity to
  put the code into a more sensible order.

* Move QAPISchema & friends to qapi/schema.py

* Move QAPIGen and its sub-classes, ifcontext,
  QAPISchemaModularCVisitor, and QAPISchemaModularCVisitor to qapi/gen.py

* Delete camel_case(), it's unused since commit e98859a9b9 "qapi:
  Clean up after recent conversions to QAPISchemaVisitor"

A number of helper functions remain in qapi/common.py.  I considered
moving the code generator helpers to qapi/gen.py, but decided not to.
Perhaps we should rewrite them as methods of QAPIGen some day.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-7-armbru@redhat.com>
[Add "# -*- coding: utf-8 -*-" lines]
2019-10-22 13:53:55 +02:00
Markus Armbruster
f01338cce6 qapi: Speed up frontend tests
"make check-qapi-schema" takes around 10s user + system time for me.
With -j, it takes a bit over 3s real time.  We have worse tests.  It's
still annoying when you work on the QAPI generator.

Some 1.4s user + system time is consumed by make figuring out what to
do, measured by making a target that does nothing.  There's nothing I
can do about that right now.  But let's see what we can do about the
other 8s.

Almost 7s are spent running test-qapi.py for every test case, the rest
normalizing and diffing test-qapi.py output.  We have 190 test cases.

If I downgrade to python2, it's 4.5s, but python2 is a goner.

Hacking up test-qapi.py to exit(0) without doing anything makes it
only marginally faster.  The problem is Python startup overhead.

Our configure puts -B into $(PYTHON).  Running without -B is faster:
4.4s.

We could improve the Makefile to run test cases only when the test
case or the generator changed.  But I'm after improvement in the case
where the generator changed.

test-qapi.py is designed to be the simplest possible building block
for a shell script to do the complete job (it's actually a Makefile,
not a shell script; no real difference).  Python is just not meant for
that.  It's for bigger blocks.

Move the post-processing and diffing into test-qapi.py, and make it
capable of testing multiple schema files.  Set executable bits while
there.

Running it once per test case now takes slightly longer than 8s.  But
running it once for all of them takes under 0.2s.

Messing with the Makefile to run it only on the tests that need
retesting is clearly not worth the bother.

Expected error output changes because the new normalization strips off
$(SRCDIR)/tests/qapi-schema/ instead of just $(SRCDIR)/.

The .exit files go away, because there is no exit status to test
anymore.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20191018074345.24034-5-armbru@redhat.com>
2019-10-22 09:26:12 +02:00
Markus Armbruster
c615550df3 qapi: Improve source file read error handling
qapi-gen.py crashes when it can't open the main schema file, and when
it can't read from any schema file.  Lazy.

Change QAPISchema.__init__() to take a file name instead of a file
object.  Move the open code from _include() to __init__(), so it's
used for the main schema file, too.

Move the read into the try for good measure, and rephrase the error
message.

Reporting open or read failure for the main schema file needs a
QAPISourceInfo representing "no source".  Make QAPISourceInfo cope
with fname=None.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-27-armbru@redhat.com>
2019-09-28 17:17:48 +02:00
Markus Armbruster
56d2df5e65 qapi: Improve reporting of redefinition
Point to the previous definition, unless it's a built-in.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-26-armbru@redhat.com>
2019-09-28 17:17:48 +02:00
Markus Armbruster
f63326985a qapi: Improve reporting of missing documentation comment
Have check_exprs() check this later, so the error message gains an "in
definition line".  Tweak the error message.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-25-armbru@redhat.com>
2019-09-28 17:17:48 +02:00
Markus Armbruster
fab12376d0 qapi: Improve reporting of invalid 'if' further
check_if()'s errors don't point to the offending part of the
expression.  For instance:

    tests/qapi-schema/alternate-branch-if-invalid.json:2: 'if' condition ' ' makes no sense

Other check_FOO() do, with the help of a @source argument.  Make
check_if() do that, too.  The example above improves to:

    tests/qapi-schema/alternate-branch-if-invalid.json:2: 'if' condition ' ' of 'data' member 'branch' makes no sense

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190927134639.4284-23-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-28 17:17:45 +02:00
Markus Armbruster
eeb57c85da qapi: Avoid redundant definition references in error messages
Many error messages refer to the offending definition even though
they're preceded by an "in definition" line.  Rephrase them.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190927134639.4284-22-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-28 17:17:32 +02:00
Markus Armbruster
3f58cc29a8 qapi: Improve reporting of missing / unknown definition keys
Have check_exprs() call check_keys() later, so its error messages gain
an "in definition" line.

Both check_keys() and check_name_is_str() check the definition's name
is a string.  Since check_keys() now runs after check_name_is_str()
rather than before, its check is dead.  Bury it.  Checking values in
check_keys() is unclean anyway.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-21-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster
a6735a5743 qapi: Improve reporting of invalid flags
Split check_flags() off check_keys() and have check_exprs() call it
later, so its error messages gain an "in definition" line.  Tweak the
error messages.

Checking values in a function named check_keys() is unclean anyway.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-20-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster
576f0b8a53 qapi: Improve reporting of invalid 'if' errors
Move check_if() from check_keys() to check_exprs() and call it later,
so its error messages gain an "in definition" line.

Checking values in a function named check_keys() is unclean anyway.
The original sin was commit 0545f6b887 "qapi: Better error messages
for bad expressions", which checks the value of key 'name'.  More
sinning in commit 2cbf09925a "qapi: More rigorous checking for type
safety bypass", commit c818408e44 "qapi: Implement boxed types for
commands/events", and commit 967c885108 "qapi: add 'if' to top-level
expressions".  This commit does penance for the latter.  The next
commits will do penance for the others.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-19-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster
4ebda5abdb qapi: Move context-free checking to the proper place
QAPISchemaCommand.check() and QAPISchemaEvent().check() check 'data'
is present when 'boxed': true.  That's context-free.  Move to
check_command() and check_event().

Tweak the error message while there.

check_exprs() & friends now check exactly what qapi-code-gen.txt calls
the second layer of syntax.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-18-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster
fa110c6a9e qapi: Move context-sensitive checking to the proper place
When we introduced the QAPISchema intermediate representation (commit
ac88219a6c), we took a shortcut: we left check_exprs() & friends
alone instead of moving semantic checks into the
QAPISchemaFOO.check().  The .check() assert check_exprs() did its job.

Time to finish the conversion job.  Move exactly the context-sensitive
checks to the .check().  They replace assertions there.  Context-free
checks stay put.

Fixes the misleading optional tag error demonstrated by test
flat-union-optional-discriminator.

A few other error message improve.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-17-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster
67fa64ce0e qapi: Move check for reserved names out of add_name()
The checks for reserved names are spread far and wide.  Move one from
add_name() to new check_defn_name_str().  This is a first step towards
collecting them all in dedicated name checking functions next to
check_name().

While there, drop the quotes around the meta-type in
check_name_str()'s error messages: "'command' uses ... name 'NAME'"
becomes "command uses ... name 'NAME'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-13-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster
64e04f7149 qapi: Report invalid '*' prefix like any other invalid name
The special "does not allow optional name" error is well meant, but
confusing in practice.  Drop it.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-12-armbru@redhat.com>
2019-09-28 17:17:19 +02:00
Markus Armbruster
d7bc17c602 qapi: Improve reporting of invalid name errors
Split check_name() into check_name_is_str() and check_name_str(), keep
check_name() as a wrapper.

Move add_name()'s call into its caller check_exprs(), and inline.

This permits delaying check_name_str() there, so its error message
gains an "in definition" line.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-10-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
481a6bd15c qapi: Improve reporting of member name clashes
We report name clashes like this:

    struct-base-clash.json: In struct 'Sub':
    struct-base-clash.json:5: 'name' (member of Sub) collides with 'name' (member of Base)

The "(member of Sub)" is redundant with "In struct 'Sub'".  Comes from
QAPISchemaMember.describe().  Pass info to it, so it can detect the
redundancy and avoid it.  Result:

    struct-base-clash.json: In struct 'Sub':
    struct-base-clash.json:5: member 'name' collides with member 'name' of type 'Base'

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-8-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
2ab218aad6 qapi: Change frontend error messages to start with lower case
Starting error messages with a capital letter complicates things when
text can get interpolated both at the beginning and in the middle of
an error message.  The next patch will do that.  Switch to lower case
to keep it simpler.

For what it's worth, the GNU Coding Standards advise the message
"should not begin with a capital letter when it follows a program name
and/or file name, because that isn’t the beginning of a sentence. (The
sentence conceptually starts at the beginning of the line.)"

While there, avoid breaking lines containing multiple arguments in the
middle of an argument.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-7-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
638c4af931 qapi: Clean up member name case checking
QAPISchemaMember.check_clash() checks for member names that map to the
same c_name().  Takes care of rejecting duplicate names.

It also checks a naming rule: no uppercase in member names.  That's a
rather odd place to do it.  Enforcing naming rules is
check_name_str()'s job.

qapi-code-gen.txt specifies the name case rule applies to the name as
it appears in the schema.  check_clash() checks c_name(name) instead.
No difference, as c_name() leaves alone case, but unclean.

Move the name case check into check_name_str(), less the c_name().
New argument @permit_upper suppresses it.  Pass permit_upper=True for
definitions (which are not members), and when the member's owner is
whitelisted with pragma name-case-whitelist.

Bonus: name-case-whitelist now applies to a union's inline base, too.
Update qapi/qapi-schema.json pragma to whitelist union CpuInfo instead
of CpuInfo's implicit base type's name q_obj_CpuInfo-base.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-6-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
7be6c51194 qapi: Prefix frontend errors with an "in definition" line
We take pains to include the offending expression in error messages,
e.g.

    tests/qapi-schema/alternate-any.json:2: alternate 'Alt' member 'one' cannot use type 'any'

But not always:

    tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings

Instead of improving them one by one, report the offending expression
whenever it is known, like this:

    tests/qapi-schema/enum-if-invalid.json: In enum 'TestIfEnum':
    tests/qapi-schema/enum-if-invalid.json:2: 'if' condition must be a string or a list of strings

Error messages that mention the offending expression become a bit
redundant, e.g.

    tests/qapi-schema/alternate-any.json: In alternate 'Alt':
    tests/qapi-schema/alternate-any.json:2: alternate 'Alt' member 'one' cannot use type 'any'

I'll take care of that later in this series.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-Id: <20190927134639.4284-5-armbru@redhat.com>
2019-09-28 17:17:18 +02:00
Markus Armbruster
dec0012ef8 qapi: Fix missing 'if' checks in struct, union, alternate 'data'
Commit 87adbbffd4..3e270dcacc "qapi: Add 'if' to (implicit
struct|union|alternate) members" (v4.0.0) neglected test coverage, and
promptly failed to check the conditions.  Review fail.

Recent commit "tests/qapi-schema: Demonstrate insufficient 'if'
checking" added test coverage, demonstrating the bug.  Fix it by add
the missing check_if().

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-13-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
c2c7065e17 qapi: Reject blank 'if' conditions in addition to empty ones
"'if': 'COND'" generates "#if COND".  We reject empty COND because it
won't compile.  Blank COND won't compile any better, so reject that,
too.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-12-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
887a2069f7 qapi: Fix broken discriminator error messages
check_union() checks the discriminator exists in base and makes sense.
Two error messages mention the base.  These are broken for anonymous
bases, as demonstrated by tests flat-union-invalid-discriminator and
flat-union-invalid-if-discriminator.err.  The third one doesn't
bother.

First broken when commit ac4338f8eb "qapi: Allow anonymous base for
flat union" (v2.6.0) neglected to adjust the "not a member of base"
error message.  Commit ccadd6bcba "qapi: Add 'if' to implicit struct
members" (v4.0.0) then cloned the flawed error message.

Dumb them down not to mention the base.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-11-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
9d55380b5a qapi: Remove null from schema language
We represent the parse tree as OrderedDict.  We fetch optional dict
members with .get().  So far, so good.

We represent null literals as None.  .get() returns None both for
"absent" and for "present, value is the null literal".  Uh-oh.

Test features-if-invalid exposes this bug: "'if': null" is
misinterpreted as absent "if".

We added null to the schema language to "allow [...] an explicit
default value" (commit e53188ada5 "qapi: Allow true, false and null in
schema json", v2.4.0).  Hasn't happened; null is still unused except
as generic invalid value in tests/.

To fix, we'd have to replace .get() by something more careful, or
represent null differently.  Feasible, but we got more and bigger fish
to fry right now.  Remove the null literal from the schema language.
Replace null in tests by another invalid value.

Test features-if-invalid now behaves as it should.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-10-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
14c3279502 qapi: Improve reporting of lexical errors
Show text up to next structural character, whitespace, or quote
character instead of just the first character.

Forgotten quotes now get reported like "Stray 'command'" instead of
"Stray 'c'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-9-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
9f5e6b088a qapi: Use quotes more consistently in frontend error messages
Consistently enclose error messages in double quotes.  Use single
quotes within, except for one case of "'".

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-8-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
4d42815587 tests/qapi-schema: Demonstrate suboptimal lexical errors
The error message for forgotten quotes around a name shows just the
name's first character, which isn't as nice as it could be.  Same for
attempting to use a number.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-7-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:23 +02:00
Markus Armbruster
ef91ab0d5f tests/qapi-schema: Demonstrate insufficient 'if' checking
Cover invalid 'if' in struct members, features, union and alternate
branches.  Four out of four are broken.  Mark FIXME.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-6-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[Comment typo fixed]
2019-09-24 14:07:23 +02:00
Markus Armbruster
31248b985e tests/qapi-schema: Demonstrate broken discriminator errors
When the union definition's base is an object, some error messages
show it as an OrderedDict.  Oops.  Mark FIXME.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-5-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:22 +02:00
Markus Armbruster
cd346bdc46 tests/qapi-schema: Demonstrate misleading optional tag error
Test flat-union-optional-discriminator declares its union tag as
'*switch': 'Enum', and points to it with 'discriminator': '*switch'.
This gets rejected as "discriminator of flat union 'MyUnion' uses
invalid name '*switch'".  Correct; member 'discriminator' doesn't
accept a '*' prefix.

However, this merely tests name validity checking, which we already
cover elsewhere.  More interesting is testing the valid name 'switch'.
This reports "discriminator 'switch' is not a member of base struct
'Base'", which is misleading.

Copy the existing 'discriminator': '*switch' test to
flat-union-discriminator-bad-name, and rewrite its comment.  Change
flat-union-optional-discriminator to test 'discriminator': 'switch',
and mark it FIXME.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Message-Id: <20190914153506.2151-4-armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2019-09-24 14:07:22 +02:00