Commit Graph

134 Commits

Author SHA1 Message Date
Markus Armbruster
e565d934d2 qapi: Fix to reject stray 't', 'f' and 'n'
Screwed up in commit e53188a.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-06-18 14:19:34 +02:00
Markus Armbruster
a136608727 qapi: Simplify inclusion cycle detection
We maintain a stack of filenames in include_hist for convenient cycle
detection.

As error_path() demonstrates, the same information is readily
available in the expr_info, so just use that, and drop include_hist.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-06-18 14:12:34 +02:00
Markus Armbruster
8608d25251 qapi: Fix file name in error messages for included files
We print the name as it appears in the include expression.  Tools
processing error messages want it relative to the working directory.
Make it so.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-06-18 14:12:34 +02:00
Markus Armbruster
54414047ec qapi: Improve a couple of confusing variable names
old name      new name
----------------------------
input_file    fname
input_relname fname
input_fname   abs_fname
include_path  incl_abs_fname
parent_info   incl_info

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-06-18 14:12:34 +02:00
Markus Armbruster
12c7079449 qapi: Eliminate superfluous QAPISchema attribute input_dir
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-06-18 14:12:33 +02:00
Markus Armbruster
09896d3f48 qapi: Drop pointless flush() before close()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:41:32 +02:00
Markus Armbruster
12f8e1b9ff qapi: Factor open_output(), close_output() out of generators
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:41:32 +02:00
Markus Armbruster
16d80f6181 qapi: Turn generators' mandatory option -i into an argument
Mandatory option is silly, and the error handling is missing: the
programs crash when -i isn't supplied.  Make it an argument, and check
it properly.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:41:23 +02:00
Markus Armbruster
b45409683e qapi: Fix generators to report command line errors decently
Report to stderr, prefix with the program name.  Also reject
extra arguments.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:39:34 +02:00
Markus Armbruster
2114f5a98d qapi: Factor parse_command_line() out of the generators
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:37:14 +02:00
Eric Blake
c6405b54b7 qapi: Make c_type() consistently convert qapi names
Continuing the string of cleanups for supporting downstream names
containing '.', this patch focuses on ensuring c_type() can
handle a downstream name.  This patch alone does not fix the
places where generator output should be calling this function
but was open-coding things instead, but it gets us a step closer.

In particular, the changes to c_list_type() and type_name() mean
that type_name(FOO) now handles the case when FOO contains '.',
'-', or is a ticklish identifier other than a builtin (builtins
are exempted because ['int'] must remain mapped to 'intList' and
not 'q_intList').  Meanwhile, ['unix'] now maps to 'q_unixList'
rather than 'unixList', to match the fact that 'unix' is ticklish;
however, our naming conventions state that complex types should
start with a capital, so no type name following conventions will
ever have the 'q_' prepended.

Likewise, changes to c_type() mean that c_type(FOO) properly
handles an enum or complex type FOO with '.' or '-' in the
name, or is a ticklish identifier (again, a ticklish identifier
as a type name violates conventions).

Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-14 18:21:07 +02:00
Eric Blake
d557344628 qapi: Tidy c_type() logic
c_type() is designed to be called on both string names and on
array designations, so 'name' is a bit misleading because it
operates on more than strings.  Also, no caller ever passes
an empty string.  Finally, + notation is a bit nicer to read
than '%s' % value for string concatenation.

Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-14 18:21:03 +02:00
Markus Armbruster
849bc5382e qapi: Move camel_to_upper(), c_enum_const() to closely related code
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:21:00 +02:00
Markus Armbruster
b42e91484d qapi: Use c_enum_const() in generate_alternate_qtypes()
Missed in commit b0b5819.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:20:57 +02:00
Markus Armbruster
02e20c7e59 qapi: Simplify c_enum_const()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:20:54 +02:00
Markus Armbruster
7c81c61f9c qapi: Rename generate_enum_full_value() to c_enum_const()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:20:51 +02:00
Markus Armbruster
fa6068a1e8 qapi: Rename _generate_enum_string() to camel_to_upper()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
2015-05-14 18:20:48 +02:00
Eric Blake
18df515ebb qapi: Rename identical c_fun()/c_var() into c_name()
Now that the two functions are identical, we only need one of them,
and we might as well give it a more descriptive name.  Basically,
the function serves as the translation from a QAPI name into a
(portion of a) C identifier, without regards to whether it is a
variable or function name.

Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-14 18:20:42 +02:00
Markus Armbruster
47299262de qapi: Fix C identifiers generated for names containing '.'
c_fun() maps '.' to '_', c_var() doesn't.  Nothing prevents '.' in
QAPI names that get passed to c_var().

Which QAPI names get passed to c_fun(), to c_var(), or to both is not
obvious.  Names of command parameters and struct type members get
passed to c_var().

c_var() strips a leading '*', but this cannot happen.  c_fun()
doesn't.

Fix c_var() to work exactly like c_fun().

Perhaps they should be replaced by a single mapping function.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
[add 'import string']
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
2015-05-14 18:20:29 +02:00
Eric Blake
ff55d72eaf qapi: Check for member name conflicts with a base class
Our type inheritance for both 'struct' and for flat 'union' merges
key/value pairs from the base class with those from the type in
question.  Although the C code currently boxes things so that there
is a distinction between which member is referred to, the QMP wire
format does not allow passing a key more than once in a single
object.  Besides, if we ever change the generated C code to not be
quite so boxy, we'd want to avoid duplicate member names there,
too.

Fix a testsuite entry added in an earlier patch, as well as adding
a couple more tests to ensure we have appropriate coverage.  Ensure
that collisions are detected, regardless of whether there is a
difference in opinion on whether the member name is optional.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:02 +02:00
Eric Blake
a7f5966b29 qapi: Support (subset of) \u escapes in strings
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings).  For other things, like '\n', it resulted in a
literal 'n' instead of a newline.

Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that.  But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX.  This gets us
closer, by supporting Unicode escapes in the ASCII range.

Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:02 +02:00
Eric Blake
6b5abc7df7 qapi: Drop support for inline nested types
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument
(see previous commit messages for more details why); but existing
use of inline nested structs conflicts with that goal. Now that
all commands have been changed to avoid inline nested structs,
nuke support for them, and turn it into a hard error. Update the
testsuite to reflect tighter parsing rules.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:02 +02:00
Eric Blake
3e391d3556 qapi: Forbid 'type' in schema
Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing.  Finish up the
conversion to using "struct" in qapi schema by removing the hack
in the generator that allowed 'type'.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Eric Blake
fd41dd4eae qapi: Prefer 'struct' over 'type' in generator
Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing.  The confusion
is only made worse by the fact that the generator mostly already
refers to struct even when dealing with expr['type'].  This
commit changes the generator to consistently refer to it as
struct everywhere, plus a single back-compat tweak that allows
accepting the existing .json files as-is, so that the meat of
this change is separate from the mindless churn of that change.

Fix the testsuite fallout for error messages that change, and
in some cases, become more legible.  Improve comments to better
match our intentions where a struct (rather than any complex
type) is required.  Note that in some cases, an error message
now refers to 'struct' while the schema still refers to 'type';
that will be cleaned up in the later commit to the schema.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Eric Blake
2cbf09925a qapi: More rigorous checking for type safety bypass
Now that we have a way to validate every type, we can also be
stricter about enforcing that callers that want to bypass
type safety in generated code.  Prior to this patch, it didn't
matter what value was associated with the key 'gen', but it
looked odd that 'gen':'yes' could result in bypassing the
generated code.  These changes also enforce the changes made
earlier in the series for documentation and consolidation of
using '**' as the wildcard type, as well as 'gen':false as the
canonical spelling for requesting type bypass.

Note that 'gen':false is a one-way switch away from the default;
we do not support 'gen':true (similar for 'success-response').
In practice, this doesn't matter.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Eric Blake
10d4d997f8 qapi: Whitelist commands that don't return dictionary
...or an array of dictionaries.  Although we have to cater to
existing commands, returning a non-dictionary means the command
is not extensible (no new name/value pairs can be added if more
information must be returned in parallel).  By making the
whitelist explicit, any new command that falls foul of this
practice will have to be self-documenting, which will encourage
developers to either justify the action or rework the design to
use a dictionary after all.

It's a little bit sloppy that we share a single whitelist among
three clients (it's too permissive for each).  If this is a
problem, a future patch could tighten things by having the
generator take the whitelist as an argument (as in
scripts/qapi-commands.py --legacy-returns=...), or by having
the generator output C code that requires explicit use of the
whitelist (as in:
 #ifndef FROBNICATE_LEGACY_RETURN_OK
 # error Command 'frobnicate' should return a dictionary
 #endif
then having the callers define appropriate macros).  But until
we need such fine-grained separation (if ever), this patch does
the job just fine.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Eric Blake
c9e0a79869 qapi: Require valid names
Previous commits demonstrated that the generator overlooked various
bad naming situations:
- types, commands, and events need a valid name
- enum members must be valid names, when combined with prefix
- union and alternate branches cannot be marked optional

Valid upstream names match [a-zA-Z][a-zA-Z0-9_-]*; valid downstream
names match __[a-zA-Z][a-zA-Z0-9._-]*.  Enumerations match the
weaker [a-zA-Z0-9._-]+ (in part thanks to QKeyCode picking an enum
that starts with a digit, which we can't change now due to
backwards compatibility).  Rather than call out three separate
regex, this patch just uses a broader combination that allows both
upstream and downstream names, as well as a small hack that
realizes that any enum name is merely a suffix to an already valid
name prefix (that is, any enum name is valid if prepending _ fits
the normal rules).

We could reject new enumeration names beginning with a digit by
whitelisting existing exceptions.  We could also be stricter
about the distinction between upstream names (no leading
underscore, no use of dot) and downstream (mandatory leading
double underscore), but it is probably not worth the bother.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Eric Blake
dd883c6f05 qapi: More rigourous checking of types
Now that we know every expression is valid with regards to
its keys, we can add further tests that those keys refer to
valid types.  With this patch, all uses of a type (the 'data':
of command, type, union, alternate, and event; the 'returns':
of command; the 'base': of type and union) must resolve to an
appropriate subset of metatypes  declared by the current qapi
parse; this includes recursing into each member of a data
dictionary.  Dealing with '**' and nested anonymous structs
will be done in later patches.

Update the testsuite to match improved output.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Fam Zheng
e53188ada5 qapi: Allow true, false and null in schema json
In the near term, we will use it for a sensible-looking
'gen':false inside command declarations, instead of the
current ugly 'gen':'no'.

In the long term, it will allow conversion from shorthand
with defaults mentioned only in side-band documentation:
 'data':{'*flag':'bool', '*string':'str'}
into an explicit default value documentation, as in:
 'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
         'string':{'type':'str', 'optional':true, 'default':null}}

We still don't parse integer values (also necessary before
we can allow explicit defaults), but that can come in a later
series.

Update the testsuite to match an improved error message.

Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Eric Blake
4dc2e6906e qapi: Better error messages for duplicated expressions
The previous commit demonstrated that the generator overlooked
duplicate expressions:
- a complex type or command reusing a built-in type name
- redeclaration of a type name, whether by the same or different
metatype
- redeclaration of a command or event
- collision of a type with implicit 'Kind' enum for a union
- collision with an implicit MAX enum constant

Since the c_type() function in the generator treats all names
as being in the same namespace, this patch adds a global array
to track all known names and their source, to prevent collisions
before it can cause further problems.  While valid .json files
won't trigger any of these cases, we might as well be nicer to
developers that make a typo while trying to add new QAPI code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Eric Blake
0545f6b887 qapi: Better error messages for bad expressions
The previous commit demonstrated that the generator overlooked some
fairly basic broken expressions:
- missing metataype
- metatype key has a non-string value
- unknown key in relation to the metatype
- conflicting metatype (this patch treats the second metatype as an
unknown key of the first key visited, which is not necessarily the
first key the user typed)

Add check_keys to cover these situations, and update testcases to
match.  A couple other tests (enum-missing-data, indented-expr) had
to change since the validation added here occurs so early.
Conversely, changes to ident-with-escape results show that we still
have problems where our handling of escape sequences differs from
true JSON, which will matter down the road if we allow arbitrary
default string values for optional parameters (but for now is not
too bad, as we currently can avoid unicode escaping as we don't
need to represent anything beyond C identifier material).

While valid .json files won't trigger any of these cases, we might
as well be nicer to developers that make a typo while trying to add
new QAPI code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:01 +02:00
Eric Blake
ab916faddd qapi: Use 'alternate' to replace anonymous union
Previous patches have led up to the point where I create the
new meta-type "'alternate':'Foo'".  See the previous patches
for documentation; I intentionally split as much work into
earlier patches to minimize the size of this patch, but a lot
of it is churn due to testsuite fallout after updating to the
new type.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Eric Blake
811d04fd0c qapi: Segregate anonymous unions into alternates in generator
Special-casing 'discriminator == {}' for handling anonymous unions
is getting awkward; since this particular type is not always a
dictionary on the wire, it is easier to treat it as a completely
different class of type, "alternate", so that if a type is listed
in the union_types array, we know it is not an anonymous union.

This patch just further segregates union handling, to make sure that
anonymous unions are not stored in union_types, and splitting up
check_union() into separate functions.  A future patch will change
the qapi grammar, and having the segregation already in place will
make it easier to deal with the distinct meta-type.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Eric Blake
268a1c5eb1 qapi: Prepare for catching more semantic parse errors
This patch widens the scope of a try block (with the attending
reindentation required by Python) in preparation for a future
patch adding more instances of QAPIExprError inside the block.
It's easier to separate indentation from semantic changes, so
this patch has no real behavior change.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Eric Blake
44bd1276a7 qapi: Tighten checking of unions
Previous commits demonstrated that the generator had several
flaws with less-than-perfect unions:
- a simple union that listed the same branch twice (or two variant
names that map to the same C enumerator, including the implicit
MAX sentinel) ended up generating invalid C code
- an anonymous union that listed two branches with the same qtype
ended up generating invalid C code
- the generator crashed on anonymous union attempts to use an
array type
- the generator was silently ignoring a base type for anonymous
unions
- the generator allowed unknown types or nested anonymous unions
as a branch in an anonymous union

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Eric Blake
a8d4a2e4d7 qapi: Forbid base without discriminator in unions
None of the existing QMP or QGA interfaces uses a union with a
base type but no discriminator; it is easier to avoid this in the
generator to save room for other future extensions more likely to
be useful.  An earlier commit added a union-base-no-discriminator
test to ensure that we eventually give a decent error message;
likewise, removing UserDefUnion outright is okay, because we moved
all the tests we wish to keep into the tests of the simple union
UserDefNativeListUnion in the previous commit.  Now is the time to
actually forbid simple union with base, and remove the last
vestiges from the testsuite.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Eric Blake
cf3935907b qapi: Better error messages for bad enums
The previous commit demonstrated that the generator had several
flaws with less-than-perfect enums:
- an enum that listed the same string twice (or two variant
strings that map to the same C enumerator) ended up generating
an invalid C enum
- because the generator adds a _MAX terminator to each enum,
the use of an enum member 'max' can also cause this clash
- if an enum omits 'data', the generator left a python stack
trace rather than a graceful message
- an enum that used a non-array 'data' was silently accepted by
the parser
- an enum that used non-string members in the 'data' member
was silently accepted by the parser

Add check_enum to cover these situations, and update testcases
to match.  While valid .json files won't trigger any of these
cases, we might as well be nicer to developers that make a typo
while trying to add new QAPI code.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Eric Blake
fe2a9303c9 qapi: Require ASCII in schema
Python 2 and Python 3 have a wild history of whether strings
default to ascii or unicode, where Python 3 requires checking
isinstance(foo, basestr) to cover all strings, but where that
code is not portable to Python 2.  It's simpler to just state
that we don't care about Unicode strings, and to just always
use the simpler isinstance(foo, str) everywhere.

I'm no python expert, so I'm basing it on this conversation:
https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg05278.html

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Eric Blake
cb17f79eef qapi: Fix generation of 'size' builtin type
We were missing the 'size' builtin type (which means that QAPI using
[ 'size' ] would fail to compile).

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Eric Blake
b52c4b9cf0 qapi: Simplify builtin type handling
There was some redundancy between builtin_types[] and
builtin_type_qtypes{}.  Merge them into one.

Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
2015-05-05 18:39:00 +02:00
Stefan Hajnoczi
7ac9a9d6e1 qapi.py: avoid Python 2.5+ any() function
There is one instance of any() in qapi.py that breaks builds on older
distros that ship Python 2.4 (like RHEL5):

  GEN   qmp-commands.h
Traceback (most recent call last):
  File "build/scripts/qapi-commands.py", line 445, in ?
    exprs = parse_schema(input_file)
  File "build/scripts/qapi.py", line 329, in parse_schema
    schema = QAPISchema(open(input_file, "r"))
  File "build/scripts/qapi.py", line 110, in __init__
    if any(include_path == elem[1]
NameError: global name 'any' is not defined

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Benoît Canet <benoit.canet@nodalink.com>
2014-08-28 13:42:25 +01:00
Wenchao Xia
d6f9c82c62 qapi script: clean up in scripts
This patch improve docs and uses c_type(argentry, is_param=True)
in script.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-06-27 09:27:56 -04:00
Wenchao Xia
21cd70dfc1 qapi script: add event support
qapi-event.py will parse the schema and generate qapi-event.c, then
the API in qapi-event.c can be used to handle events in qemu code.
All API have prefix "qapi_event".

The script mainly includes two parts: generate API for each event
define, generate an enum type for all defined events.

Since in some cases the real emit behavior may change, for example,
qemu-img would not send a event, a callback layer is used to
control the behavior. As a result, the stubs at compile time
can be saved, the binding of block layer code and monitor code
will become looser.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-06-23 11:01:25 -04:00
Amos Kong
05dfb26cd2 qapi: Suppress unwanted space between type and identifier
We always generate a space between type and identifier in parameter
and variable declarations, even when idiomatic C style doesn't have
a space there.  Suppress it.

Signed-off-by: Amos Kong <akong@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-06-23 11:01:24 -04:00
Amos Kong
0d14eeb233 qapi: add const prefix to 'char *' insider c_type()
It's ugly to add const prefix for parameter type by an if statement
outside c_type(). This patch adds a parameter to do it.

Signed-off-by: Amos Kong <akong@redhat.com>
Suggested-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-06-23 11:01:24 -04:00
Luiz Capitulino
3478881130 scripts/qapi.py: Avoid syntax not supported by Python 2.4
The Python "except Foo as x" syntax was only introduced in
Python 2.6, but we aim to support Python 2.4 and later.
Use the old-style "except Foo, x" syntax instead, thus
fixing configure/compile on systems with older Python.

Reported-by: Peter Maydell <peter.maydell@linaro.org>
Tested-by: Andreas Färber <andreas.faerber@web.de>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-05-21 09:04:03 -04:00
Benoît Canet
24fd848950 qapi: skip redundant includes
The purpose of this change is to help create a json file containing
common definitions; each bit of generated C code must be emitted
only one time.

A second history global to all QAPISchema instances has been added
to detect when a file is included more than one time and skip these
includes.
It does not act as a stack and the changes made to it by the
__init__ function are propagated back to the caller so it's really
a global state.

Signed-off-by: Benoit Canet <benoit@irqsave.net>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-05-16 10:35:59 -04:00
Lluís Vilanova
a719a27c82 qapi: Add a primitive to include other files from a QAPI schema file
The primitive uses JSON syntax, and include paths are relative to the file using the directive:

  { 'include': 'path/to/file.json' }

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-05-08 14:19:58 -04:00
Lluís Vilanova
33aaad529e qapi: Use an explicit input file
Use an explicit input file on the command-line instead of reading from standard
input.

It also outputs the proper file name when there's an error.

Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-05-08 14:19:58 -04:00
Wenchao Xia
5d371f41b4 qapi script: do not add "_" for every capitalized char in enum
Now "enum AIOContext" will generate AIO_CONTEXT instead of A_I_O_CONTEXT,
"X86CPU" will generate X86_CPU instead of X86_C_P_U.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:42 -04:00
Wenchao Xia
5223070c47 qapi script: do not allow string discriminator
Since enum based discriminators provide better type-safety and
ensure that future qapi additions do not forget to adjust dependent
unions, forbid using string as discriminator from now on.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:42 -04:00
Wenchao Xia
bceae7697f qapi script: support enum type as discriminator in union
By default, any union will automatically generate a enum type as
"[UnionName]Kind" in C code, and it is duplicated when the discriminator
is specified as a pre-defined enum type in schema. After this patch,
the pre-defined enum type will be really used as the switch case
condition in generated C code, if discriminator is an enum field.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:42 -04:00
Wenchao Xia
b0b58195e4 qapi script: use same function to generate enum string
Prior to this patch, qapi-visit.py used custom code to generate enum
names used for handling a qapi union. Fix it to instead reuse common
code, with identical generated results, and allowing future updates to
generation to only need to touch one place.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:42 -04:00
Wenchao Xia
6299659f54 qapi script: code move for generate_enum_name()
Later both qapi-types.py and qapi-visit.py need a common function
for enum name generation.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:42 -04:00
Wenchao Xia
b86b05ed60 qapi script: check correctness of union
Since line info is remembered as QAPISchema.line now, this patch
uses it as additional info for every expr in QAPISchema inside qapi.py,
then improves error message with it in checking of exprs.

For common union the patch will check whether base is a valid complex
type if specified. For flat union it will check whether base presents,
whether discriminator is found in base, whether the key of every branch
is correct when discriminator is an enum type.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:41 -04:00
Wenchao Xia
515b943a91 qapi script: remember line number in schema parsing
Before this patch, 'QAPISchemaError' scans whole input until 'pos'
to get error line number. After this patch, the scan is avoided since
line number is remembered in schema parsing. This patch also benefits
other error report functions, which would be introduced later.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:41 -04:00
Wenchao Xia
4b35991a3b qapi script: add check for duplicated key
It is bad that same key was specified twice, especially when a union has
two branches with same condition. This patch can prevent it.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:41 -04:00
Wenchao Xia
dad1fcab91 qapi script: remember explicitly defined enum values
Later other scripts will need to check the enum values.

Signed-off-by: Wenchao Xia <wenchaoqemu@gmail.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-11 09:07:41 -04:00
Markus Armbruster
678e48a2e4 qapi: Fix licensing of scripts
The scripts carry this copyright notice:

    # This work is licensed under the terms of the GNU GPLv2.
    # See the COPYING.LIB file in the top-level directory.

The sentences contradict each other, as COPYING.LIB contains the LGPL
2.1.  Michael Roth says this was a simple pasto, and he meant to refer
COPYING.  Let's fix that.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2014-03-03 11:16:45 -05:00
Max Reitz
8592a545b6 qapi: Add "errno" to the list of polluted words
Using "errno" directly as an identifier results in various syntax
errors; therefore it should be added to the list of polluted words.

Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
2014-01-22 12:07:18 +01:00
Tomoki Sekiyama
6f88009ee5 Add c++ keywords to QAPI helper script
Add c++ keywords to avoid errors in compiling with c++ compiler.
This also renames class member of PciDeviceInfo to q_class.

Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama@hds.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
2013-09-09 14:17:56 -05:00
Peter Maydell
21e0043bad scripts/qapi.py: Avoid syntax not supported by Python 2.4
The Python "except Foo as x" syntax was only introduced in
Python 2.6, but we aim to support Python 2.4 and later.
Use the old-style "except Foo, x" syntax instead, thus
fixing configure/compile on systems with older Python.

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-08-20 11:52:00 -04:00
Markus Armbruster
f1a145e154 qapi.py: Permit comments starting anywhere on the line
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-10-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-29 10:37:11 -05:00
Markus Armbruster
28b8bd4c75 qapi.py: Rename expr_eval to expr in parse_schema()
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-9-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-29 10:37:11 -05:00
Markus Armbruster
5f3cd2b717 qapi.py: Fix diagnosing non-objects at a schema's top-level
Report syntax error instead of crashing.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-8-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-29 10:37:11 -05:00
Markus Armbruster
6974ccd542 qapi.py: Fix schema parser to check syntax systematically
Fixes at least the following parser bugs:

* accepts any token in place of a colon

* treats comma as optional

* crashes when closing braces or brackets are missing

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-7-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-29 10:37:11 -05:00
Markus Armbruster
9213aa5391 qapi.py: Reject invalid characters in schema file
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-6-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-29 10:37:11 -05:00
Markus Armbruster
2caba36cc6 qapi.py: Decent syntax error reporting
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-5-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-29 10:37:10 -05:00
Markus Armbruster
c7a3f25200 qapi.py: Restructure lexer and parser
The parser has a rather unorthodox structure:

    Until EOF:

        Read a section:

            Generator function get_expr() yields one section after the
            other, as a string.  An unindented, non-empty line that
            isn't a comment starts a new section.

        Lexing:

            Split section into a list of tokens (strings), with help
            of generator function tokenize().

        Parsing:

            Parse the first expression from the list of tokens, with
            parse(), throw away any remaining tokens.

            In parse_schema(): record value of an enum, union or
            struct key (if any) in the appropriate global table,
            append expression to the list of expressions.

    Return list of expressions.

Known issues:

(1) Indentation is significant, unlike in real JSON.

(2) Neither lexer nor parser have any idea of source positions.  Error
    reporting is hard, let's go shopping.

(3) The one error we bother to detect, we "report" via raise.

(4) The lexer silently ignores invalid characters.

(5) If everything in a section gets ignored, the parser crashes.

(6) The lexer treats a string containing a structural character exactly
    like the structural character.

(7) Tokens trailing the first expression in a section are silently
    ignored.

(8) The parser accepts any token in place of a colon.

(9) The parser treats comma as optional.

(10) parse() crashes on unexpected EOF.

(11) parse_schema() crashes when a section's expression isn't a JSON
    object.

Replace this piece of original art by a thoroughly unoriginal design.
Takes care of (1), (2), (5), (6) and (7), and lays the groundwork for
addressing the others.  Generated source files remain unchanged.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Message-id: 1374939721-7876-4-git-send-email-armbru@redhat.com
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2013-07-29 10:37:10 -05:00
Kevin Wolf
69dd62dfd6 qapi: Anonymous unions
The discriminator for anonymous unions is the data type. This allows to
have a union type that allows both of these:

    { 'file': 'my_existing_block_device_id' }
    { 'file': { 'filename': '/tmp/mydisk.qcow2', 'read-only': true } }

Unions like this are specified in the schema with an empty dict as
discriminator. For this example you could take:

    { 'union': 'BlockRef',
      'discriminator': {},
      'data': { 'definition': 'BlockOptions',
                'reference': 'str' } }
    { 'type': 'ExampleObject',
      'data: { 'file': 'BlockRef' } }

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2013-07-26 21:10:11 +02:00
Kevin Wolf
ea66c6d881 qapi.py: Maintain a list of union types
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
2013-07-26 21:10:11 +02:00
Kevin Wolf
b35284ea20 qapi.py: Allow top-level type reference for command definitions
If 'data' for a command definition isn't a dict, but a string, it is
taken as a (struct) type name and the fields of this struct are directly
used as parameters.

This is useful for transactionable commands that can use the same type
definition for both the transaction action and the arguments of the
standalone command.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-07-10 13:39:37 -04:00
Kevin Wolf
bd9927fee4 qapi.py: Avoid code duplication
The code that interprets the read JSON expression and appends types to
the respective global variables was duplicated. We can avoid that by
splitting off the part that reads from the file.

Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-07-10 13:39:37 -04:00
Michael Roth
c0afa9c5f7 qapi: qapi-types.py, native list support
Teach type generators about native types so they can generate the
appropriate linked list types.

Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Reviewed-by: Laszlo Ersek <lersek@redhat.com>
Reviewed-by: Amos Kong <akong@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2013-05-23 09:44:20 -04:00
Paolo Bonzini
1057725f66 qapi: add "unix" to the set of reserved words
It is #defined to 1.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2012-09-26 10:45:02 -03:00
Paolo Bonzini
eda50a656f qapi: do not protect enum values from namespace pollution
Enum values are always preceded by the uppercase name of the enum, so
they do not conflict with reserved words.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2012-09-26 10:45:02 -03:00
Blue Swirl
427a1a2cb1 qapi: avoid reserved keywords
Clang compiler complained about use of reserved word 'restrict' in SLIRP
and QAPI.

Prefix C keywords with "q_", adjust SLIRP accordingly.

Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Blue Swirl <blauwirbel@gmail.com>
2012-08-09 18:34:57 +00:00
Luiz Capitulino
e0d45df7a5 qapi: qapi.py: allow the "'" character to be escaped
Support escaping the escape character, and make more robust (don't die
for '', handle ' without matching '.

Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2012-08-01 18:09:29 -03:00
Laszlo Ersek
092705d4eb qapi: introduce "size" type
v1->v2:
- fall back to uint64 rather than int

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
2012-07-23 11:55:17 +01:00
Laszlo Ersek
c46f18ce2b qapi: generate C types for fixed-width integers
(Long line folded using parens:
<http://www.python.org/dev/peps/pep-0008/#maximum-line-length>.)

Signed-off-by: Laszlo Ersek <lersek@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@linux.vnet.ibm.com>
2012-07-23 11:55:17 +01:00
Federico Simoncelli
c9da228b49 qapi: add c_fun to escape function names
Signed-off-by: Federico Simoncelli <fsimonce@redhat.com>
Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2012-03-26 13:21:46 -05:00
Michael Roth
d8e1f214a0 qapi: fix guardname generation
Fix a bug in handling dotted paths, and exclude directory prefixes
from generated guardnames to avoid odd/pseudo-random guardnames in
generated headers.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
2011-11-30 09:28:32 -06:00
Anthony Liguori
776574d641 qapi: add code generation support for middle mode
To get the ball rolling merging QAPI, this patch introduces a "middle mode" to
the code generator.  In middle mode, the code generator generates marshalling
functions that are compatible with the current QMP server.  We absolutely need
to replace the current QMP server in order to support proper asynchronous
commands but using a middle mode provides a middle-ground that lets us start
converting commands in tree.

Note that all of the commands have been converted already in my glib branch.
Middle mode only exists until we finish merging them from my branch into the
main tree.

Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
2011-10-04 11:00:46 -03:00
Michael Roth
0f923be253 qapi: add qapi.py helper libraries
Signed-off-by: Michael Roth <mdroth@linux.vnet.ibm.com>
Signed-off-by: Luiz Capitulino <lcapitulino@gmail.com>
2011-07-21 16:48:14 -03:00