commit 9b8424d573
"exec: split length -> used_length/max_length"
changed field names in struct RAMBlock
It turns out that scripts/dump-guest-memory.py was
poking at this field, update it accordingly.
Cc: qemu-stable@nongnu.org
Cc: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Message-Id: <1440666378-3152-1-git-send-email-mst@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Coccinelle chokes on some idioms from compiler.h and queue.h.
Extract those in a macro file, to be used with "--macro-file
scripts/cocci-macro-file.h".
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Fully removing Sparse support requires more invasive changes. Only
remove the really kernel-specific parts such as address space names.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Mostly change severity levels, but some tests can also be adjusted to refer
to QEMU APIs or data structures.
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
check_type() first checks and peels off the array type, then checks
the element type. For two out of four error messages, it takes pains
to report errors for "array of T" instead of just T. Odd. Let's
examine the errors.
* Unknown element type, e.g.
tests/qapi-schema/args-array-unknown.json:
Member 'array' of 'data' for command 'oops' uses unknown type
'array of NoSuchType'
To make sense of this, you need to know that 'array of NoSuchType'
refers to '[NoSuchType]'. Easy enough. However, simply reporting
Member 'array' of 'data' for command 'oops' uses unknown type
'NoSuchType'
is at least as easy to understand.
* Element type's meta-type is inadmissible, e.g.
tests/qapi-schema/returns-whitelist.json:
'returns' for command 'no-way-this-will-get-whitelisted' cannot
use built-in type 'array of int'
'array of int' is technically not a built-in type, but that's
pedantry. However, simply reporting
'returns' for command 'no-way-this-will-get-whitelisted' cannot
use built-in type 'int'
avoids the issue, and is at least as easy to understand.
* The remaining two errors are unreachable, because the array checking
ensures that value is a string.
Thus, reporting some errors for "array of T" instead of just T works,
but doesn't really improve things. Drop it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The first check ensures the second one can't trigger. Drop the first
one, because the second one is in a more logical place, and emits a
nicer error message.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Clean up white-space, brace placement, and superfluous #ifdef
QAPI_TYPES_BUILTIN_CLEANUP_DEF.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
In generated command handlers, the assignment to retval dominates its
only use. Therefore, its initialization is useless. Drop it.
Suggested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Multiple passes through mcgen() is prone to produce unwanted blank
lines, which we then combat by sprinkling .rstrip() on top. Just
don't do it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
gen_err_check() hard-codes 'local_err' instead of substituting the
argument. Currently harmless, since all callers pass either None or
'local_err'.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reproducer: with
{ 'command': 'user_def_cmd4', 'returns': { 'a': 'int' } }
added to qapi-schema-test.json, qapi-commands.py dies when it tries to
generate the command handler function
Traceback (most recent call last):
File "/work/armbru/qemu/scripts/qapi-commands.py", line 359, in <module>
ret = generate_command_decl(cmd['command'], arglist, ret_type) + "\n"
File "/work/armbru/qemu/scripts/qapi-commands.py", line 29, in generate_command_decl
ret_type=c_type(ret_type), name=c_name(name),
File "/work/armbru/qemu/scripts/qapi.py", line 927, in c_type
assert isinstance(value, str) and value != ""
AssertionError
because the return type doesn't exist.
Simply outlaw this usage, and drop or dumb down test cases accordingly.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
A command's or event's 'data' must be a struct type, given either as a
dictionary, or as struct type name.
Commit dd883c6 tightened the checking there, but not enough: we still
accept 'union'. Fix to reject it.
We may want to support union types there, but we'll have to extend
qapi-commands.py and qapi-events.py for it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
We don't actually enforce our "other than downstream extensions [...],
all names should begin with a letter" rule. Add a FIXME.
We should reject names that differ only in '_' vs. '.' vs. '-',
because they're liable to clash in generated C. Add a FIXME.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Add a FIXME to remind us to fully audit whether removing the
'void *data' branch of each qapi union type can be done safely.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1438297637-26789-1-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Most functions that can return a pointer or set an Error ** value
are decent enough to guarantee a NULL return when reporting an error.
Not so with our generated qapi visitor functions. If the caller
is not careful to clean up partially-allocated objects on error,
then the caller suffers a memory leak.
Properly fixing it is probably complex enough to save for a later
day, so merely document it for now.
Signed-off-by: Eric Blake <eblake@redhat.com>
Message-Id: <1438295587-19069-1-git-send-email-eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The generated code passes mangled schema names to visit_type_enum()
and union's visit_start_struct(). Fix it to pass the names
unadulterated, like we do everywhere else.
Only qapi-schema-test.json actually has names where this makes a
difference: enum __org.qemu_x-Enum, flat union __org.qemu_x-Union2,
simple union __org.qemu_x-Union1 and its implicit enum
__org.qemu_x-Union1Kind.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Use set because that's what it is. While there, rename to
implicit_structs_seen.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The visit_type_implicit_FOO() are generated on demand, right before
their first use. Used by visit_type_STRUCT_fields() when STRUCT has
base FOO, and by visit_type_UNION() when flat UNION has member a FOO.
If the schema defines FOO after its first use as struct base or flat
union member, visit_type_implicit_FOO() calls
visit_type_implicit_FOO() before its definition, which doesn't
compile.
Rearrange qapi-schema-test.json to demonstrate the bug.
Fix by generating the necessary forward declaration.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The struct generated for a flat union is weird: the members of its
base are at the end, except for the union tag, which is at the
beginning.
Example: qapi-schema-test.json has
{ 'struct': 'UserDefUnionBase',
'data': { 'string': 'str', 'enum1': 'EnumOne' } }
{ 'union': 'UserDefFlatUnion',
'base': 'UserDefUnionBase',
'discriminator': 'enum1',
'data': { 'value1' : 'UserDefA',
'value2' : 'UserDefB',
'value3' : 'UserDefB' } }
We generate:
struct UserDefFlatUnion
{
EnumOne enum1;
union {
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
char *string;
};
Change to put all base members at the beginning, unadulterated. Not
only is this easier to understand, it also permits casting the flat
union to its base, if that should become useful.
We now generate:
struct UserDefFlatUnion
{
/* Members inherited from UserDefUnionBase: */
char *string;
EnumOne enum1;
/* Own members: */
union { /* union tag is @enum1 */
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
};
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
A flat union's tag member gets renamed to 'kind' in the generated
code. Breaks when another member named 'kind' exists.
Example, adapted from qapi-schema-test.json:
{ 'struct': 'UserDefUnionBase',
'data': { 'kind': 'str', 'enum1': 'EnumOne' } }
We generate:
struct UserDefFlatUnion
{
EnumOne kind;
union {
void *data;
UserDefA *value1;
UserDefB *value2;
UserDefB *value3;
};
char *kind;
};
Kill the silly rename.
Reported-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
gen_sync_call()'s parameter indent is useless: gen_sync_call() uses it
only as optional argument for push_indent() and pop_indent(), their
default is four, and gen_sync_call()'s only caller passes four. Drop
the parameter.
gen_visitor_input_containers_decl()'s parameter obj is always
"QOBJECT(args)". Use that, and drop the parameter.
Drop unused parameters of gen_marshal_output(),
gen_marshal_input_decl(), generate_visit_struct_body(),
generate_visit_list(), generate_visit_enum(), generate_declaration(),
generate_enum_declaration(), generate_decl_enum().
Drop unused variables in generate_event_enum_lookup(),
generate_enum_lookup(), generate_visit_struct_fields(), check_event().
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
qapi-event.py breaks when you ask for a funny prefix like '@'.
Protect it.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Use c_name() instead of ad hoc code. Doesn't upcase the -p prefix,
which is an improvement in my book. Unbreaks prefix containing '.',
but other funny characters remain broken. To be fixed next.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
The guards around built-in declarations lose their _H. It never made
much sense anyway.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Commit 05dfb26 added eatspace stripping to mcgen(). Move it to
cgen(), just in case somebody gets tempted to use cgen() directly
instead of via mcgen().
cgen() indents blank lines. No such lines get generated right now,
but fix it anyway.
We use triple-quoted strings for program text, like this:
'''
Program text
any number of lines
'''
Keeps the program text relatively readable, but puts an extra newline
at either end. mcgen() "fixes" that by dropping the first and last
line outright. Drop only the newlines.
This unmasks a bug in qapi-commands.py: four quotes instead of three.
Fix it up.
Output doesn't change
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
I should probably document the changes that were made.
Signed-off-by: John Snow <jsnow@redhat.com>
Message-Id: <1435775149-17285-1-git-send-email-jsnow@redhat.com>
Reviewed-By: Kashyap Chamarthy <kchamart@redhat.com>
Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Drop from include/standard-headers/linux/input.h
Add to hw/input/virtio-input-host.c instead.
That allows to build virtio-input (except pass-through) on windows.
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
It seems to make sense to import pci_regs.h from linux:
why maintain our own?
As a first step, move the header to standard-headers,
and add it to the update script.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
In particular, don't include it into headers.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>
The traditional QMP command handler interface
int qmp_FOO(Monitor *mon, const QDict *params, QObject **ret_data);
doesn't provide for returning an Error object. Instead, the handler
is expected to stash it in the monitor with qerror_report().
When we rebased QMP on top of QAPI, we didn't change this interface.
Instead, commit 776574d introduced "middle mode" as a temporary aid
for converting existing QMP commands to QAPI one by one. More than
three years later, we're still using it.
Middle mode has two effects:
* Instead of the native input marshallers
static void qmp_marshal_input_FOO(QDict *, QObject **, Error **)
it generates input marshallers conforming to the traditional QMP
command handler interface.
* It suppresses generation of code to register them with
qmp_register_command()
This permits giving them internal linkage.
As long as we need qmp-commands.hx, we can't use the registry behind
qmp_register_command(), so the latter has to stay for now.
The former has to go to get rid of qerror_report(). Changing all QMP
commands to fit the QAPI mold in one go was impractical back when we
started, but by now there are just a few stragglers left:
do_qmp_capabilities(), qmp_qom_set(), qmp_qom_get(), qmp_object_add(),
qmp_netdev_add(), do_device_add().
Switch middle mode to generate native input marshallers, and adapt the
stragglers. Simplifies both the monitor code and the stragglers.
Rename do_qmp_capabilities() to qmp_capabilities(), and
do_device_add() to qmp_device_add, because that's how QMP command
handlers are named today.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Luiz Capitulino <lcapitulino@redhat.com>
The enum string table parameters in various QOM/QAPI methods
are declared 'const char *strings[]'. This results in const
warnings if passed a variable that was declared as
static const char * const strings[] = { .... };
Add the extra const annotation to the parameters, since
neither the string elements, nor the array itself should
ever be modified.
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Andreas Färber <afaerber@suse.de>
Add processing of optional argument path as "tree base".
Signed-off-by: Martin Cerveny <M.Cerveny@computer.org>
Signed-off-by: Andreas Färber <afaerber@suse.de>
Useless, because it can only occur in commands, and we're not dealing
with commands here.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Insert comments to separate sections dealing with parsing, semantic
analysis, code generation, and so forth.
Move helpers to their proper section.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
To have expression semantic analysis in one place rather than two.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
We maintain a stack of filenames in include_hist for convenient cycle
detection.
As error_path() demonstrates, the same information is readily
available in the expr_info, so just use that, and drop include_hist.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
We print the name as it appears in the include expression. Tools
processing error messages want it relative to the working directory.
Make it so.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
old name new name
----------------------------
input_file fname
input_relname fname
input_fname abs_fname
include_path incl_abs_fname
parent_info incl_info
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Mandatory option is silly, and the error handling is missing: the
programs crash when -i isn't supplied. Make it an argument, and check
it properly.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Report to stderr, prefix with the program name. Also reject
extra arguments.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Anything but --type sync (which is the default) suppresses output
entirely, which makes no sense.
Dates back to the initial commit c17d990. Commit message says
"Currently only generators for synchronous qapi/qmp functions are
supported", so maybe output other than "synchronous qapi/qmp" was
planned at the time, to be selected with --type.
Should other kinds of output ever materialize, we can put the option
back.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Enhance the testsuite to cover downstream events and commands.
Events worked without more tweaks, but commands needed a few final
updates in the generator to mangle names in the appropriate places.
In making those tweaks, it was easier to drop type_visitor() and
inline its actions instead.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Enhance the testsuite to cover downstream alternates, including
whether the branch name or type is downstream. Update the
generator to mangle alternate names in the appropriate places.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Enhance the testsuite to cover downstream flat unions, including
the base type, discriminator name and type, and branch name and
type. Update the generator to mangle the union names in the
appropriate places.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Enhance the testsuite to cover downstream simple unions, including
when a union branch is a downstream name. Update the generator to
mangle the union names in the appropriate places.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Enhance the testsuite to cover downstream structs, including struct
members and base structs. Update the generator to mangle the
struct names in the appropriate places.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Enhance the testsuite to cover a downstream enum type and enum
string. Update the generator to mangle the enum name in the
appropriate places.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Continuing the string of cleanups for supporting downstream names
containing '.', this patch focuses on ensuring c_type() can
handle a downstream name. This patch alone does not fix the
places where generator output should be calling this function
but was open-coding things instead, but it gets us a step closer.
In particular, the changes to c_list_type() and type_name() mean
that type_name(FOO) now handles the case when FOO contains '.',
'-', or is a ticklish identifier other than a builtin (builtins
are exempted because ['int'] must remain mapped to 'intList' and
not 'q_intList'). Meanwhile, ['unix'] now maps to 'q_unixList'
rather than 'unixList', to match the fact that 'unix' is ticklish;
however, our naming conventions state that complex types should
start with a capital, so no type name following conventions will
ever have the 'q_' prepended.
Likewise, changes to c_type() mean that c_type(FOO) properly
handles an enum or complex type FOO with '.' or '-' in the
name, or is a ticklish identifier (again, a ticklish identifier
as a type name violates conventions).
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
c_type() is designed to be called on both string names and on
array designations, so 'name' is a bit misleading because it
operates on more than strings. Also, no caller ever passes
an empty string. Finally, + notation is a bit nicer to read
than '%s' % value for string concatenation.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Now that the two functions are identical, we only need one of them,
and we might as well give it a more descriptive name. Basically,
the function serves as the translation from a QAPI name into a
(portion of a) C identifier, without regards to whether it is a
variable or function name.
Signed-off-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
c_fun() maps '.' to '_', c_var() doesn't. Nothing prevents '.' in
QAPI names that get passed to c_var().
Which QAPI names get passed to c_fun(), to c_var(), or to both is not
obvious. Names of command parameters and struct type members get
passed to c_var().
c_var() strips a leading '*', but this cannot happen. c_fun()
doesn't.
Fix c_var() to work exactly like c_fun().
Perhaps they should be replaced by a single mapping function.
Signed-off-by: Markus Armbruster <armbru@redhat.com>
[add 'import string']
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Add a verbose flag that shows the QMP command that was
constructed, to allow for later copy/pasting, reference,
debugging, etc.
The QMP is converted from a Python literal to JSON first,
to ensure that it is viable input to the actual QMP parser.
As a side-effect, this JSON output will helpfully show all
the necessary conversions that were performed on the input,
illustrating that "True" was transformed back into "true",
literal values are now escaped with "" instead of '', and so on.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Add a special processing mode to craft transactions.
By entering "transaction(" the shell will enter a special
mode where each subsequent command will be saved as a transaction
instead of executed as an individual command.
The transaction can be submitted by entering ")" on a line by itself.
Examples:
Separate lines:
(QEMU) transaction(
TRANS> block-dirty-bitmap-add node=drive0 name=bitmap1
TRANS> block-dirty-bitmap-clear node=drive0 name=bitmap0
TRANS> )
With a transaction action included on the first line:
(QEMU) transaction( block-dirty-bitmap-add node=drive0 name=bitmap2
TRANS> block-dirty-bitmap-add node=drive0 name=bitmap3
TRANS> )
As a one-liner, with just one transaction action:
(QEMU) transaction( block-dirty-bitmap-add node=drive0 name=bitmap0 )
As a side-effect of this patch, blank lines are now parsed as no-ops,
regardless of which shell mode you are in.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
This includes support for [] expressions, single-quotes in
QMP expressions (which is not strictly a part of JSON), and
the ability to use "True", "False" and "None" literals instead
of JSON's equivalent true, false, and null literals.
qmp-shell currently allows you to describe values as
JSON expressions:
key={"key":{"key2":"val"}}
But it does not currently support arrays, which are needed
for serializing and deserializing transactions:
key=[{"type":"drive-backup","data":{...}}]
qmp-shell also only currently accepts doubly quoted strings
as-per JSON spec, but QMP allows single quotes.
Lastly, python allows you to utilize "True" or "False" as
boolean literals, but JSON expects "true" or "false". Expand
qmp-shell to allow the user to type either, converting to the
correct type.
As a consequence of the above, the key=val parsing is also improved
to give better error messages if a key=val token is not provided.
CAVEAT: The parser is still extremely rudimentary and does not
expect to find spaces in {} nor [] expressions. This patch does
not improve this functionality.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Refactor the qmp-shell command line processing function
into two components. This will be used to allow sub-expressions,
which will assist us in adding transactional support to qmp-shell.
Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Kashyap Chamarthy <kchamart@redhat.com>
Signed-off-by: Luiz Capitulino <lcapitulino@redhat.com>
Our type inheritance for both 'struct' and for flat 'union' merges
key/value pairs from the base class with those from the type in
question. Although the C code currently boxes things so that there
is a distinction between which member is referred to, the QMP wire
format does not allow passing a key more than once in a single
object. Besides, if we ever change the generated C code to not be
quite so boxy, we'd want to avoid duplicate member names there,
too.
Fix a testsuite entry added in an earlier patch, as well as adding
a couple more tests to ensure we have appropriate coverage. Ensure
that collisions are detected, regardless of whether there is a
difference in opinion on whether the member name is optional.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The handling of \ inside QAPI strings was less than ideal, and
really only worked JSON's \/, \\, \", and our extension of \'
(an obvious extension, when you realize we use '' instead of ""
for strings). For other things, like '\n', it resulted in a
literal 'n' instead of a newline.
Of course, at the moment, we really have no use for escaped
characters, as QAPI has to map to C identifiers, and we currently
support ASCII only for that. But down the road, we may add
support for default values for string parameters to a command
or struct; if that happens, it would be nice to correctly support
all JSON escape sequences, such as \n or \uXXXX. This gets us
closer, by supporting Unicode escapes in the ASCII range.
Since JSON does not require \OCTAL or \xXX escapes, and our QMP
implementation does not understand them either, I intentionally
reject it here, but it would be an easy addition if we desired it.
Likewise, intentionally refusing the NUL byte means we don't have
to worry about C strings being shorter than the qapi input.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Now that we no longer have nested structs to visit, the use of
prefix strings is no longer required. Remove the code that is
no longer reachable.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
A future patch will be using a 'name':{dictionary} entry in the
QAPI schema to specify a default value for an optional argument
(see previous commit messages for more details why); but existing
use of inline nested structs conflicts with that goal. Now that
all commands have been changed to avoid inline nested structs,
nuke support for them, and turn it into a hard error. Update the
testsuite to reflect tighter parsing rules.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing. Finish up the
conversion to using "struct" in qapi schema by removing the hack
in the generator that allowed 'type'.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Referring to "type" as both a meta-type (built-in, enum, union,
alternate, or struct) and a specific type (the name that the
schema uses for declaring structs) is confusing. The confusion
is only made worse by the fact that the generator mostly already
refers to struct even when dealing with expr['type']. This
commit changes the generator to consistently refer to it as
struct everywhere, plus a single back-compat tweak that allows
accepting the existing .json files as-is, so that the meat of
this change is separate from the mindless churn of that change.
Fix the testsuite fallout for error messages that change, and
in some cases, become more legible. Improve comments to better
match our intentions where a struct (rather than any complex
type) is required. Note that in some cases, an error message
now refers to 'struct' while the schema still refers to 'type';
that will be cleaned up in the later commit to the schema.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Now that we have a way to validate every type, we can also be
stricter about enforcing that callers that want to bypass
type safety in generated code. Prior to this patch, it didn't
matter what value was associated with the key 'gen', but it
looked odd that 'gen':'yes' could result in bypassing the
generated code. These changes also enforce the changes made
earlier in the series for documentation and consolidation of
using '**' as the wildcard type, as well as 'gen':false as the
canonical spelling for requesting type bypass.
Note that 'gen':false is a one-way switch away from the default;
we do not support 'gen':true (similar for 'success-response').
In practice, this doesn't matter.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
...or an array of dictionaries. Although we have to cater to
existing commands, returning a non-dictionary means the command
is not extensible (no new name/value pairs can be added if more
information must be returned in parallel). By making the
whitelist explicit, any new command that falls foul of this
practice will have to be self-documenting, which will encourage
developers to either justify the action or rework the design to
use a dictionary after all.
It's a little bit sloppy that we share a single whitelist among
three clients (it's too permissive for each). If this is a
problem, a future patch could tighten things by having the
generator take the whitelist as an argument (as in
scripts/qapi-commands.py --legacy-returns=...), or by having
the generator output C code that requires explicit use of the
whitelist (as in:
#ifndef FROBNICATE_LEGACY_RETURN_OK
# error Command 'frobnicate' should return a dictionary
#endif
then having the callers define appropriate macros). But until
we need such fine-grained separation (if ever), this patch does
the job just fine.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Previous commits demonstrated that the generator overlooked various
bad naming situations:
- types, commands, and events need a valid name
- enum members must be valid names, when combined with prefix
- union and alternate branches cannot be marked optional
Valid upstream names match [a-zA-Z][a-zA-Z0-9_-]*; valid downstream
names match __[a-zA-Z][a-zA-Z0-9._-]*. Enumerations match the
weaker [a-zA-Z0-9._-]+ (in part thanks to QKeyCode picking an enum
that starts with a digit, which we can't change now due to
backwards compatibility). Rather than call out three separate
regex, this patch just uses a broader combination that allows both
upstream and downstream names, as well as a small hack that
realizes that any enum name is merely a suffix to an already valid
name prefix (that is, any enum name is valid if prepending _ fits
the normal rules).
We could reject new enumeration names beginning with a digit by
whitelisting existing exceptions. We could also be stricter
about the distinction between upstream names (no leading
underscore, no use of dot) and downstream (mandatory leading
double underscore), but it is probably not worth the bother.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Now that we know every expression is valid with regards to
its keys, we can add further tests that those keys refer to
valid types. With this patch, all uses of a type (the 'data':
of command, type, union, alternate, and event; the 'returns':
of command; the 'base': of type and union) must resolve to an
appropriate subset of metatypes declared by the current qapi
parse; this includes recursing into each member of a data
dictionary. Dealing with '**' and nested anonymous structs
will be done in later patches.
Update the testsuite to match improved output.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
For a few QMP commands, we are forced to pass an arbitrary type
without tracking it properly in QAPI. Among the existing clients,
this unnamed type was spelled 'dict', 'visitor', and '**'; this
patch standardizes on '**', matching the documentation changes
earlier in the series.
Meanwhile, for the 'gen' key, we have been ignoring the value,
although the schema consistently used "'no'" ('success-response'
was hard-coded to checking for 'no'). But now that we can support
a literal "false" in the schema, we might as well use that rather
than ignoring the value or special-casing a random string. Note
that these are one-way switches (use of 'gen':true is not the same
as omitting 'gen'). Also, the use of '**' requires 'gen':false,
but the use of 'gen':false does not mandate the use of '**'.
There is no difference to the generated code. Add some tests on
what we'd like to guarantee, although it will take later patches
to clean up test results and actually enforce the use of a bool
parameter.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
In the near term, we will use it for a sensible-looking
'gen':false inside command declarations, instead of the
current ugly 'gen':'no'.
In the long term, it will allow conversion from shorthand
with defaults mentioned only in side-band documentation:
'data':{'*flag':'bool', '*string':'str'}
into an explicit default value documentation, as in:
'data':{'flag':{'type':'bool', 'optional':true, 'default':true},
'string':{'type':'str', 'optional':true, 'default':null}}
We still don't parse integer values (also necessary before
we can allow explicit defaults), but that can come in a later
series.
Update the testsuite to match an improved error message.
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The previous commit demonstrated that the generator overlooked
duplicate expressions:
- a complex type or command reusing a built-in type name
- redeclaration of a type name, whether by the same or different
metatype
- redeclaration of a command or event
- collision of a type with implicit 'Kind' enum for a union
- collision with an implicit MAX enum constant
Since the c_type() function in the generator treats all names
as being in the same namespace, this patch adds a global array
to track all known names and their source, to prevent collisions
before it can cause further problems. While valid .json files
won't trigger any of these cases, we might as well be nicer to
developers that make a typo while trying to add new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The previous commit demonstrated that the generator overlooked some
fairly basic broken expressions:
- missing metataype
- metatype key has a non-string value
- unknown key in relation to the metatype
- conflicting metatype (this patch treats the second metatype as an
unknown key of the first key visited, which is not necessarily the
first key the user typed)
Add check_keys to cover these situations, and update testcases to
match. A couple other tests (enum-missing-data, indented-expr) had
to change since the validation added here occurs so early.
Conversely, changes to ident-with-escape results show that we still
have problems where our handling of escape sequences differs from
true JSON, which will matter down the road if we allow arbitrary
default string values for optional parameters (but for now is not
too bad, as we currently can avoid unicode escaping as we don't
need to represent anything beyond C identifier material).
While valid .json files won't trigger any of these cases, we might
as well be nicer to developers that make a typo while trying to add
new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Previous patches have led up to the point where I create the
new meta-type "'alternate':'Foo'". See the previous patches
for documentation; I intentionally split as much work into
earlier patches to minimize the size of this patch, but a lot
of it is churn due to testsuite fallout after updating to the
new type.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Special-casing 'discriminator == {}' for handling anonymous unions
is getting awkward; since this particular type is not always a
dictionary on the wire, it is easier to treat it as a completely
different class of type, "alternate", so that if a type is listed
in the union_types array, we know it is not an anonymous union.
This patch just further segregates union handling, to make sure that
anonymous unions are not stored in union_types, and splitting up
check_union() into separate functions. A future patch will change
the qapi grammar, and having the segregation already in place will
make it easier to deal with the distinct meta-type.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
This patch widens the scope of a try block (with the attending
reindentation required by Python) in preparation for a future
patch adding more instances of QAPIExprError inside the block.
It's easier to separate indentation from semantic changes, so
this patch has no real behavior change.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Previous commits demonstrated that the generator had several
flaws with less-than-perfect unions:
- a simple union that listed the same branch twice (or two variant
names that map to the same C enumerator, including the implicit
MAX sentinel) ended up generating invalid C code
- an anonymous union that listed two branches with the same qtype
ended up generating invalid C code
- the generator crashed on anonymous union attempts to use an
array type
- the generator was silently ignoring a base type for anonymous
unions
- the generator allowed unknown types or nested anonymous unions
as a branch in an anonymous union
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
None of the existing QMP or QGA interfaces uses a union with a
base type but no discriminator; it is easier to avoid this in the
generator to save room for other future extensions more likely to
be useful. An earlier commit added a union-base-no-discriminator
test to ensure that we eventually give a decent error message;
likewise, removing UserDefUnion outright is okay, because we moved
all the tests we wish to keep into the tests of the simple union
UserDefNativeListUnion in the previous commit. Now is the time to
actually forbid simple union with base, and remove the last
vestiges from the testsuite.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
The previous commit demonstrated that the generator had several
flaws with less-than-perfect enums:
- an enum that listed the same string twice (or two variant
strings that map to the same C enumerator) ended up generating
an invalid C enum
- because the generator adds a _MAX terminator to each enum,
the use of an enum member 'max' can also cause this clash
- if an enum omits 'data', the generator left a python stack
trace rather than a graceful message
- an enum that used a non-array 'data' was silently accepted by
the parser
- an enum that used non-string members in the 'data' member
was silently accepted by the parser
Add check_enum to cover these situations, and update testcases
to match. While valid .json files won't trigger any of these
cases, we might as well be nicer to developers that make a typo
while trying to add new QAPI code.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
Python 2 and Python 3 have a wild history of whether strings
default to ascii or unicode, where Python 3 requires checking
isinstance(foo, basestr) to cover all strings, but where that
code is not portable to Python 2. It's simpler to just state
that we don't care about Unicode strings, and to just always
use the simpler isinstance(foo, str) everywhere.
I'm no python expert, so I'm basing it on this conversation:
https://lists.gnu.org/archive/html/qemu-devel/2014-09/msg05278.html
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
We were missing the 'size' builtin type (which means that QAPI using
[ 'size' ] would fail to compile).
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
There was some redundancy between builtin_types[] and
builtin_type_qtypes{}. Merge them into one.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Markus Armbruster <armbru@redhat.com>
