This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Nothing is using FAIL_TEST or FAIL_TEST_P and that is good. Remove them to not
encourage using them.
If we need a test that should fail, it always needs to fail in a very specific
way which needs to be checked. For this we have e.g. expect_protocol_error().
We never want a fail-test to pass because it failed in a way we did not expect.
Therefore these macros are useless.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This avoids confusing it with the opaque struct weston_test from
protocol/weston-test.xml.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
When we move on to TAP, stdout will be reserved for TAP and stderr is for free
chatter. Set up an example that tests should use testlog() instead of fprintf
or printf to chat in the right place.
Most statements were already printing to stderr, so this just makes then a
little shorter. There are also some statements that printed to stdout and are
now corrected.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Without this attribute, the test macros were making Weston fail to
build with LTO enabled.
Signed-off-by: Markus Trippelsdorf <markus@trippelsdorf.de>
Signed-off-by: Emmanuel Gil Peyrot <emmanuel.peyrot@collabora.com>
Tested-by: Emmanuel Gil Peyrot <emmanuel.peyrot@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
This is a preparatory patch for the next one.
Signed-off-by: Emmanuel Gil Peyrot <emmanuel.peyrot@collabora.com>
Reviewed-by: Daniel Stone <daniels@collabora.com>
Screenshot tests often want to use the test name for writing out images.
This is a helper to get the test name without writing it multiple times
in the source.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.co.uk>
Reviewed-by: Emilio Pozuelo Monfort <emilio.pozuelo@collabora.co.uk>
Reviewed-by: Micah Fedke <micah.fedke@collabora.co.uk>
Reviewed-by: Quentin Glidic <sardemff7+git@sardemff7.net>
To help reduce code duplication and also 'kitchen-sink' includes
the ARRAY_LENGTH macro was moved to a stand-alone file and
referenced from the sources consuming it. Other macros will be
added in subsequent passes.
Signed-off-by: Jon A. Cruz <jonc@osg.samsung.com>
Reviewed-by: Peter Hutterer <peter.hutterer@who-t.net>
Reviewed-by: Bryce Harrington <bryce@osg.samsung.com>
There was an issue recently in screen-share.c where config.h was not
being included, resulting in the wrong definition for off_t being used on
32 bit systems. I checked and I don't think this problem is happening
elsewhere, but to help avoid this sort of problem in the future, I went
through and made sure that config.h is included first whenever system
headers are included.
The config.h header should be included before any system headers, failing
to do this can result in the wrong type sizes being defined on certain
systems, e.g. off_t from sys/types.h
Signed-off-by: Andrew Wedgbury <andrew.wedgbury@realvnc.com>
The new TEST_P macro takes a function name and a "data" argument to
point to an arbitrary array of known size of test data. This allows
multiple tests to be run with different datasets. The array is stored
as a void * but advanced by a known size on each iteration.
The data for each invocation of the test is provided as a "data" argument,
it is the responsibility of the test to cast it to something sensible.
Also fixed single-test running to only run the tests specified
This adds a weston-test-runner for the weston test extension and
some weston test client helper methods.
Converted keyboard-test to use the new test interface, runner,
and helper methods.
Fixes https://bugs.freedesktop.org/show_bug.cgi?id=56822
Signed-off-by: U. Artie Eoff <ullysses.a.eoff@intel.com>
