The test suite is throttled by the headless backend repaint
timer. This commit uses the headless refresh rate option to speed up
runtime by using the immediate repaint-only-on-capture mode by
default. Tests which don't support that mode yet override the refresh
value to use the highest rate possible.
Fixes#682
Signed-off-by: Loïc Molinari <loic.molinari@collabora.com>
Move our DRM test fixture setup later, where we already have a bunch of
per-backend splits, so we can choose to skip our tests at the right
time.
Doing this allows us to skip DRM tests with no memory leaks.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Now that enum weston_renderer_type is public, there is no need for a
local enum renderer_type in the tests.
Signed-off-by: Philipp Zabel <philipp.zabel@gmail.com>
Rather than reinventing --use-pixman and --use-gl throughout each
backend, just have a common --renderer=foo argument which can be used to
explicitly specify the renderer.
The old arguments are still handled for backwards compatibility.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Instead of passing --shell=foo-shell.so, just pass --shell=foo, whilst
accepting the old form for compatibility.
Whilst we're at it, document the --shell argument in the manpage and
README.
Signed-off-by: Daniel Stone <daniels@collabora.com>
While the --backend parameter looks like it takes a file name, it really
is selected from a list of supported strings that are then funneled
through a translation to enum weston_compositor_backend [1].
Because all backend parameters are of the form "...-backend.so", and
writing "--backend=...-backend.so" is boring, allow the --backend option
to match the backend name without "-backend.so" suffix instead.
For example, this allows to use "--backend=headless" instead of
"--backend=headless-backend.so".
Update help text and documentation. Keep the old way working for
backwards compatibility.
[1] 50dbf38514 ("libweston: use enum to choose the backend")
Signed-off-by: Philipp Zabel <philipp.zabel@gmail.com>
Spinning up the full desktop-shell is pointless overhead for most tests,
which don't need the full load.
Signed-off-by: Daniel Stone <daniels@collabora.com>
Setting up the arguments may consume some of the arguments, e.g. if we
provide a config file or extra modules, then the test harness is
expected to be responsible for freeing those arguments.
Checking the requirements and bailing first means that we never do that,
and thus skipped tests result in leaks. Flip the order so we set up the
args first and skip last, so we can consistently take ownership of all
the provided setup parameters.
Signed-off-by: Daniel Stone <daniels@collabora.com>
We treat the argv we pass into the compositor as its to mangle, just as
it is free to do so for POSIX argv. To support this, we stash argv away
and free the saved copy later so as to not leak.
This works perfectly, except when we never call the compositor at all,
and have no saved array to free. Make sure we free the args in this
case, which can be seen as a leak of any generated args when a test
skips on preflight checks, e.g. drm-smoke when not running in CI.
Signed-off-by: Daniel Stone <daniels@collabora.com>
This reverts commit 1618697dc3.
The original commit was a workaround for
https://gitlab.freedesktop.org/mesa/mesa/-/issues/2219 which was fixed
in Mesa:
- c7617d8908a970124321ce731b43d5996c3c5775 released as 20.1.0-rc1
- a0e6341fe4417e41cda0b19e4fa7f8bbe4e1dba1 released as 19.3.5
- f27e5d9df5bc9c85d45c2cb1f2a4997b453365fe released as 20.0.0
This workaround should not be necessary anymore, we don't use it in our
CI, and it was manual to begin with. Therefore remove it.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Fbdev backend was deprecated in the Weston 10.0.0 release with
6338dbd581. Before that, I suggested
already in 2019 to remove it, but it was too soon then. Now it seems the
final voices asking for fbdev to be kept have been satisfied, see the
linked issue.
Fbdev-backend uses a kernel graphics UAPI (fbdev) which is sub-par for a
Wayland compositor: you cannot do GPU accelerated graphics in any
reasonable way, no hotplug support, multi-output support is tedious, and
so on. Most importantly, Linux has deprecated fbdev a long time ago due
to the UAPI fitting modern systems and use cases very poorly, but cannot
get rid of it if any users remain. Let's do here what we can to reduce
fbdev usage.
I am doing color management related additions to libweston which require
adding checks to every backend. One backend less is less churn to write
and review.
Libweston major version has already been bumped to 11, so the next
release will be Weston 11, without fbdev. enum weston_compositor_backend
entries change their numerical values.
Fixes: https://gitlab.freedesktop.org/wayland/weston/-/issues/581
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
We have a string helper which wraps asprintf(). Uses that one because it
clears out the destination string, but also it won't return the number
of bytes unlinke asprintf().
Fixes warnings like:
warning: ignoring return value of ‘asprintf’ declared with attribute
‘warn_unused_result’.
Signed-off-by: Marius Vlad <marius.vlad@collabora.com>
This fixes a leak found running drm-smoke-test with ASan. Do not forget
to call prog_args_fini() when:
- skipping DRM-backend tests because we don't have
WESTON_TEST_SUITE_DRM_DEVICE set.
- we fail to acquire the lock needed to run DRM-backend tests.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
This allows tests to skip when required capabilities are not present.
The output damage test for the shadow buffer case needs this.
required_capabilities is added to struct weston_testsuite_quirks which
is libweston public API just because there is no better place currently.
This is a little weird because the code to check it is in compositor,
not libweston.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Until now we had two different protocol extensions: one for the
test suite screenshooter and other for the screenshooter client.
As they are identical and this won't change, make the test suite
use the same protocol that the screenshooter client uses.
Besides the cleanup, this change will also allow us to use the
DRM writeback screenshooter in the test suite, as the test suite
implementation is hardcoded to use a renderer based screenshooter.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
There are some specific cases in which we need Weston to
behave differently when running in the test suite. This
adds a new API to allow the tests to select these behaviors.
For instance, in the DRM backend we plan to add a writeback
connector screenshooter. In case it fails for some
reason, it should fallback to the renderer screenshooter
that all other backends use. But if we add a test to
ensure the correctness of the writeback screenshooter,
we don't want it to fallback to the renderer one, we
want it to fail. With this new API we can choose to
disable the fallback behavior specifically for this test.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
Currently doesn't exist a standard way to write a weston.ini inside a test.
Here, two new functions `weston_ini_setup` and `cfgln` are introduced to
help the test writer to write a weston.ini file and load it to the test.
And `internal-screenshot-test` is converted to use the new method of write
a weston.ini. This conversion serves as example and initial API test.
The tester needs to call `weston_test_harness_execute_as_client` or
`weston_test_harness_execute_as_plugin` in the same way as before.
The `weston_ini_setup` will fill the setup->config_file with the
correct path to the weston.ini file.
The main design goal is to avoid pre-made or build-made weston.ini(s)
and keep the test as self-contained as possible.
Closes:#410
Signed-off-by: Igor Matheus Andrade Torrente <igormtorrente@gmail.com>
With this patch we add support to run DRM-backend tests locally
in the test suite. For now this won't work in the CI, as there
are no cards available. But the plan is to achieve this by using
VKMS (virtual KMS) in the future.
To run DRM-backend tests locally, first of all the user has to
set the environment variable WESTON_TEST_SUITE_DRM_DEVICE to
'card0', 'card1' or any other device where he wants to run
the tests. Also, for now it only works if it is run as root,
but in the future this problem will be solved.
The tests will run on a non-default seat. The reason for that
is that we want to avoid opening input devices unnecessarily.
Also, since DRM-backend usage requires gaining DRM master status
on a DRM KMS device, nothing else must be using the device at
the same time. To achieve this we use a lock to run the
DRM-backend tests sequentially.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
The test suite is dealing only with headless-backend tests.
In order to make it able to run DRM-backend tests, we have
to properly select the renderer that it will use.
This patch add the command line option --use-pixman if the test
defines the DRM-backend renderer as RENDERER_PIXMAN, and it will
add nothing to the command line if it defines RENDERER_GL (the
DRM-backend default renderer is already GL). Also, if the user
defines the DRM-backend renderer as RENDERER_NOOP, the test will
fail (as it should, since DRM-backend does not implement it).
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
In the test suite we have some default options which
are command line arguments used by most of the tests.
Two of these are width==320 and height==240. But
when we have DRM or fbdev backends, width and height
are not possible command line arguments. This makes
impossible to run tests that uses one of these types
of backends, as the compositor won't open if the
command line string is wrong.
Fix this by not passing command line arguments width
and height if the backend is DRM or fbdev.
Signed-off-by: Leandro Ribeiro <leandro.ribeiro@collabora.com>
With these, a test can initialize the headless-backend with non-default scale
and transform which allows testing output scales and transforms.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This shall be used by CI due to https://gitlab.freedesktop.org/mesa/mesa/issues/2219
It defaults to true, meaning that people by default will be running the
GL-renderer tests. It works fine on hardware drivers, just not llvmpipe.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Instead of relying on Meson setting up environment so that Weston and tests
find all their files, build those values into the tests. This way one can
execute a test program successfully wihtout Meson, simply by running it.
The old environment variables are still honoured if set. This might change in
the future.
Baking the source or build directory paths into the tests should not regress
reproducible builds, because the binaries where test-config.h values are used
will not be installed.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
This replaces the old test harness with a new one.
The old harness relied on fork()'ing each test which makes tests independent,
but makes debugging them harder. The new harness runs client code in a thread
instead of a new process. A side-effect of not fork()'ing anymore is that any
failure will stop running a test series short. Fortunately we do not have any
tests that are expected to crash or fail.
The old harness executed 'weston' from Meson, with lots of setup as both
command line options and environment variables. The new harness executes
wet_main() instead: the test program itself calls the compositor main function
to execute the compositor in-process. Command line arguments are configured in
the test program itself, not in meson.build. Environment variables aside, you
are able to run a test by simply executing the test program, even if it is a
plugin test.
The new harness adds a new type of iteration: fixtures. For now, fixtures are
used to set up the compositor for tests that need a compositor. If necessary, a
fixture setup may include a data array of arbitrary type for executing the test
series for each element in the array. This will be most useful for running
screenshooting tests with both Pixman- and GL-renderers.
The new harness outputs TAP formatted results into stdout. Meson is not
switched to consume TAP yet though, because it would require a Meson version
requirement bump and would not have any benefits at this time. OTOH outputting
TAP is trivial and sets up a clear precedent of random test chatter belonging
to stderr.
This commit migrates only few tests to actually make use of the new features:
roles is a basic client test, subsurface-shot is a client test that
demonstrates the fixture array, and plugin-registry is a plugin test. The rest
of the tests will be migrated later.
Once all tests are migrated, we can remove the test-specific setup from
meson.build, leaving only the actual build instructions in there.
The not migrated tests and stand-alone tests suffer only a minor change: they
no longer fork() for each TEST(), otherwise they keep running as before.
Signed-off-by: Pekka Paalanen <pekka.paalanen@collabora.com>
Daniel Stone