Commit Graph

13 Commits

Author SHA1 Message Date
Thomas Huth
36a7ab5f04 tests/acceptance: Increase the timeout in the replay tests
Our gitlab-CI just showed a failed test_ppc_mac99 since it was apparently
killed some few seconds before the test finished. Allow it some more
time to complete.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Acked-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210127065222.48650-1-thuth@redhat.com>
2021-02-08 09:40:51 +00:00
Philippe Mathieu-Daudé
b48580ad92 tests/acceptance: Disable Spartan-3A DSP 1800A test
This test is regularly failing on CI:

   (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
  Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
  ...
  Freeing unused kernel memory: 1444K
  This architecture does not have kernel memory protection.
  [nothing happens here]
  Runner error occurred: Timeout reached (90.91 s)

This is a regression. Until someone figure out the problem,
disable the test to keep CI pipeline useful.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201109091719.2449141-1-f4bug@amsat.org>
Message-Id: <20201110192316.26397-9-alex.bennee@linaro.org>
2020-11-16 11:08:04 +00:00
Pavel Dovgalyuk
1d2ff14b72 tests/acceptance: Add MIPS record/replay tests
This patch adds MIPS-targeted acceptance tests for
record/replay functions.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <160276110297.2705.10918105269658307206.stgit@pasha-ThinkPad-X280>
[PMD: Moved 'override timeout' comment from instance to class,
      moved nanomips tests to ReplayKernelSlow class,
      tagged ReplayKernelSlow class with AVOCADO_TIMEOUT_EXPECTED]
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-10-17 13:59:40 +02:00
Philippe Mathieu-Daudé
a58bf4b3d9 tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Armbian servers are not very reliable and confused the GitLab CI
users a few times this month (path updated, archives moved, and
now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add
them to the cache).

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20200917163954.50514-1-philmd@redhat.com>
2020-09-18 19:44:22 +02:00
Thomas Huth
4d6862ffc7 tests/acceptance: Disable the rx sash and arm cubieboard replay test on Gitlab
These tests always time out on Gitlab, not sure what's happening here.
Let's disable them until somebody has enough spare time to debug the
issues.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200730141326.8260-3-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-08-05 10:38:27 +02:00
Alex Bennée
c5008c76ee gitlab: add acceptance testing to system builds
As part of migrating things from Travis to GitLab add the acceptance
tests. To do this:

  - rename system1 to system-ubuntu-main
  - rename system2 to system-fedora-misc
  - split into build/check/acceptance
  - remove -j from check stages
  - use artifacts to save build stage
  - add post acceptance template and use

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200701135652.1366-31-alex.bennee@linaro.org>
2020-07-11 15:53:00 +01:00
Pavel Dovgalyuk
b52d7e216c tests/acceptance: record/replay tests with advcal images
This patch adds more record/replay tests with kernel images.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
Message-Id: <159073592589.20809.5156301499042635614.stgit@pasha-ThinkPad-X280>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[PMD: Use os.path.join(), add avocado 'cpu' tags]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-06-21 13:58:52 +02:00
Pavel Dovgalyuk
20b1bf2ea9 tests/acceptance: add record/replay test for m68k
This patch adds a test for record/replay of the kernel
image boot for m68k platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
Reviewed-by: Laurent Vivier <laurent@vivier.eu>
Message-Id: <159073592033.20809.1838967871297177313.stgit@pasha-ThinkPad-X280>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-06-21 13:58:52 +02:00
Pavel Dovgalyuk
2e1206b9cb tests/acceptance: add record/replay test for ppc64
This patch adds a test for record/replay of the kernel
image boot for ppc64 platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
Message-Id: <159073591363.20809.15658672985367330140.stgit@pasha-ThinkPad-X280>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-06-21 13:58:52 +02:00
Pavel Dovgalyuk
2f2d83ad2e tests/acceptance: add record/replay test for arm
This patch adds a test for record/replay of the kernel
image boot for two different arm platforms.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
Message-Id: <159073590785.20809.17654573764167037499.stgit@pasha-ThinkPad-X280>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-06-21 13:58:52 +02:00
Pavel Dovgalyuk
152a41b774 tests/acceptance: add record/replay test for aarch64
This patch adds a test for record/replay of the kernel
image boot for aarch64 platform.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
Message-Id: <159073590231.20809.9842179251741585482.stgit@pasha-ThinkPad-X280>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-06-21 13:58:52 +02:00
Pavel Dovgalyuk
904be130b2 tests/acceptance: add kernel record/replay test for x86_64
This patch adds a test for record/replay an execution of x86_64 machine.
Execution scenario includes simple kernel boot, which allows testing
basic hardware interaction in RR mode.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
Message-Id: <159073589656.20809.14010247947948822435.stgit@pasha-ThinkPad-X280>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[PMD: Skip test_x86_64_pc on Travis-CI]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-06-21 13:58:52 +02:00
Pavel Dovgalyuk
c7ebab0f16 tests/acceptance: add base class record/replay kernel tests
This patch adds a base for testing kernel boot recording and replaying.
Each test has the phase of recording and phase of replaying.
Virtual machines just boot the kernel and do not interact with
the network.
Structure and image links for the tests are borrowed from boot_linux_console.py
Testing controls the message pattern at the end of the kernel
boot for both record and replay modes. In replay mode QEMU is also
intended to finish the execution automatically.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgaluk@ispras.ru>
Tested-by: Philippe Mathieu-Daude <philmd@redhat.com>
Message-Id: <159073589099.20809.14078431743098373301.stgit@pasha-ThinkPad-X280>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[PMD: Keep imports sorted alphabetically]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-06-21 13:58:45 +02:00