Commit Graph

306 Commits

Author SHA1 Message Date
Cleber Rosa
d214740c99 tests/acceptance/virtiofs_submounts.py: fix setup of SSH pubkey
The public key argument should be a path to a file, and not the
public key data.

Reported-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210412044644.55083-12-crosa@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
Cleber Rosa
fd1ce58d90 Acceptance Tests: introduce CPU hotplug test
Even though there are qtest based tests for hotplugging CPUs (from
which this test took some inspiration from), this one adds checks
from a Linux guest point of view.

It should also serve as an example for tests that follow a similar
pattern and need to interact with QEMU (via qmp) and with the Linux
guest via SSH.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Message-Id: <20210412044644.55083-11-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:21 -04:00
Cleber Rosa
a273387aec tests/acceptance/virtiofs_submounts.py: remove launch_vm()
The LinuxTest class' launch_and_wait() method now behaves the same way
as this test's custom launch_vm(), so let's just use the upper layer
(common) method.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-9-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
c6620c443d Acceptance Tests: set up SSH connection by default after boot for LinuxTest
The LinuxTest specifically targets users that need to interact with Linux
guests.  So, it makes sense to give a connection by default, and avoid
requiring it as boiler-plate code.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-8-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
d8c6a89968 Acceptance Tests: make username/password configurable
This makes the username/password used for authentication configurable,
because some guest operating systems may have restrictions on accounts
to be used for logins, and it just makes it better documented.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-7-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
54914114af Acceptance Tests: add port redirection for ssh by default
For users of the LinuxTest class, let's set up the VM with the port
redirection for SSH, instead of requiring each test to set the same
arguments.

It also sets the network device, by default, to virtio-net.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-6-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
7edee7ad94 Acceptance Tests: move useful ssh methods to base class
Both the virtiofs submounts and the linux ssh mips malta tests
contains useful methods related to ssh that deserve to be made
available to other tests.  Let's move them to an auxiliary, mix-in
class that will be used on the base LinuxTest class.

The method that helps with setting up an ssh connection will now
support both key and password based authentication, defaulting to key
based.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210412044644.55083-5-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
976218cbe7 Python: add utility function for retrieving port redirection
Slightly different versions for the same utility code are currently
present on different locations.  This unifies them all, giving
preference to the version from virtiofs_submounts.py, because of the
last tweaks added to it.

While at it, this adds a "qemu.utils" module to host the utility
function and a test.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-4-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
[Squashed in below fix. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210601154546.130870-2-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
c028691e65 tests/acceptance/virtiofs_submounts.py: evaluate string not length
If the vmlinuz variable is set to anything that evaluates to True,
then the respective arguments should be set.  If the variable contains
an empty string, than it will evaluate to False, and the extra
arguments will not be set.

This keeps the same logic, but improves readability a bit.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210412044644.55083-3-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Cleber Rosa
f084e148aa tests/acceptance/virtiofs_submounts.py: add missing accel tag
The tag is useful to select tests that depend/use a particular
feature.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Eric Auger <eric.auger@redhat.com>
Message-Id: <20210412044644.55083-2-crosa@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
2021-06-01 16:21:20 -04:00
Joel Stanley
224f010ba8 tests/acceptance: Test ast2600 machine
This tests a Debian multi-soc arm32 Linux kernel on the AST2600 based
Tacoma BMC machine.

There is no root file system so the test terminates when boot reaches
the stage where it attempts and fails to mount something.

Cc: Cleber Rosa <crosa@redhat.com>
Cc: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Tested-by: Cédric Le Goater <clg@kaod.org>
[ clg : - removed comment
        - removed ending self.vm.shutdown() ]
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20210304123951.163411-3-joel@jms.id.au>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210407171637.777743-13-clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2021-05-01 10:03:51 +02:00
Joel Stanley
a3a178c663 tests/acceptance: Test ast2400 and ast2500 machines
Test MTD images from the OpenBMC project on AST2400 and AST2500 SoCs
from ASPEED, by booting Palmetto and Romulus BMC machines.

The images are fetched from OpenBMC's release directory on github.

Cc: Cleber Rosa <crosa@redhat.com>
Cc: Wainer dos Santos Moschetta <wainersm@redhat.com>
Co-developed-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Cédric Le Goater <clg@kaod.org>
Tested-by: Cédric Le Goater <clg@kaod.org>
Signed-off-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
[ clg : - removed comment
        - removed ending self.vm.shutdown() ]
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Message-Id: <20210304123951.163411-2-joel@jms.id.au>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210407171637.777743-12-clg@kaod.org>
Signed-off-by: Cédric Le Goater <clg@kaod.org>
2021-05-01 10:03:51 +02:00
Peter Maydell
1b507e55f8 Remove many old deprecated features
The following features have been deprecated for well over the 2
 release cycle we promise
 
   ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
   ``-vnc acl`` (since 4.0.0)
   ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
   ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
   ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
   ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
   ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
   ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
   ``query-cpus`` (since 2.12.0)
   ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
   ``query-events`` (since 4.0)
   chardev client socket with ``wait`` option (since 4.0)
   ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
   ``ide-drive`` (since 4.2)
   ``scsi-disk`` (since 4.2)
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE2vOm/bJrYpEtDo4/vobrtBUQT98FAmBTHBsACgkQvobrtBUQ
 T9/6cA//QVlWFvNJjmYuXMFOkYdokqde9W66iMO2vt+p3LSqlcxVa59+ZOcb5XWc
 7xYcZlDHEHRMe8DWrTwUyjftJ7Ild+/HY5Nz8NqDFvX08umIhC6qmzBSG64O3fU5
 KtSLWe+KfbUH2usmczQO1zGG8mSIAMnOdugpLln9W13v7X9NO4yC+/O36Mm8l4VK
 VEF+o9w7+s2l7QloLdhVe7RVOrqz6+9TFHx1+q1pPhMbzlv5vbfp9vIreiCgmqiO
 rPLXB4bjklktWSkJBe/2Jt7ixNQZrT0DfnCmshHJmmBxTCI1+b798Xy+7j9Dz8ER
 LzKHhlsilCv/jb8QKlIZSxvoDE02vAgwyVXlqxc7FdyaLmg/Emwr5LGUFHKCyhDR
 3hmJafNokdWPcHwBYwRTdbG1OlgKA+yHSBzQ4OnjsTsKrrzlBrm9B9DiDVABXadS
 5NnhvjZfaRmzjXZinf1th7fdMtLs4k+9XKn6szkrp2QhtlX9gyhkXuOhdwRVcyJB
 V/8dLZDdOB3Kpbu5DGlmmIzdHQx5bo/tQqZlvJJNmcOWe+TKwHg56WRUYQ3PFQZK
 M2A4XTd5kil6+PT7AnDHLoYAFdfScq6R32V38AI7S34kPrejftbSbDz8EVRXnrzj
 zzXGNWtSQpIDZGgNEg3Ta6aIf28TxLCA7nRhuPYIDcswlUsHn4c=
 =cFRH
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange-gitlab/tags/dep-many-pull-request' into staging

Remove many old deprecated features

The following features have been deprecated for well over the 2
release cycle we promise

  ``-drive file=json:{...{'driver':'file'}}`` (since 3.0)
  ``-vnc acl`` (since 4.0.0)
  ``-mon ...,control=readline,pretty=on|off`` (since 4.1)
  ``migrate_set_downtime`` and ``migrate_set_speed`` (since 2.8.0)
  ``query-named-block-nodes`` result ``encryption_key_missing`` (since 2.10.0)
  ``query-block`` result ``inserted.encryption_key_missing`` (since 2.10.0)
  ``migrate-set-cache-size`` and ``query-migrate-cache-size`` (since 2.11.0)
  ``query-named-block-nodes`` and ``query-block`` result dirty-bitmaps[i].status (since 4.0)
  ``query-cpus`` (since 2.12.0)
  ``query-cpus-fast`` ``arch`` output member (since 3.0.0)
  ``query-events`` (since 4.0)
  chardev client socket with ``wait`` option (since 4.0)
  ``acl_show``, ``acl_reset``, ``acl_policy``, ``acl_add``, ``acl_remove`` (since 4.0.0)
  ``ide-drive`` (since 4.2)
  ``scsi-disk`` (since 4.2)

# gpg: Signature made Thu 18 Mar 2021 09:23:39 GMT
# gpg:                using RSA key DAF3A6FDB26B62912D0E8E3FBE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg:                 aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E  8E3F BE86 EBB4 1510 4FDF

* remotes/berrange-gitlab/tags/dep-many-pull-request:
  block: remove support for using "file" driver with block/char devices
  block: remove 'dirty-bitmaps' field from 'BlockInfo' struct
  block: remove dirty bitmaps 'status' field
  block: remove 'encryption_key_missing' flag from QAPI
  hw/scsi: remove 'scsi-disk' device
  hw/ide: remove 'ide-drive' device
  chardev: reject use of 'wait' flag for socket client chardevs
  machine: remove 'arch' field from 'query-cpus-fast' QMP command
  machine: remove 'query-cpus' QMP command
  migrate: remove QMP/HMP commands for speed, downtime and cache size
  monitor: remove 'query-events' QMP command
  monitor: raise error when 'pretty' option is used with HMP
  ui, monitor: remove deprecated VNC ACL option and HMP commands

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-18 19:00:50 +00:00
Daniel P. Berrangé
8af54b9172 machine: remove 'query-cpus' QMP command
The newer 'query-cpus-fast' command avoids side effects on the guest
execution. Note that some of the field names are different in the
'query-cpus-fast' command.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
2021-03-18 09:22:55 +00:00
Pavel Dovgalyuk
affc55e761 tests/acceptance: linux-related tests fix
This patch allows cloudinit images download when ssh
key is not specified.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Message-Id: <161373266228.1608713.7614311331725780044.stgit@pasha-ThinkPad-X280>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-03-16 22:24:44 -04:00
Jagannathan Raman
9afa1766dd tests: Add functional test for out-of-process device emulation
Runs the Avocado acceptance test to check if a
remote lsi53c895a device gets identified by the guest.

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[WR: Refactored code]
Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210303205320.146047-3-willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-03-16 22:18:27 -04:00
Willian Rampazzo
cd093d5a2c avocado_qemu: add exec_command function
Sometimes a test needs to send a command to a console without waiting
for a pattern as a result, or the command issued do not produce any kind
of output, like, for example, a `mount` command.

This introduces the `exec_command` function to the avocado_qemu,
allowing the test to send a command to the console without the need to
match a pattern produced as a result.

Signed-off-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210303205320.146047-2-willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-03-16 22:17:33 -04:00
Wainer dos Santos Moschetta
1cf54443d9 tests/acceptance: Print expected message on wait_for_console_pattern
For the sake of improve debuggability of tests which use the
wait_for_console_pattern(), this changed the _console_interaction() so that
the expected message is printed if the test fail.

Signed-off-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210309153507.1905682-1-wainersm@redhat.com>
Reviewed-by: Joel Stanley <joel@jms.id.au>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-03-16 20:54:35 -04:00
Niek Linnenbank
f4223d2e75 tests/acceptance: drop ARMBIAN_ARTIFACTS_CACHED condition for orangepi-pc, cubieboard tests
Previously the ARMBIAN_ARTIFACTS_CACHED pre-condition was added to allow running
tests that have already existing armbian.com artifacts stored in the local avocado cache,
but do not have working URLs to download a fresh copy.

At this time of writing the URLs for artifacts on the armbian.com server are updated and working.
Any future broken URLs will result in a skipped acceptance test, for example:

 (1/5) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:
  CANCEL: Missing asset https://apt.armbian.com/pool/main/l/linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb (0.53 s)

This commits removes the ARMBIAN_ARTIFACTS_CACHED pre-condition such that
the acceptance tests for the orangepi-pc and cubieboard machines can run.

Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-id: 20210310195820.21950-6-nieklinnenbank@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-12 12:40:11 +00:00
Niek Linnenbank
e384db41d8 tests/acceptance: update sunxi kernel from armbian to 5.10.16
The linux kernel 4.20.7 binary for sunxi has been removed from apt.armbian.com:

  $ ARMBIAN_ARTIFACTS_CACHED=yes AVOCADO_ALLOW_LARGE_STORAGE=yes avocado --show=app,console run -t machine:orangepi-pc tests/acceptance/boot_linux_console.py
  Fetching asset from tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi
  ...
  (1/6) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_arm_orangepi:
    CANCEL: Missing asset https://apt.armbian.com/pool/main/l/linux-4.20.7-sunxi/linux-image-dev-sunxi_5.75_armhf.deb (0.55 s)

This commit updates the sunxi kernel to 5.10.16 for the acceptance
tests of the orangepi-pc and cubieboard machines.

Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-id: 20210310195820.21950-5-nieklinnenbank@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-12 12:40:11 +00:00
Niek Linnenbank
875ee4179b tests/acceptance/boot_linux_console: change URL for test_arm_orangepi_bionic_20_08
Update the download URL of the Armbian 20.08 Bionic image for
test_arm_orangepi_bionic_20_08 of the orangepi-pc machine.

The archive.armbian.com URL contains more images and should keep stable
for a longer period of time than dl.armbian.com.

Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-id: 20210310195820.21950-4-nieklinnenbank@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-12 12:40:10 +00:00
Niek Linnenbank
a30e114f3c tests/acceptance/boot_linux_console: remove Armbian 19.11.3 bionic test for orangepi-pc machine
The image for Armbian 19.11.3 bionic has been removed from the armbian server.
Without the image as input the test arm_orangepi_bionic_19_11 cannot run.

This commit removes the test completely and merges the code of the generic function
do_test_arm_orangepi_uboot_armbian back with the 20.08 test.

Signed-off-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-id: 20210310195820.21950-3-nieklinnenbank@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-03-12 12:40:10 +00:00
Alex Bennée
2ceb7c03a2 tests/avocado: add boot_xen tests
These tests make sure we can boot the Xen hypervisor with a Dom0
kernel using the guest-loader. We currently have to use a kernel I
built myself because there are issues using the Debian kernel images.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210303173642.3805-8-alex.bennee@linaro.org>
2021-03-10 15:34:12 +00:00
Alex Bennée
d98946450d tests/acceptance: allow a "graceful" failing for virtio-gpu test
This is a band-aid with a TODO for cases when QEMU doesn't start due
to missing VirGL. Longer term we could do with some proper feature
probing.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20210222101455.12640-4-alex.bennee@linaro.org>
2021-02-24 11:05:14 +00:00
Jiaxun Yang
c0928e6ddc tests/acceptance: Test PMON with Loongson-3A1000 CPU
Test booting of PMON bootloader on loongson3-virt platform.

$ (venv) AVOCADO_ALLOW_UNTRUSTED_CODE=1 \
    avocado --show=app,console \
      run -t machine:loongson3-virt tests/acceptance
Fetching asset from tests/acceptance/machine_mips_loongson3v.py:MipsLoongson3v.test_pmon_serial_console
JOB ID     : 8e202b3727847c9104d0d3d6546ed225d35f6706
JOB LOG    : /home/flygoat/avocado/job-results/job-2021-01-12T10.02-8e202b3/job.log
 (1/1) tests/acceptance/machine_mips_loongson3v.py:MipsLoongson3v.test_pmon_serial_console:
console: PMON2000 MIPS Initializing. Standby...
console: 00000000Jump to 9fc
console: Init Memory done.
console: The uncache data is:
console: 00000000:  5555555555555555
console: 00000008:  aaaaaaaaaaaaaaaa
console: 00000010:  3333333333333333
console: 00000018:  cccccccccccccccc
console: 00000020:  7777777777777777
console: 00000028:  8888888888888888
console: 00000030:  1111111111111111
console: 00000038:  eeeeeeeeeeeeeeee
console: The cached  data is:
console: 00000000:  5555555555555555
console: 00000008:  aaaaaaaaaaaaaaaa
console: 00000010:  3333333333333333
console: 00000018:  cccccccccccccccc
console: 00000020:  7777777777777777
console: 00000028:  8888888888888888
console: 00000030:  1111111111111111
console: 00000038:  eeeeeeeeeeeeeeee
console: Copy PMON to execute location...
console: start = 0x8f900000
console: s0 = 0x30300000
console: _edata = 0x8f989010
console: _end = 0x8f98a028copy text section done.
console: Copy PMON to execute location done.
console: sp=8f8fc000
console: Uncompressing Bios............................................................................OK,Booting Bios
console: FREQ
console: DONE
console: DEVI
console: ENVI
console: MAPV
console: NVRAM@8f7ff898
console: STDV
console: 80100000:  memory between 8f7ff400-8f800000  is already been allocated,heap is already above this point
console: SBDD
console: P12PCIH
console: PCIH
console: PCID
console: setting up 1 bus
console: PCI bus 0 slot 1: probe...completed
console: PCI bus 0 slot 1/0: vendor/product: 0x106b/0x003f (serialbus, USB, interface: 0x10, revision: 0x00)
console: PCI bus 0 slot 1/0: reg 0x10 = 0xffffff00
console: PCI bus 0 slot 2: probe...completed
console: PCI bus 0 slot 2/0: vendor/product: 0x1af4/0x1000 (network, ethernet, interface: 0x00, revision: 0x00)
console: PCI bus 0 slot 2/0: reg 0x10 = 0xffffffe1
console: PCI bus 0 slot 2/0: reg 0x14 = 0xfffff000
console: PCI bus 0 slot 2/0: reg 0x20 = 0xffffc00c
console: PCI bus 0 slot 2/0: reg 0x30 = 0xfffc0000
console: PCI bus 0 slot 3: probe...completed
console: PCI bus 0 slot 4: probe...completed
console: PCI bus 0 slot 5: probe...completed
console: PCI bus 0 slot 6: probe...completed
console: PCI bus 0 slot 7: probe...completed
console: PCI bus 0 slot 8: probe...completed
console: PCI bus 0 slot 9: probe...completed
console: PCI bus 0 slot 10: probe...completed
console: PCI bus 0 slot 11: probe...completed
console: PCI bus 0 slot 12: probe...completed
console: PCI bus 0 slot 13: probe...completed
console: PCI bus 0 slot 14: probe...completed
console: PCI bus 0 slot 15: probe...completed
console: PCI bus 0 slot 16: probe...completed
console: PCI bus 0 slot 17: probe...completed
console: PCI bus 0 slot 18: probe...completed
console: PCI bus 0 slot 19: probe...completed
console: PCI bus 0 slot 20: probe...completed
console: PCI bus 0 slot 21: probe...completed
console: PCI bus 0 slot 22: probe...completed
console: PCI bus 0 slot 23: probe...completed
console: PCI bus 0 slot 24: probe...completed
console: PCI bus 0 slot 25: probe...completed
console: PCI bus 0 slot 26: probe...completed
console: PCI bus 0 slot 27: probe...completed
console: PCI bus 0 slot 28: probe...completed
console: PCI bus 0 slot 29: probe...completed
console: PCI bus 0 slot 30: probe...completed
console: PCI bus 0 slot 31: probe...completed
console: PCIS
console: PCIR
console: PCIW
console: PCI bus 0 slot 2/0: mem @0x40000000, reg 0x30 262144 bytes
console: PCI bus 0 slot 2/0: mem @0x40040000, reg 0x20 16384 bytes
console: PCI bus 0 slot 2/0: mem @0x40044000, reg 0x14 4096 bytes
console: PCI bus 0 slot 1/0: mem @0x40045000, reg 0x10 256 bytes
console: PCI bus 0 slot 2/0: exp @0x40000000, 262144 bytes
console: PCI bus 0 slot 2/0: i/o @0x00004000, reg 0x10 32 bytes
console: NETI
console: RTCL
console: PCID
console: VGAI
console: memorysize=c000000,base=8f6ff508,sysMem=8f6ef500
console: in setup_int_vect!done!VESA
console: vga bios init failed, rc=-1
console: in configure
console: mainbus0 (root)
console: localbus0 at mainbus0
console: loopdev0 at mainbus0pcibr0 at mainbus0
console: pci0 at pcibr0 bus 0
console: ohci0 at pci0 dev 1 function 0 vendor/product: 0x106b/0x003f (serialbus, USB, interface: 0x10, revision: 0x00)usb base addr : 0xc0045000, bus_base is : 0xc0000000
console: OHCI revision: 0x00000010
console: RH: a: 0x00000203 b: 0x00000000
console: early period(0x0)
console: OHCI 8c01ec00 initialized ok
console: New Device 0
console: usb_get_descriptor
console: bLength = 12
console: bDescriptorType =1
console: bcdUSB =  110
console: bDeviceClass =9
console: bDeviceSubClass =0
console: bDeviceProtocol =0
console: bMaxPacketSize0 =8
console: set address 1
console: usb_get_descriptor
console: bLength = 12
console: bDescriptorType =1
console: bcdUSB =  110
console: bDeviceClass =9
console: bDeviceSubClass =0
console: bDeviceProtocol =0
console: bMaxPacketSize0 =8
console: idVendor =0
console: idProduct =0
console: bcdDevice =0
console: iManufacturer=0
console: iProduct =1
console: iSerialNumber=0
console: bNumConfigurations=1
console: usb_get_descriptor
console: usb_get_descriptor
console: get_conf_no 0 Result 25, wLength 25
console: if 0, ep 0
console: bLength=9
console: bDescriptorType=2
console: wTotalLength=19
console: bNumInterfaces=1
console: bConfigurationValue=1
console: iConfiguration=0
console: bmAttributes=40
console: MaxPower=0
console: 09 04 00 00 01 09 00 00 00 07 05 81 03 02 00 ff
console: ##EP epmaxpacketin[1] = 2
console: set configuration 1
console: new device strings: Mfr=0, Product=1, SerialNumber=0
console: USB device number 1 default language ID 0x409
console: Manufacturer
console: Product      OHCI Root Hub
console: SerialNumber
console: New Device 1
console: usb_get_descriptor
console: bLength = 12
console: bDescriptorType =1
console: bcdUSB =  200
console: bDeviceClass =0
console: bDeviceSubClass =0
console: bDeviceProtocol =0
console: bMaxPacketSize0 =8
console: set address 2
console: usb_get_descriptor
console: bLength = 12
console: bDescriptorType =1
console: bcdUSB =  200
console: bDeviceClass =0
console: bDeviceSubClass =0
console: bDeviceProtocol =0
console: bMaxPacketSize0 =8
console: idVendor =627
console: idProduct =1
console: bcdDevice =0
console: iManufacturer=1
console: iProduct =4
console: iSerialNumber=b
console: bNumConfigurations=1
console: usb_get_descriptor
console: usb_get_descriptor
console: get_conf_no 0 Result 34, wLength 34
console: unknown Description Type : 21
console: 09 21 11 01 00 01 22 3F 00
console: if 0, ep 0
console: bLength=9
console: bDescriptorType=2
console: wTotalLength=22
console: bNumInterfaces=1
console: bConfigurationValue=1
console: iConfiguration=8
console: bmAttributes=a0
console: MaxPower=32
console: 09 04 00 00 01 03 01 01 00 09 21 11 01 00 01 22 3f 00 07 05 81 03 08 00 0a
console: ##EP epmaxpacketin[1] = 8
console: set configuration 1
console: new device strings: Mfr=1, Product=4, SerialNumber=11
console: USB device number 2 default language ID 0x409
console: Manufacturer QEMU
console: Product      QEMU USB Keyboard
console: SerialNumber 68284-0000:00:01.0-1
console: drive at ohci0 devnum 2, Product QEMU USB Keyboard
console: not configured
console: New Device 2
console: usb_get_descriptor
console: bLength = 12
console: bDescriptorType =1
console: bcdUSB =  200
console: bDeviceClass =0
console: bDeviceSubClass =0
console: bDeviceProtocol =0
console: bMaxPacketSize0 =8
console: set address 3
console: usb_get_descriptor
console: bLength = 12
console: bDescriptorType =1
console: bcdUSB =  200
console: bDeviceClass =0
console: bDeviceSubClass =0
console: bDeviceProtocol =0
console: bMaxPacketSize0 =8
console: idVendor =627
console: idProduct =1
console: bcdDevice =0
console: iManufacturer=1
console: iProduct =3
console: iSerialNumber=a
console: bNumConfigurations=1
console: usb_get_descriptor
console: usb_get_descriptor
console: get_conf_no 0 Result 34, wLength 34
console: unknown Description Type : 21
console: 09 21 01 00 00 01 22 4A 00
console: if 0, ep 0
console: bLength=9
console: bDescriptorType=2
console: wTotalLength=22
console: bNumInterfaces=1
console: bConfigurationValue=1
console: iConfiguration=7
console: bmAttributes=a0
console: MaxPower=32
console: 09 04 00 00 01 03 00 00 00 09 21 01 00 00 01 22 4a 00 07 05 81 03 08 00 0a
console: ##EP epmaxpacketin[1] = 8
console: set configuration 1
console: new device strings: Mfr=1, Product=3, SerialNumber=10
console: USB device number 3 default language ID 0x409
console: Manufacturer QEMU
console: Product      QEMU USB Tablet
console: SerialNumber 28754-0000:00:01.0-2
console: drive at ohci0 devnum 3, Product QEMU USB Tablet
console: not configured
console: drive at ohci0 devnum 1, Product OHCI Root Hub
console: not configured
console: vendor/product: 0x1af4/0x1000 (network, ethernet, interface: 0x00, revision: 0x00) at pci0 dev 2 function 0 not configured
console: out configure
console: Press <Del> to set BIOS,waiting for 3 seconds here.....
console: devconfig done.
console: ifinit done.
console: domaininit done.
console: init_proc....
console: HSTI
console: SYMI
console: SBDE
console: [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[
console: [[  [[[[[[[[[       [[[[[       [[[[   [[[[[  [[[[[      [[[[[       [[[[[       [[[[   [[[[[  [[
console: [[  [[[[[[[[   [[[[  [[[   [[[[  [[[    [[[[  [[[[  [[[[  [[[   [[[[  [[[   [[[[  [[[    [[[[  [[
console: [[  [[[[[[[[  [[[[[[ [[[  [[[[[[ [[[  [  [[[  [[[  [[[[[[[[[[[[   [[[[[[[  [[[[[[ [[[  [  [[[  [[
console: [[  [[[[[[[[  [[[[[[ [[[  [[[[[[ [[[  [[  [[  [[[  [[[    [[[[[[[    [[[[  [[[[[[ [[[  [[  [[  [[
console: [[  [[[[[[[[  [[[[[[ [[[  [[[[[[ [[[  [[[  [  [[[  [[[[[  [[[[[[[[[[  [[[  [[[[[[ [[[  [[[  [  [[
console: [[  [[[[[[[[   [[[[  [[[   [[[[  [[[  [[[[    [[[   [[[[  [[[   [[[  [[[[   [[[[  [[[  [[[[    [[
console: [[       [[[[       [[[[[       [[[[  [[[[[   [[[[       [[[[[      [[[[[[       [[[[  [[[[[   [[
console: [[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[2011 Loongson][[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[
console: Configuration [Bonito,EL,NET,SCSI,IDE]
console: Version: PMON2000 3.3 (Bonito) #0: Tue Dec 22 01:58:09 UTC 2020 commit b3ece66234adbf7d4e453f0ba4f326c099ac2a76 Author: Jiaxun Yang <jiaxun.yang@flygoat.com> Date:   Tue Dec 22 09:51:10 2020 +0800 .
console: Supported loaders [txt, srec, elf, bin]
console: Supported filesystems [net, fat, fs, disk, iso9660, socket, tty, ram]
console: This software may be redistributed under the BSD copyright.
console: Copyright 2000-2002, Opsycon AB, Sweden.
console: Copyright 2005, ICT CAS.
console: CPU GODSON3 BogoMIPS: 1327
PASS (3.89 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME   : 4.38 s

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210112020708.62922-1-jiaxun.yang@flygoat.com>
[PMD: Set timeout to 60sec, simply test for ''CPU GODSON3 BogoMIPS']
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-02-21 19:42:33 +01:00
Alex Bennée
df55e2a701 tests/acceptance: add a memory callback check
This test makes sure that the inline and callback based memory checks
count the same number of accesses.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210213130325.14781-24-alex.bennee@linaro.org>
2021-02-18 08:19:23 +00:00
Alex Bennée
36fc4a2fa4 tests/acceptance: add a new tests to detect counting errors
The insn plugin has a simple heuristic to detect if an instruction is
detected running twice in a row. Check the plugin log after the run
and pass accordingly.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210213130325.14781-22-alex.bennee@linaro.org>
2021-02-18 08:19:23 +00:00
Alex Bennée
4c134d07b9 tests/acceptance: add a new set of tests to exercise plugins
This is just a simple test to count the instructions executed by a
kernel. However a later test will detect a failure condition when
icount is enabled.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20210213130325.14781-16-alex.bennee@linaro.org>
2021-02-18 08:19:23 +00:00
Cleber Rosa
c0c5a7f18e Acceptance Tests: set up existing ssh keys by default
It's questionable whether it's necessary to create one brand new pair
for each test.  It's not questionable that it takes less time and
resources to just use the keys available at "tests/keys" that exist
for that exact reason.

If a location for the public key is not given explicitly, the
LinuxTest will now set up the existing pair of keys as the default.
This removes the need for a lot of boilerplate code.

To avoid the ssh client from erroring on permission issues, a
directory with restrictive permissions is created for the private key.
This should still be a lot cheaper than creating a new key.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210203172357.1422425-19-crosa@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
[marcandre: fix typos in commit message]
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15 22:30:06 -05:00
Cleber Rosa
e8197c6e0c Acceptance Tests: fix population of public key in cloudinit image
Currently the path of the ssh public key is being set, but its
content is obviously what's needed.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210203172357.1422425-18-crosa@redhat.com>
Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15 22:28:15 -05:00
Cleber Rosa
efe30d5011 Acceptance Tests: introduce method for requiring an accelerator
Some tests explicitly require a QEMU accelerator to be available.
Given that this depends on some runtime aspects not known before
the test is started, such as the currently set QEMU binary, it's
left to be checked also at runtime.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210203172357.1422425-17-crosa@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15 22:23:40 -05:00
Cleber Rosa
6dd7457916 Acceptance Tests: introduce LinuxTest base class
This is basically the infrastructure around "boot_linux.py" tests, but
now made into a base class for general use.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20210203172357.1422425-15-crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15 22:17:25 -05:00
Cleber Rosa
cdcc50bf6e tests/acceptance/virtio-gpu.py: preserve virtio-user-gpu log
Preserve log at location already prepared for keeping the test's log
files.

While at it, log info about its location (in the main test log
file), instead of printing it out.

Reference: https://avocado-framework.readthedocs.io/en/85.0/api/test/avocado.html#avocado.Test.logdir
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
[philmd: use full sentence]
Message-Id: <20210211220146.2525771-7-crosa@redhat.com>

Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15 21:45:07 -05:00
Max Reitz
3e5c180be0 virtiofs_submounts.py test: Note on vmlinuz param
From the cancel message, it is not entirely clear why this parameter is
mandatory now, or that it will be optional in the future.  Add such a
more detailed explanation as a comment in the test source file.

Suggested-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20210212151649.252440-1-mreitz@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Cleber Rosa <crosa@redhat.com>
2021-02-15 19:07:23 -05:00
Peter Maydell
34b7d4193e Integration testing patches
Tests added:
 - Armbian 20.08 on Orange Pi PC (Philippe)
 - MPC8544ds machine (Thomas)
 - Virtex-ml507 ppc machine (Thomas)
 - Re-enable the microblaze test (Thomas)
 
 Various fixes and documentation improvements from Cleber.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAmAhnMAACgkQ4+MsLN6t
 wN4TuA/+Lsk5Sxpe5+3eEaXUAo0v9jHm0Nqh+fbvf/aGkcIbNh0Un3eNpLiv44vE
 vdJlFsfWvwomHylyjPnINBnBjbdIHCC0KdHl6k1YEHVONWpL8t0LyT/eQNXnH4lO
 I6m1uhqR6L5bBIl8gyaQ4PNvDcKtADwqCQUSjuMpSjLzuiwbiwI4Jw137blq9pEm
 zf+LEsD+AZ0zWA1q1bOkRkHDEXaorcUuPGv/es7Xy3tELoXTEBNLEOcu/hNcwimE
 C+X6fyJPCkUxicCRN4X29MEXUgBJin070f/0fxUO3BGpnx1N2lSK5uJwUQ1FilSP
 yPZ6wu9+RakvABuuWlO/mKRR6y9lPhTM2o1niL/hqrVN9DaIysFNYGXosJZ7Cmwq
 KTwcU8fWBT6/66L+4cKxRO7iUHRHeedu+tIqXEjMHkpBpMe7mrVVmcORp8i12Eca
 JVCpjedrX+7X/tWqUKSf63UJrTSFdsdUsdlxWd6uHl5PISJkMWwPsOKxHdMmASqr
 dJmSmCSZ/Gz6+LrmJcB/5/on6NjkiuQ/wFJIkFF1ekhwXJ1Pg/POd1yB7GJd3GZJ
 6XzB2gwNVn3lcuL6kBLHeInUXqDvNAApuLfjHprWQ5N469hCpbXobSqP7Bmvu9Ei
 I56V6EOeLZYb3MYBhMGETz7mJ+zsanYisBPO7gL0ApuPwTQDNdk=
 =mUpx
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/integration-testing-20210208' into staging

Integration testing patches

Tests added:
- Armbian 20.08 on Orange Pi PC (Philippe)
- MPC8544ds machine (Thomas)
- Virtex-ml507 ppc machine (Thomas)
- Re-enable the microblaze test (Thomas)

Various fixes and documentation improvements from Cleber.

# gpg: Signature made Mon 08 Feb 2021 20:19:12 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/integration-testing-20210208:
  Acceptance Tests: remove unnecessary tag from documentation example
  Acceptance tests: clarify ssh connection failure reason
  tests/acceptance/virtiofs_submounts: required space between IP and port
  tests/acceptance/virtiofs_submounts: standardize port as integer
  tests/acceptance/virtiofs_submounts: use a virtio-net device instead
  tests/acceptance/virtiofs_submounts: do not ask for ssh key password
  tests/acceptance/virtiofs_submounts: use workdir property
  tests/acceptance/boot_linux: rename misleading cloudinit method
  tests/acceptance/boot_linux: fix typo on cloudinit error message
  tests/acceptance: Re-enable the microblaze test
  tests/acceptance: Add a test for the virtex-ml507 ppc machine
  tests/acceptance: Test the mpc8544ds machine
  tests/acceptance: Move the pseries test to a separate file
  tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC
  tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() method
  tests/acceptance: Introduce tesseract_ocr() helper
  tests/acceptance: Extract tesseract_available() helper in new namespace

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2021-02-08 20:22:54 +00:00
Cleber Rosa
4f0d032185 Acceptance tests: clarify ssh connection failure reason
If the connection to the ssh server fails, it may indeed be a "sshd"
issue, but it may also not be that.  Let's state what we know: the
establishment of the connection from the client side was not possible.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210203172357.1422425-13-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Cleber Rosa
d7f57c25e1 tests/acceptance/virtiofs_submounts: required space between IP and port
AFAICT, there should not be a situation where IP and port do not have
at least one whitespace character separating them.

This may be true for other '\s*' patterns in the same regex too.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210203172357.1422425-10-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Cleber Rosa
c0d1681e50 tests/acceptance/virtiofs_submounts: standardize port as integer
Instead of having to cast it whenever it's going to be used, let's
standardize it as an integer, which is the data type that will be
used most often.

Given that the regex will only match digits, it's safe that we'll
end up getting a integer, but, it could as well be a zero.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Message-Id: <20210203172357.1422425-9-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Cleber Rosa
14a79778e1 tests/acceptance/virtiofs_submounts: use a virtio-net device instead
In a virtiofs based tests, it seems safe to assume that the guest will
be capable of a virtio-net device.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210203172357.1422425-7-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Cleber Rosa
c70a6d1b9f tests/acceptance/virtiofs_submounts: do not ask for ssh key password
Tests are supposed to be non-interactive, and ssh-keygen is asking for
a passphrase when creating a key.  Let's set an empty passphrase to
avoid the prompt.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210203172357.1422425-6-crosa@redhat.com>
[PMD: Reword description per Alex Bennée comment]
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Cleber Rosa
fef453ee18 tests/acceptance/virtiofs_submounts: use workdir property
For Avocado Instrumented based tests, it's a better idea to just use
the property.  The environment variable is a fall back for tests not
written using that Python API.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reference: https://avocado-framework.readthedocs.io/en/84.0/api/test/avocado.html#avocado.Test.workdir
Message-Id: <20210203172357.1422425-5-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Cleber Rosa
b5a86c482d tests/acceptance/boot_linux: rename misleading cloudinit method
There's no downloading happening on that method, so let's call it
"prepare" instead.  While at it, and because of it, the current
"prepare_boot" and "prepare_cloudinit" are also renamed.

The reasoning here is that "prepare_" methods will just work on the
images, while "set_up_" will make them effective to the VM that will
be launched.  Inspiration comes from the "virtiofs_submounts.py"
tests, which this expects to converge more into.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210203172357.1422425-3-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Cleber Rosa
834736c9d1 tests/acceptance/boot_linux: fix typo on cloudinit error message
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Beraldo Leal <bleal@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20210203172357.1422425-2-crosa@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Thomas Huth
ee6c14d016 tests/acceptance: Re-enable the microblaze test
The microblaze kernel sometimes gets stuck during boot (ca. 1 out of 200
times), so we disabled the corresponding acceptance tests some months
ago. However, it's likely better to check that the kernel is still
starting than to not testing it at all anymore. Move the test to
a separate file, enable it again and check for an earlier console
message that should always appear.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20210128152815.585478-1-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 17:22:36 +01:00
Thomas Huth
299ab1b0ce tests/acceptance: Add a test for the virtex-ml507 ppc machine
The "And a hippo new year" image from the QEMU advent calendar 2020
can be used to test the virtex-ml507 ppc machine.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <20210112164045.98565-4-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 12:41:03 +01:00
Thomas Huth
46c647e69d tests/acceptance: Test the mpc8544ds machine
We can use the "Stupid creek" image to test the mpc8544ds ppc machine.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <20210112164045.98565-3-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 12:41:03 +01:00
Thomas Huth
c592f70cae tests/acceptance: Move the pseries test to a separate file
Let's gather the POWER-related tests in a separate file.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: David Gibson <david@gibson.dropbear.id.au>
Message-Id: <20210112164045.98565-2-thuth@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 12:41:02 +01:00
Philippe Mathieu-Daudé
093aac4ab2 tests/acceptance: Test U-Boot/Linux from Armbian 20.08 on Orange Pi PC
Test U-Boot and Linux on the recent Armbian release 20.08.

Suggested-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Tested-by: Bin Meng <bin.meng@windriver.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201023131808.3198005-5-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 12:40:56 +01:00
Philippe Mathieu-Daudé
108a76da76 tests/acceptance: Extract do_test_arm_orangepi_armbian_uboot() method
As we want to reuse the same U-Boot test for multiple
Armbian releases, extract the common part as
do_test_arm_orangepi_armbian_uboot().

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Bin Meng <bin.meng@windriver.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-Id: <20201023131808.3198005-4-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 12:40:49 +01:00
Philippe Mathieu-Daudé
ca82244928 tests/acceptance: Introduce tesseract_ocr() helper
We are going to reuse the tesseract OCR code.
Create a new tesseract_ocr() helper and use it.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-5-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 12:37:33 +01:00
Philippe Mathieu-Daudé
162127f29f tests/acceptance: Extract tesseract_available() helper in new namespace
We are going to reuse tesseract_available(). Extract it to
a new 'tesseract_utils' namespace.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201021105035.2477784-4-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2021-02-08 12:37:33 +01:00
Thomas Huth
36a7ab5f04 tests/acceptance: Increase the timeout in the replay tests
Our gitlab-CI just showed a failed test_ppc_mac99 since it was apparently
killed some few seconds before the test finished. Allow it some more
time to complete.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Acked-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210127065222.48650-1-thuth@redhat.com>
2021-02-08 09:40:51 +00:00
Marc-André Lureau
73240f104d tests: add some virtio-gpu & vhost-user-gpu acceptance test
This will check virtio/vhost-user-vga & virgl are correctly initialized
by the Linux kernel on an egl-headless display.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Message-Id: <20210204105232.834642-21-marcandre.lureau@redhat.com>
Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
2021-02-04 15:58:54 +01:00
Paolo Bonzini
3e3267a9d7 acceptance: switch to QMP change-vnc-password command
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2021-01-23 15:55:07 -05:00
Thomas Huth
333168efe5 tests/acceptance: Fix race conditions in s390x tests & skip fedora on gitlab-CI
There was a race condition in the first test where there was already the
"crw" output in the dmesg, but the "0.0.4711" entry has not been created
in the /sys fs yet. Fix it by waiting until it is there.

The second test has even more problems on gitlab-CI. Even after adding some
more synchronization points (that wait for some messages in the "dmesg"
output to make sure that the modules got loaded correctly), there are still
occasionally some hangs in this test when it is running in the gitlab-CI.
So far I was unable to reproduce these hangs locally on my computer, so
this issue might take a while to debug. Thus disable the 2nd test in the
gitlab-CI until the problems are better understood and fixed.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20210108185645.86351-1-thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2021-01-11 14:59:21 +01:00
Alex Bennée
93eaabde61 tests/acceptance: bound the size of readline in s390_ccw_virtio
The read binary data as text via a PPM export of the frame buffer
seems a bit sketchy and it did blow up in the real world when the
assertion failed:

  https://gitlab.com/qemu-project/qemu/-/jobs/943183183

However short of cleaning up the test to be more binary focused at
least limit the attempt to dump the whole file as hexified zeros in
the logs.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Daniel P. Berrangé <berrange@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Halil Pasic <pasic@linux.ibm.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20210105124405.15424-1-alex.bennee@linaro.org>
2021-01-06 17:30:02 +00:00
Jiaxun Yang
4570272987 tests/acceptance: Test boot_linux_console for fuloong2e
The kernel comes from debian archive so it's trusted.

Invoking the test can be done as follows:

  $ avocado --show=app,console run -t machine:fuloong2e tests/acceptance/
   (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_mips64el_fuloong2e:
  console: [    0.000000] Initializing cgroup subsys cpuset
  console: [    0.000000] Initializing cgroup subsys cpu
  console: [    0.000000] Initializing cgroup subsys cpuacct
  console: [    0.000000] Linux version 3.16.0-6-loongson-2e (debian-kernel@lists.debian.org) (gcc version 4.8.4 (Debian 4.8.4-1) ) #1 Debian 3.16.56-1+deb8u1 (2018-05-08)
  console: [    0.000000] memsize=256, highmemsize=0
  console: [    0.000000] CpuClock = 533080000
  console: [    0.000000] bootconsole [early0] enabled
  console: [    0.000000] CPU0 revision is: 00006302 (ICT Loongson-2)
  console: [    0.000000] FPU revision is: 00000501
  console: [    0.000000] Checking for the multiply/shift bug... no.
  console: [    0.000000] Checking for the daddiu bug... no.
  console: [    0.000000] Determined physical RAM map:
  console: [    0.000000]  memory: 0000000010000000 @ 0000000000000000 (usable)
  console: [    0.000000]  memory: 0000000004000000 @ 0000000010000000 (reserved)
  console: [    0.000000]  memory: 0000000003ffffff @ 000000001c000001 (reserved)
  console: [    0.000000] Initrd not found or empty - disabling initrd
  console: [    0.000000] Zone ranges:
  console: [    0.000000]   DMA      [mem 0x00000000-0x00ffffff]
  console: [    0.000000]   Normal   [mem 0x01000000-0x0fffffff]
  console: [    0.000000] Movable zone start for each node
  console: [    0.000000] Early memory node ranges
  console: [    0.000000]   node   0: [mem 0x00000000-0x0fffffff]
  console: [    0.000000] Reserving 0MB of memory at 0MB for crashkernel
  console: [    0.000000] Primary instruction cache 64kB, VIPT, direct mapped, linesize 32 bytes.
  console: [    0.000000] Primary data cache 64kB, 4-way, VIPT, no aliases, linesize 32 bytes
  console: [    0.000000] Unified secondary cache 512kB 4-way, linesize 32 bytes.
  console: [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 16327
  console: [    0.000000] Kernel command line: printk.time=0 console=ttyS0
  PASS (2.27 s)

Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Huacai Chen <chenhuacai@kernel.org>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201224031750.52146-9-jiaxun.yang@flygoat.com>
[PMD: Added command line example]
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2021-01-04 23:36:03 +01:00
Thomas Huth
24bfaae139 tests/acceptance: Add a test with the Fedora 31 kernel and initrd
This initrd contains a virtio-net and a virtio-gpu kernel module,
so we can check that we can set a MAC address for the network device
and whether we can hot-plug and -unplug a virtio-crypto device.
But the most interesting part is maybe that we can also successfully
write some stuff into the emulated framebuffer of the virtio-gpu
device and make sure that we can read back that data from a screenshot.

Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201221143423.23607-1-thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2020-12-22 09:23:09 +01:00
Thomas Huth
d986bc4a1c tests/acceptance: Test the virtio-balloon device on s390x
Inflate the balloon and check whether the size of the memory changes.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201215183623.110128-4-thuth@redhat.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2020-12-21 18:11:33 +01:00
Thomas Huth
7e549424f0 tests/acceptance: Test virtio-rng on s390 via /dev/hwrng
/dev/hwrng is only functional if virtio-rng is working right, so let's
add a sanity check for this device node.

Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201215183623.110128-3-thuth@redhat.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2020-12-21 18:11:33 +01:00
Thomas Huth
09d4455e42 tests/acceptance: Extract the code to clear dmesg and wait for CRW reports
We will use this in more spots soon, so it's easier to put this into
a separate function.

Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201215183623.110128-2-thuth@redhat.com>
Signed-off-by: Cornelia Huck <cohuck@redhat.com>
2020-12-21 18:11:33 +01:00
Cornelia Huck
44637c4b2b tests/acceptance: test hot(un)plug of ccw devices
Hotplug a virtio-net-ccw device, and then hotunplug it again.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201208122843.147186-1-cohuck@redhat.com>
2020-12-21 18:11:33 +01:00
Cornelia Huck
085cec59cf tests/acceptance: test s390x zpci fid propagation
Verify that a fid specified on the command line shows up correctly
as the function_id in the guest.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
[re-formatted overlong lines]
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201130180216.15366-4-cohuck@redhat.com>
2020-12-11 11:38:10 +01:00
Cornelia Huck
864852ccb8 tests/acceptance: verify s390x device detection
The kernel/initrd combination does not provide the virtio-net
driver; therefore, simply check whether the presented device type
is indeed virtio-net for the two virtio-net-{ccw,pci} devices.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
[re-formatted overlong lines]
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201130180216.15366-3-cohuck@redhat.com>
2020-12-11 11:38:10 +01:00
Cornelia Huck
083470b579 tests/acceptance: test virtio-ccw revision handling
The max_revision prop of virtio-ccw devices can be used to force
an older revision for compatibility handling. The easiest way to
check this is to force a device to revision 0, which turns off
virtio-1.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
[re-formatted overlong lines]
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201130180216.15366-2-cohuck@redhat.com>
2020-12-11 11:38:10 +01:00
Cornelia Huck
2d9ca5a37b tests/acceptance: add a test for devices on s390x
This adds a very basic test for checking that we present devices
in a way that Linux can consume: boot with both virtio-net-ccw and
virtio-net-pci attached and then verify that Linux is able to see
and detect these devices.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Message-Id: <20201126130158.1471985-1-cohuck@redhat.com>
2020-12-11 11:38:10 +01:00
Alex Bennée
69272bec1a tests/avocado: clean-up socket directory after run
Previously we were leaving temporary directories behind. While the
QEMUMachine does make efforts to clean up after itself the directory
belongs to the calling function. We use TemporaryDirectory to wrap
this although we explicitly clear the reference in tearDown() as it
doesn't get cleaned up otherwise.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20201117173635.29101-4-alex.bennee@linaro.org>
2020-11-23 09:52:24 +00:00
Alex Bennée
8c175c63ee tests: add prefixes to the bare mkdtemp calls
The first step to debug a thing is to know what created the thing in
the first place. Add some prefixes so random tmpdir's have something
grep in the code.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201117173635.29101-3-alex.bennee@linaro.org>
2020-11-23 09:51:43 +00:00
Peter Maydell
cb5ed407a1 Fix Lesser GPL license versions (should be "2.1" and not "2")
-----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCAAvFiEEJ7iIR+7gJQEY8+q5LtnXdP5wLbUFAl+xVToRHHRodXRoQHJl
 ZGhhdC5jb20ACgkQLtnXdP5wLbX2ow/+MKI6s5QeaAGMJfEG1KkfCCfrybJSy+1X
 BN7ANBxkwndPIZEDfE1Hnxh0nH+jYLhtVm1N7F7EJ2PuyEw+yoVIVE4xp1HFQYEb
 gowE2QSjc1TqyoJhGu/Pj6cGxKFp14ZNIj8w1gKttgjK7doDom76ZZFfRoSc5Qw4
 EoMe5Bnit4qOrjCKnfa0XEMlcB598KKyTxPiwlpA2Tnf2Hl7dAwnkT+fbaQ0iTcL
 f0t8O1GVqyrQSkadf+n94l4kwATk92A7ZLrq7imImRfFo9JwmaAl8rZj3KUdrUax
 IckyUWsiQ+RyH7Th7h8F5jqPLjDYgNOfx/BMaNcLUNBLyaco4lcZT3X0JUxuzV7Y
 KPMk1PuUzv2yhEu1QV6o/0WQ/AwcZNQbNR98z5rqZsCG2rF+AVSMAEz97B7AIfbb
 hJH3f7lmIPcAqkdGXxy/PphzD8qVpm0bTQUWVJi0+ASw73ucTSzgWV29g/b8gh+X
 DzjFAnwYY21oUH8nhZrraQRd1Nu+GDF/+BXrIsBvznxNDyyYS12V2PurWaUwGX5o
 TYLRwMHlvOlypEh0C2mjU+X/1TmLoMueSouDQ22R01sVIUG7PxbNCn46BJTGnxdE
 6kd6rEC804vC2n96WFaHozBbOBwssFDrVsuwUg5HCTIu6BFyCRVTa2P5WxzYPSdM
 lTs9rBwdXPc=
 =GI5d
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-11-15' into staging

Fix Lesser GPL license versions (should be "2.1" and not "2")

# gpg: Signature made Sun 15 Nov 2020 16:20:10 GMT
# gpg:                using RSA key 27B88847EEE0250118F3EAB92ED9D774FE702DB5
# gpg:                issuer "thuth@redhat.com"
# gpg: Good signature from "Thomas Huth <th.huth@gmx.de>" [full]
# gpg:                 aka "Thomas Huth <thuth@redhat.com>" [full]
# gpg:                 aka "Thomas Huth <huth@tuxfamily.org>" [full]
# gpg:                 aka "Thomas Huth <th.huth@posteo.de>" [unknown]
# Primary key fingerprint: 27B8 8847 EEE0 2501 18F3  EAB9 2ED9 D774 FE70 2DB5

* remotes/huth-gitlab/tags/pull-request-2020-11-15: (26 commits)
  nomaintainer: Fix Lesser GPL version number
  test: Fix LGPL information in the file headers
  tests/acceptance: Fix LGPL information in the file headers
  tests/migration: Fix LGPL information in the file headers
  sparc tcg cpus: Fix Lesser GPL version number
  e1000e: Fix Lesser GPL version number
  x86 hvf cpus: Fix Lesser GPL version number
  nvdimm: Fix Lesser GPL version number
  w32: Fix Lesser GPL version number
  tpm: Fix Lesser GPL version number
  overall/alpha tcg cpus|hppa: Fix Lesser GPL version number
  overall usermode...: Fix Lesser GPL version number
  migration: Fix Lesser GPL version number
  parallel nor flash: Fix Lesser GPL version number
  arm tcg cpus: Fix Lesser GPL version number
  x86 tcg cpus: Fix Lesser GPL version number
  linux user: Fix Lesser GPL version number
  usb: Fix Lesser GPL version number
  tricore tcg cpus: Fix Lesser GPL version number
  xtensa tcg cpus: Fix Lesser GPL version number
  ...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-11-16 17:00:36 +00:00
Philippe Mathieu-Daudé
b48580ad92 tests/acceptance: Disable Spartan-3A DSP 1800A test
This test is regularly failing on CI:

   (05/34) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_microblaze_s3adsp1800:
  Linux version 4.11.3 (thuth@thuth.remote.csb) (gcc version 6.4.0 (Buildroot 2018.05.2) ) #5 Tue Dec 11 11:56:23 CET 2018
  ...
  Freeing unused kernel memory: 1444K
  This architecture does not have kernel memory protection.
  [nothing happens here]
  Runner error occurred: Timeout reached (90.91 s)

This is a regression. Until someone figure out the problem,
disable the test to keep CI pipeline useful.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Acked-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20201109091719.2449141-1-f4bug@amsat.org>
Message-Id: <20201110192316.26397-9-alex.bennee@linaro.org>
2020-11-16 11:08:04 +00:00
Gan Qixin
6a2cd9570e tests/acceptance: Fix LGPL information in the file headers
There never was a "Lesser GPL version 2.0", It is either "GPL version 2.0"
or "Lesser GPL version 2.1". This patch replaces all "Lesser GPL version 2.0"
with "Lesser GPL version 2.1" in the tests/acceptance folder.

Signed-off-by: Gan Qixin <ganqixin@huawei.com>
Message-Id: <20201110184223.549499-3-ganqixin@huawei.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-11-15 17:04:40 +01:00
Max Reitz
af1bb3fe7f tests/acceptance: Add virtiofs_submounts.py
This test invokes several shell scripts to create a random directory
tree full of submounts, and then check in the VM whether every submount
has its own ID and the structure looks as expected.

(Note that the test scripts must be non-executable, so Avocado will not
try to execute them as if they were tests on their own, too.)

Because at this commit's date it is unlikely that the Linux kernel on
the image provided by boot_linux.py supports submounts in virtio-fs, the
test will be cancelled if no custom Linux binary is provided through the
vmlinuz parameter.  (The on-image kernel can be used by providing an
empty string via vmlinuz=.)

So, invoking the test can be done as follows:
$ avocado run \
    tests/acceptance/virtiofs_submounts.py \
    -p vmlinuz=/path/to/linux/build/arch/x86/boot/bzImage

This test requires root privileges (through passwordless sudo -n),
because at this point, virtiofsd requires them.  (If you have a
timestamp_timeout period for sudoers (e.g. the default of 5 min), you
can provide this by executing something like "sudo true" before invoking
Avocado.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20201102161859.156603-8-mreitz@redhat.com>
Tested-by: Eduardo Habkost <ehabkost@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-11-02 19:23:48 +00:00
Max Reitz
0a7798037a tests/acceptance/boot_linux: Accept SSH pubkey
Let download_cloudinit() take an optional pubkey, which subclasses of
BootLinux can pass through setUp().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-Id: <20201102161859.156603-7-mreitz@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-11-02 19:22:51 +00:00
Peter Maydell
a5e7fb4d20 Testing and gitdm updates
- add some more individual contributors
   - include SDL2 in centos images
   - skip checkpatch check when no commits found
   - use random port for gdb reverse debugging
   - make gitlab use it's own mirrors to clone
   - fix detection of make -nqp
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAl+X7qsACgkQ+9DbCVqe
 KkSpmAf/To0P/c/B6qMEvNfCiWtn/Nuypau8W60q/ZvE/J4IzFarwa5uc7COmyNR
 xjvxdoeKp17ykFdwBzGhiKeO83SO8DUH1JmzA6STs4RH5VSFGYycuE0cQ9KugrpE
 4yOruOdEgAn2+35oYalE9u6/EugdIVBZsjMS/ofsgR9ysYPuju4Q9by6DEK6ArW0
 AmdwJzKAYw75v+D2wKNq9coSWvbqWClsV6j0gaKgn+OQYqWo25T/DzWdr1okCRET
 TvB0HZLCDQHz9Friq5PtBtVsmNbZaL+CM9RRekFBGUFEvZE8i4VEDHFfzXsN3U0w
 VA5yhlBJzl1/prBQe1qRbrZQL4x58w==
 =Bnt1
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-and-misc-271020-1' into staging

Testing and gitdm updates

  - add some more individual contributors
  - include SDL2 in centos images
  - skip checkpatch check when no commits found
  - use random port for gdb reverse debugging
  - make gitlab use it's own mirrors to clone
  - fix detection of make -nqp

# gpg: Signature made Tue 27 Oct 2020 09:55:55 GMT
# gpg:                using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8  DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-and-misc-271020-1:
  makefile: handle -n / -k / -q correctly
  gitlab-ci: Clone from GitLab itself
  tests/acceptance: pick a random gdb port for reverse debugging
  scripts: fix error from checkpatch.pl when no commits are found
  gitlab: skip checkpatch.pl checks if no commit delta on branch
  tests/docker/dockerfiles/centos: Use SDL2 instead of SDL1
  contrib/gitdm: Add more individual contributors
  Adding ani's email as an individual contributor

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-28 20:40:09 +00:00
Alex Williamson
33dc9914ea Revert series: virtiofsd: Announce submounts to the guest
This reverts the following commits due to their basis on a bogus
linux kernel header update:

c93a656f7b ("tests/acceptance: Add virtiofs_submounts.py")
45ced7ca2f ("tests/acceptance/boot_linux: Accept SSH pubkey")
08dce386e7 ("virtiofsd: Announce sub-mount points")
eba8b096c1 ("virtiofsd: Store every lo_inode's parent_dev")
ede24b6be7 ("virtiofsd: Add fuse_reply_attr_with_flags()")
e2577435d3 ("virtiofsd: Add attr_flags to fuse_entry_param")
2f10415abf ("virtiofsd: Announce FUSE_ATTR_FLAGS")
97d741cc96 ("linux/fuse.h: Pull in from Linux")

Cc: Max Reitz <mreitz@redhat.com>
Cc: Stefan Hajnoczi <stefanha@redhat.com>
Cc: Dr. David Alan Gilbert <dgilbert@redhat.com>
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Message-id: 160385090886.20017.13382256442750027666.stgit@gimli.home
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-28 13:17:32 +00:00
Peter Maydell
cfc1105649 Acceptance testing patches
- More ARM tests
 - Documentation update
 
 CI jobs results:
 . https://cirrus-ci.com/build/5754555684093952
 . https://gitlab.com/philmd/qemu/-/pipelines/207807648
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl+XHkwACgkQ4+MsLN6t
 wN4WJBAAxfsfZV3DjoKP/mAjVKJCHkA82MzzoN+YH5WQs4QG1Hytod7OyWC101xj
 wPuL85NR+bo+he12Uylmbm35TCyKxH3kOk2GAqiHvS16mxNZtAssehVt5a1hoUlw
 N9R37dtccGbURX3WzEgJralaZaST96ZbP+y76A95ygriXLnZRisVJG0t/FKBb22m
 8B8+lgHpb11hFop/COY93Aa/BPx9S7cPpBQX4p8e/gXhdJTtqzbhWYSkLO60FqJh
 l1D8KAB7dUtVPoW1ZPSL0YsSGVlxKslOQ8T2Ikrhu8whDZNpYWaoXP0nK3aFK5Th
 jSYObXj/fXtj+APJZILoYAzEtlFsX0FFPfPZy2ffvX6+P1S4COsU2mkN9jJSu42C
 i8Ii5r1WAN6OPFbgPVQ2tQP9jq+scwBUT50GIzgLvja2dWPBM7drxB/hAByItgRt
 ARaDupnC6ShXPLMPm14fSfThyQQw+wiTHo9sZaks5NuoOz9WAh6XUK3JLR5S5zEG
 OxYunho2zSoqYAmnYtymNQ3L0ABW679EbbB2uZJX9m4iFGSxnNJoFFZyAXsZm/tM
 sRQBfTdBH1OCsVdy9/uYrNdZmj0QMBU5mJmv8Qsv722PLV8GuBF8O64C4F4Y42BP
 ApADllyRIONxXm95iBXISuZAF7qAMfGcvI9ldpUyetUT59oz4YI=
 =5Xi7
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/acceptance-testing-20201026' into staging

Acceptance testing patches

- More ARM tests
- Documentation update

CI jobs results:
. https://cirrus-ci.com/build/5754555684093952
. https://gitlab.com/philmd/qemu/-/pipelines/207807648

# gpg: Signature made Mon 26 Oct 2020 19:06:52 GMT
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/acceptance-testing-20201026:
  docs/devel/testing.rst: Update outdated Avocado URLs
  tests/acceptance: Allow running Orange Pi test using cached artifacts
  tests/acceptance: Add a 'virt_kvm' test using the GICv3
  tests/boot_linux_console: Boot Trusted Firmware-A on the Raspberry Pi 3

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-10-27 16:58:39 +00:00
Alex Bennée
0395b19438 tests/acceptance: pick a random gdb port for reverse debugging
Currently the test randomly fails if you are using a shared machine
due to contention on the well known port 1234. We can ameliorate this
a bit by picking a random non-ephemeral port although it doesn't
totally avoid the problem. While we could use a totally unique socket
address for debugging it is fiddly to probe for gdb support. While gdb
socket debugging is not yet ubiquitous this a sub-optimal but workable
option.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021163136.27324-7-alex.bennee@linaro.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
2020-10-27 09:53:51 +00:00
Max Reitz
c93a656f7b tests/acceptance: Add virtiofs_submounts.py
This test invokes several shell scripts to create a random directory
tree full of submounts, and then check in the VM whether every submount
has its own ID and the structure looks as expected.

(Note that the test scripts must be non-executable, so Avocado will not
try to execute them as if they were tests on their own, too.)

Because at this commit's date it is unlikely that the Linux kernel on
the image provided by boot_linux.py supports submounts in virtio-fs, the
test will be cancelled if no custom Linux binary is provided through the
vmlinuz parameter.  (The on-image kernel can be used by providing an
empty string via vmlinuz=.)

So, invoking the test can be done as follows:
$ avocado run \
    tests/acceptance/virtiofs_submounts.py \
    -p vmlinuz=/path/to/linux/build/arch/x86/boot/bzImage

This test requires root privileges (through passwordless sudo -n),
because at this point, virtiofsd requires them.  (If you have a
timestamp_timeout period for sudoers (e.g. the default of 5 min), you
can provide this by executing something like "sudo true" before invoking
Avocado.)

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-9-mreitz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-26 18:35:32 +00:00
Max Reitz
45ced7ca2f tests/acceptance/boot_linux: Accept SSH pubkey
Let download_cloudinit() take an optional pubkey, which subclasses of
BootLinux can pass through setUp().

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200909184028.262297-8-mreitz@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: WIllian Rampazzo <willianr@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
2020-10-26 18:35:32 +00:00
Philippe Mathieu-Daudé
9eb716513d tests/acceptance: Allow running Orange Pi test using cached artifacts
Unfortunately the Armbian 19.11.3 image has been removed from the
dl.armbian.com file server. Developers having the artifact cached
can still run the test. Allow them to, until we find a proper
solution to share binaries with the whole community.

This avoids (when file manually added to cache):

  BootLinuxConsole.test_arm_orangepi_bionic_19_11: CANCEL: Missing asset https://dl.armbian.com/orangepipc/archive/Armbian_19.11.3_Orangepipc_bionic_current_5.3.9.7z (1.06 s)

Reported-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Bin Meng <bin.meng@windriver.com>
Tested-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Reviewed-by: Niek Linnenbank <nieklinnenbank@gmail.com>
Message-Id: <20201023131808.3198005-3-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-10-26 18:20:51 +01:00
Philippe Mathieu-Daudé
9923375807 tests/acceptance: Add a 'virt_kvm' test using the GICv3
The current 'virt_kvm' test is restricted to GICv2, but can also
work with a GICv3. Duplicate it but add a GICv3 test which can be
tested on some hardware.

Noticed while running:

 $ avocado --show=app run -t machine:virt tests/acceptance/
 ...
 (2/6) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm: ERROR: Unexpected empty reply from server (1.82 s)

The job.log content is:

  L0351 DEBUG| Output: 'qemu-system-aarch64: host does not support in-kernel GICv2 emulation\n'

With this patch:

 $ avocado --show=app run -t device:gicv3 tests/acceptance/
 (1/1) tests/acceptance/boot_linux.py:BootLinuxAarch64.test_virt_kvm_gicv3: PASS (55.10 s)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Message-Id: <20200929224857.1225107-1-philmd@redhat.com>
2020-10-26 18:20:51 +01:00
Philippe Mathieu-Daudé
7fb1f203d7 tests/boot_linux_console: Boot Trusted Firmware-A on the Raspberry Pi 3
This test runs Trusted Firmware-A on the Raspberry Pi 3.
We deliberately stop the boot process when the EDK2 UEFI version
is displayed.

The binary is build on AppVeyor CI using Pete Batard repository [1].
ATF v2.1 binary are used (see [2]).

It is very simple and fast:

  $ avocado --show=app,console run -t atf tests/acceptance
  JOB ID     : 1e748d7c9e9011cf0af3250ddc8ebf2389d6204e
  JOB LOG    : avocado/job-results/job-2020-02-16T18.08-1e748d7/job.log
   (1/1) tests/acceptance/boot_linux_console.py:BootLinuxConsole.test_aarch64_raspi3_atf:
  console: NOTICE:  Booting Trusted Firmware
  console: NOTICE:  BL1: v2.1(release):v2.1
  console: NOTICE:  BL1: Built : 15:26:06, May 13 2019
  console: NOTICE:  rpi3: Detected: Raspberry Pi 3 Model B (1GB, Sony, UK) [0x00a02082]
  console: NOTICE:  BL1: Booting BL2
  console: ERROR:   rpi3_sdhost: timeout status 0x40
  console: NOTICE:  BL2: v2.1(release):v2.1
  console: NOTICE:  BL2: Built : 15:26:01, May 13 2019
  console: NOTICE:  BL1: Booting BL31
  console: NOTICE:  BL31: v2.1(release):v2.1
  console: NOTICE:  BL31: Built : 15:26:04, May 13 2019
  console: =UEFI firmware (version UEFI Firmware v1.15 built at 11:58:44 on Feb 14 2020)
  PASS (1.54 s)
  RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
  JOB TIME   : 1.88 s

[1] https://github.com/pbatard/RPi3#summary
[2] https://github.com/ARM-software/arm-trusted-firmware/blob/v2.1/docs/plat/rpi3.rst

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Luc Michel <luc.michel@greensocs.com>
Message-Id: <20200217103442.30318-1-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-10-26 18:20:51 +01:00
Philippe Mathieu-Daudé
28bbe20ce2 tests/acceptance: Use .ppm extention for Portable PixMap files
The HMP 'screendump' command generates Portable PixMap files.
Make it obvious by using the .ppm file extention.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-3-f4bug@amsat.org>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Philippe Mathieu-Daudé
239f0d455b tests/acceptance: Remove unused import
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20201021105035.2477784-2-f4bug@amsat.org>
Tested-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Thomas Huth
1d60f46fc6 test/acceptance: Remove the CONTINUOUS_INTEGRATION tags
We are not running the acceptance tests on Travis anymore, so these
checks can be removed now.

Message-Id: <20201023073351.251332-3-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Thomas Huth
a60f755c9c tests/acceptance/ppc_prep_40p: Fix the URL to the NetBSD-4.0 archive
The current URL on cdn.netbsd.org is failing - using archive.netbsd.org
instead seems to be fine.

Message-Id: <20201023073351.251332-2-thuth@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-10-26 09:54:16 +01:00
Pavel Dovgalyuk
1d2ff14b72 tests/acceptance: Add MIPS record/replay tests
This patch adds MIPS-targeted acceptance tests for
record/replay functions.

Signed-off-by: Pavel Dovgalyuk <pavel.dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <160276110297.2705.10918105269658307206.stgit@pasha-ThinkPad-X280>
[PMD: Moved 'override timeout' comment from instance to class,
      moved nanomips tests to ReplayKernelSlow class,
      tagged ReplayKernelSlow class with AVOCADO_TIMEOUT_EXPECTED]
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
2020-10-17 13:59:40 +02:00
Alex Bennée
e5d402b28f tests/acceptance: disable machine_rx_gdbsim on GitLab
While I can get the ssh test to fail on my test setup this seems a lot
more stable except when on GitLab. Hopefully we can re-enable both
once the serial timing patches have been added.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20201007160038.26953-23-alex.bennee@linaro.org>
2020-10-09 17:27:55 +01:00
Pavel Dovgalyuk
be52eca309 tests/acceptance: add reverse debugging test
This is a test for GDB reverse debugging commands: reverse step and reverse continue.
Every test in this suite consists of two phases: record and replay.
Recording saves the execution of some instructions and makes an initial
VM snapshot to allow reverse execution.
Replay saves the order of the first instructions and then checks that they
are executed backwards in the correct order.
After that the execution is replayed to the end, and reverse continue
command is checked by setting several breakpoints, and asserting
that the execution is stopped at the last of them.

Signed-off-by: Pavel Dovgalyuk <Pavel.Dovgalyuk@ispras.ru>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>

--

v5:
 - disabled (as some other tests) when running on gitlab
   due to the unidentified timeout problem
Message-Id: <160174524678.12451.13258942849173670277.stgit@pasha-ThinkPad-X280>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-10-06 08:34:50 +02:00
Philippe Mathieu-Daudé
b6eea069ab tests/acceptance: Skip slow quanta-gsj U-boot+Linux test
The 'arm_quanta_gsj_initrd' test is timeouting on GitLab CI:
https://gitlab.com/philmd/qemu/-/jobs/745483978#L846
and also sometimes on my workstation, so proceed as with
the other slow tests: do not run it by default.
The test can still be run setting the AVOCADO_TIMEOUT_EXPECTED
environment variable.

Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Havard Skinnemoen <hskinnemoen@google.com>
Tested-by: Havard Skinnemoen <hskinnemoen@google.com>
Message-Id: <20200918143355.153522-1-f4bug@amsat.org>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-09-18 19:46:38 +02:00
Philippe Mathieu-Daudé
a58bf4b3d9 tests/acceptance: Disable tests dependent of unreliable apt.armbian.com
Armbian servers are not very reliable and confused the GitLab CI
users a few times this month (path updated, archives moved, and
now the SSL: CERTIFICATE_VERIFY_FAILED "certificate has expired"
error). Time to disable these tests.
Users can still use the artifacts from the cache (or manually add
them to the cache).

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Acked-by: Gerd Hoffmann <kraxel@redhat.com>
Message-Id: <20200917163954.50514-1-philmd@redhat.com>
2020-09-18 19:44:22 +02:00
Havard Skinnemoen
4fe986dd44 tests/acceptance: console boot tests for quanta-gsj
This adds two acceptance tests for the quanta-gsj machine.

One test downloads a lightly patched openbmc flash image from github and
verifies that it boots all the way to the login prompt.

The other test downloads a kernel, initrd and dtb built from the same
openbmc source and verifies that the kernel detects all CPUs and boots
to the point where it can't find the root filesystem (because we have no
flash image in this case).

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Tested-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Signed-off-by: Havard Skinnemoen <hskinnemoen@google.com>
Message-id: 20200911052101.2602693-15-hskinnemoen@google.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-09-14 14:24:59 +01:00
Philippe Mathieu-Daudé
89e076f37d tests/acceptance: Add Test.fetch_asset(cancel_on_missing=True)
Invert the default of avocado.Test.fetch_asset 'cancel_on_missing'
keyword: accept missing artefacts by default. If a test is certain
an artifact can't be missing, it will set cancel_on_missing=False.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20200908202352.298506-3-philmd@redhat.com>
Message-Id: <20200909112742.25730-10-alex.bennee@linaro.org>
2020-09-10 10:46:57 +01:00
Peter Maydell
d6f83a72a7 Acceptance tests patches
- Use stable URLs for the Debian and Ubuntu installer
   (Ubuntu has been updated last Wednesday, August 5, 2020).
 
 CI jobs results:
 . https://cirrus-ci.com/build/6385815351721984
 . https://gitlab.com/philmd/qemu/-/pipelines/177054604
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl8z69UACgkQ4+MsLN6t
 wN7nHQ/9Hfu29LidcGyFgzPTdlepeKKRLtgCvMgyqIIYC2WhiW6AuhccfZMSB1Zs
 C9z/ubvASyFFav+slmPBLnWKPUm5pf5pCLapvb9c7hS5+wNlq9Jdg3cq4pZp6lGb
 bWstciGkXQzHXOVN3fv7yGX0Q1trtCHKbxJoFJPhEYg7dFons+XOLg2/bGWgoJmT
 fd5T08+CBM86o9q6zd2xyypJAU3DIFX6LNrVfxVtGID3Kn8wJ9g05nG7SENlHQeK
 cI/rf+/2WOQYU+RWUkpRw+O5++9I56SpvZ80a+Dw82vkM7ThtSb98utqvoiVQwLb
 T6cSTYl03+r6Mr2MhmW/RRoe5P5vaUiNcnclboP0kaF02kvCo2UMcWxj57gsu89K
 5EHJnAKTDJexSpDZsz9GGYQhXjpNGAaSUyi4HrN+Dij5EOB1XDHBmZ+VwQAdIi2f
 dUTmRbr+3ZyMzaANb+HL3VlU+yWR3m8faCkTQguSfNmfFEWS1tDpib1taDwy/Esu
 GvR1AccQue7XnoBrqPN3rxr064pe96n7JZiUtcPosxpULgSb4/H724VZMaPAX497
 O3kc8D6tCtZ8HGll/Dv6vgJOt/SOHMJya0xTG4WnHnNAMFxyVrDuz3aUmrUzmHtI
 Qrd68VhjWkl4/+7COAO7QOLpuAxjozQj7r5h01sYbSodmByDhas=
 =pqT+
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/acceptance-testing-20200812' into staging

Acceptance tests patches

- Use stable URLs for the Debian and Ubuntu installer
  (Ubuntu has been updated last Wednesday, August 5, 2020).

CI jobs results:
. https://cirrus-ci.com/build/6385815351721984
. https://gitlab.com/philmd/qemu/-/pipelines/177054604

# gpg: Signature made Wed 12 Aug 2020 14:17:09 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/acceptance-testing-20200812:
  acceptance: use stable URLs for the Debian and Ubuntu installer
  tests/acceptance/boot_linux: Extract common URL from xlnx-versal test

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-08-21 14:51:43 +01:00
Paolo Bonzini
64ed6f92ff meson: link emulators without Makefile.target
The binaries move to the root directory, e.g. qemu-system-i386 or
qemu-arm.  This requires changes to qtests, CI, etc.

Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2020-08-21 06:30:40 -04:00
Paolo Bonzini
0eca1f4b5c acceptance: use stable URLs for the Debian and Ubuntu installer
The kernel and initrd hashes seem to have changed for the Bionic
aarch64 installer, causing BootLinuxConsole.test_aarch64_xlnx_versal_virt
to fail.  Correct the paths to use the previous binaries instead of
the latest.  Do the same for the Lenny alpha installer for
consistency, even though those are unlikely to change.

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200810092941.154911-1-pbonzini@redhat.com>
2020-08-12 14:14:28 +02:00
Philippe Mathieu-Daudé
c45aed126f tests/acceptance/boot_linux: Extract common URL from xlnx-versal test
Both files refer to the same directory. Store the common part in
a new variable.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Message-Id: <20200810093050.28744-2-philmd@redhat.com>
2020-08-12 14:13:02 +02:00
Thomas Huth
4d6862ffc7 tests/acceptance: Disable the rx sash and arm cubieboard replay test on Gitlab
These tests always time out on Gitlab, not sure what's happening here.
Let's disable them until somebody has enough spare time to debug the
issues.

Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200730141326.8260-3-thuth@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Thomas Huth <thuth@redhat.com>
2020-08-05 10:38:27 +02:00
Peter Maydell
673205379f Python patches for 5.1
- Reduce race conditions on QEMUMachine::shutdown()
 
  1. Remove the "bare except" pattern in the existing shutdown code,
     which can mask problems and make debugging difficult.
  2. Ensure that post-shutdown cleanup is always performed, even when
     graceful termination fails.
  3. Unify cleanup paths such that no matter how the VM is terminated,
     the same functions and steps are always taken to reset the object
     state.
  4. Rewrite shutdown() such that any error encountered when attempting
     a graceful shutdown will be raised as an AbnormalShutdown exception.
     The pythonic idiom is to allow the caller to decide if this is a
     problem or not.
 
 - Modify part of the python/qemu library to comply with:
 
   . mypy --strict
   . pylint
   . flake8
 
 - Script for the TCG Continuous Benchmarking project that uses
   callgrind to dissect QEMU execution into three main phases:
 
   . code generation
   . JIT execution
   . helpers execution
 
 CI jobs results:
 . https://cirrus-ci.com/build/5421349961203712
 . https://gitlab.com/philmd/qemu/-/pipelines/166556001
 . https://travis-ci.org/github/philmd/qemu/builds/708102347
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEE+qvnXhKRciHc/Wuy4+MsLN6twN4FAl8OGCUACgkQ4+MsLN6t
 wN52ZQ//cB8pRhOeirMzPTamtAePAHhU9kRKSQ4fFzVM/AHxaO3qO3KrMKAv4JnR
 1fwdyAbum0Dv+HrRAXbIVz/lgrlTb6tLqwk20YWlmNUR2CtMmno9fPQzg6LeFWBe
 +qr2/anXWBqRZu5uHzQ3Ly3LxlVXE3qmStxlttNxed9wK9wdhsQ4ijOrExXPmMI5
 IQi9tKijsGvOczcSpIwJET6Px0L/GIYHSFGXfHv4LiJCMQUX4gqY1akr+/2adqlr
 ElcOHlvWG90GHftTC+hq+KbSNmpkwJEnXLHC+HwMaYmQuhH4+U6f9AcrnzYGHjoQ
 84muuYt5CBvT/Q38qO3dUsr42oD8uiDZe2cpD5DmcAHj6U9sza4UwU2Nkd4cNvRD
 OiYRWuHQDzQhwPGbzBym1GRxRlejmPXCpvZ4XqaYODkzhEHi8UVq5AOsNhpTGWVk
 WoDZ3NtnZb9WbU93wDordhWTtHKHOLHNk4CuJLbdRvkW+sg8aFqUObSRvc6RqyU8
 1DXWMMQr9UfCDfMMbp/OCymphn0VrAaixIRIjmYc6nL5mQlyugMwij55jzyugqBH
 rN5bdcsqxAhBTD03lu0lbDn1BYIqbZFXABXeDnJNbxmY/6gD0E1HKWRvIKjxzH+t
 L19QXsOWLoVUpUIjxlOgMlb9QJLPc9m/N3yVOanO0zVrMSBmOKQ=
 =Jp6n
 -----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/philmd-gitlab/tags/python-next-20200714' into staging

Python patches for 5.1

- Reduce race conditions on QEMUMachine::shutdown()

 1. Remove the "bare except" pattern in the existing shutdown code,
    which can mask problems and make debugging difficult.
 2. Ensure that post-shutdown cleanup is always performed, even when
    graceful termination fails.
 3. Unify cleanup paths such that no matter how the VM is terminated,
    the same functions and steps are always taken to reset the object
    state.
 4. Rewrite shutdown() such that any error encountered when attempting
    a graceful shutdown will be raised as an AbnormalShutdown exception.
    The pythonic idiom is to allow the caller to decide if this is a
    problem or not.

- Modify part of the python/qemu library to comply with:

  . mypy --strict
  . pylint
  . flake8

- Script for the TCG Continuous Benchmarking project that uses
  callgrind to dissect QEMU execution into three main phases:

  . code generation
  . JIT execution
  . helpers execution

CI jobs results:
. https://cirrus-ci.com/build/5421349961203712
. https://gitlab.com/philmd/qemu/-/pipelines/166556001
. https://travis-ci.org/github/philmd/qemu/builds/708102347

# gpg: Signature made Tue 14 Jul 2020 21:40:05 BST
# gpg:                using RSA key FAABE75E12917221DCFD6BB2E3E32C2CDEADC0DE
# gpg: Good signature from "Philippe Mathieu-Daudé (F4BUG) <f4bug@amsat.org>" [full]
# Primary key fingerprint: FAAB E75E 1291 7221 DCFD  6BB2 E3E3 2C2C DEAD C0DE

* remotes/philmd-gitlab/tags/python-next-20200714:
  python/qmp.py: add QMPProtocolError
  python/qmp.py: add casts to JSON deserialization
  python/qmp.py: Do not return None from cmd_obj
  python/qmp.py: re-absorb MonitorResponseError
  iotests.py: use qemu.qmp type aliases
  python/qmp.py: Define common types
  python/machine.py: change default wait timeout to 3 seconds
  python/machine.py: re-add sigkill warning suppression
  python/machine.py: split shutdown into hard and soft flavors
  tests/acceptance: Don't test reboot on cubieboard
  tests/acceptance: wait() instead of shutdown() where appropriate
  python/machine.py: Make wait() call shutdown()
  python/machine.py: Add a configurable timeout to shutdown()
  python/machine.py: Prohibit multiple shutdown() calls
  python/machine.py: Perform early cleanup for wait() calls, too
  python/machine.py: Add _early_cleanup hook
  python/machine.py: Close QMP socket in cleanup
  python/machine.py: consolidate _post_shutdown()
  scripts/performance: Add dissect.py script

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
2020-07-15 13:04:27 +01:00
John Snow
fdb87f0dc2 tests/acceptance: Don't test reboot on cubieboard
cubieboard does not have a functioning reboot, it halts and QEMU does
not exit.

vm.shutdown() is modified in a forthcoming patch that makes it less tolerant
of race conditions on shutdown; tests should consciously decide to WAIT
or to SHUTDOWN qemu.

So long as this test is attempting to reboot, the correct choice would
be to WAIT for the VM to exit. However, since that's broken, we should
SHUTDOWN instead.

SHUTDOWN is indeed what already happens when the test performs teardown,
however, if anyone fixes cubieboard reboot in the future, this test will
develop a new race condition that might be hard to debug.

Therefore: remove the reboot test and make it obvious that the VM is
still running when the test concludes, where the test teardown will do
the right thing.

Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Cleber Rosa <crosa@redhat.com>
Message-Id: <20200710050649.32434-10-jsnow@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
2020-07-14 22:22:22 +02:00