summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa/utils
AgeCommit message (Collapse)Author
2021-07-11oeqa/qemurunner: Handle files no longer existing gracefullyRichard Purdie
Files in /proc/xxx/map_files/ may no longer exist, just ignore this rather than raising an exception. (From OE-Core rev: fb1027896a263cd91e2378a4e97dbdf0807b306b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02oeqa/dump.py: Add support for QMP command argumentsSaul Wold
Need to ensure that the dump_dir is created correctly and available When command arguemnts are passed construct a filename if needed and convert the arguements to a json object to pass to QMP. (From OE-Core rev: 9a2f4e1e95f4a3f7ebbf08f46445c8ea670adce3) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-02oeqa/qemurunner: add support qmp cmd argsSaul Wold
This will enable passing arguments to qmp commands for those that require additional information (From OE-Core rev: 4d34676b6226a34df2877adbeea25abb5b2a31be) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-27qemurunner: Add info log for qemu startupSaul Wold
Add a couple of logging info to track time between activities, first is from after the Popen(launch_cmd) to after qmp.connect(), second is from qmp.connect() to the release of the qemu via the qmp("cont") command this includes the mmap() activity. Example output: QMP connected to QEMU at 06/24/21 11:11:56 and took 0.9556229114532471 seconds from launch QMP released QEMU at 06/24/21 11:11:56 and took 0.26789021492004395 seconds from connect (From OE-Core rev: 547f49230ba4ebeefe5b696e0460ebaffa8e91e6) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-22qemurunner: add second qmp portSaul Wold
This will allow for direct ssh connection without breaking the first one that is used for monitoring. The "nowait" option will cause qmp server connection to NOT block waiting. (From OE-Core rev: 40f09e184afd42decf2f924896fef03beacddc4b) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-11qemurunner: Increase startup timeout 120 -> 300Richard Purdie
We now spend time copying the VM image into a tmpfs and with IO load on the system, the time + the boot time of the VM can take longer than 120s. Increase the timeout to match the added overhead of copying the image file. (From OE-Core rev: a40087c966af5ffb9309e1ddfdb3d06973e0bddd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-11qemurunner: Try to ensure mmap'd libs are paged inRichard Purdie
We've seeing issues where IO load appears to cause strange failures due to timeouts within qemu. One theory for these is that it is is hitting hard page faults at in-opportune moments which cause timing problems within the VM. This patch is a bit of a hack which tries to ensure the data is paged in at a point when we know we can take the time delays (waiting for the QMP start signal). Whilst this isn't ideal, it does seem to improve things on the autobuilder and shouldn't harm anything. The code figures out which files to read my looking at the mmap'd files the process has open from /proc. On Centos7 systems these files are not user readable, if that is the case we just skip them. (From OE-Core rev: e77844314d09ceff9c22338d366519928f4f7284) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-01oeqa/qemurunner: Improve timeout handlingRichard Purdie
We had debugging for qemu faiing to start which was no longer reachable after the QMP changes. Reorder the code to enable this debugging to work again which may allow insight into autobuilder failures in this area. (From OE-Core rev: 8fac8c61565977c775d8ede5bddc856b7767a3e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-09oeqa/qemurunner: Improve handling of run_serial for shutdown commandsRichard Purdie
When running a shutdown command, the serial port can close without the command returning. This is seen as the socket being readable but having no data. Change the way this case is handled in the code to avoid tracebacks. (From OE-Core rev: 396a3ba884820d040c91f7592daf20ac28c49b5d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-09oeqa/qemurunner: Fix binary vs str issueRichard Purdie
The recent logging changes for qemurunner showed up as errors on the autobuilder where decode couldn't be called on the returned string. Since the code returns binary data, return b'' instead of '' to match to avoid tracebacks. One of these cases was newly added, copied from the other which has been there for a long time, always broken. (From OE-Core rev: b8995b27db265b0a0b2d2ca595915f70f9f96e07) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06qemurunner: change warning to infoSaul Wold
This information is useful, but should not be a warning level. [YOCTO #14382] (From OE-Core rev: cd17d8bb00be1ecb7c92ab13eb8b162807aefed9) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06oeqa/qemurunner: Handle path length issues for qmp socketRichard Purdie
After the addition of the qmp socket, runqemu started failing: ERROR - Failed to run qemu: qemu-system-aarch64: -qmp unix:/home/yocto/actions-runner-meta-openembedded/_work/meta-openembedded/meta-openembedded/yoe/build/tmp/.3eg5fiid,server,wait: UNIX socket path '/home/yocto/actions-runner-meta-openembedded/_work/meta-openembedded/meta-openembedded/yoe/build/tmp/.3eg5fiid' is too long Path must be less than 108 bytes To avoid this, run qemu within tmpdir and use a relative path to the socket. This avoids having to patch the socket code within qemu. Update the client code to chdir and only use a relative path to the socket to match. (From OE-Core rev: 5c56e72fca18dc942f5c1fd377e98d46ae0126f1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06oeqa/qemurunner: Improve logging thread exit handling for qemu shutdown testRichard Purdie
Rather than totally disabling the logging, inform it we're about to exit so we can log messages over the exit cleanly too. This aids debugging. It also avoids a race where the logging handler could still error whilst shutting down. Also remove a race window by notificing the handler of the shutdown first, before triggering it. This removes a race window I watched in local testing. (From OE-Core rev: 0e19f31a1005f94105e1cef252abfffcef2aafad) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06Revert "oeqa: Set LD_LIBRARY_PATH when executing native commands"Alexander Kanavin
LD_LIBRARY_PATH leaks into host executables too, and breaks them as they are not uninative-enabled. E.g. on ubuntu 18.04 trying to run host bash with a sysroot that was built on Fedora 33: akanavin@ubuntu1804-ty-3:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-24341/tmp/work/x86_64-linux/gnupg-native/2.3.1-r0/recipe-sysroot-native$ LD_LIBRARY_PATH=./usr/lib /bin/bash /bin/bash: ./usr/lib/libtinfo.so.5: no version information available (required by /bin/bash) /bin/bash: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./usr/lib/libtinfo.so.5) This was seen e.g. here: https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/2090/steps/14/logs/stdio (From OE-Core rev: 0e9850486b74a3de934527ca1077df001d3a8d22) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-27qemurunner: Add support for qmp commandsSaul Wold
This adds support for the Qemu Machine Protocol [0] extending the current dump process for Host and Target. The commands are added in the testimage.bbclass. Currently, we setup qemu to stall until qmp gets connected and sends the initialization and continue commands, this works correctly. If the UNIX Socket does not exist, we wait an timeout to ensure to socket file is created. With this version, the monitor_dumper is created in OEQemuTarget but then set in OESSHTarget as that's where we get the SSH failure happens. Python's @property is used to create a setter/getter type of setup in OESSHTarget to get overridden by OEQemuTarget. By default the data is currently dumped to files for each command in TMPDIR/log/runtime-hostdump/<date>_qmp/unknown_<seq>_qemu_monitor as this is the naming convenstion in the dump.py code. We use the qmp.py from qemu, which needs to get installed in the recipe-sysroot-native of the target image. [0] https://github.com/qemu/qemu/blob/master/docs/interop/qmp-spec.txt (From OE-Core rev: 42af4cd2df72fc8ed9deb3fde4312909842fcf91) Signed-off-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-06oeqa/runqemu: Support RUNQEMU_TMPFS_DIR as a location to copy snapshot images toRichard Purdie
We have a working theory that IO queues on the autobuilder are impacting runtime testing under qemu, particularly async writes which inice does not influence. We already pass the snapshot option to qemu which copies the image and runs out of the copy. Add in the ability to copy the image to a specificed location which can be a tmpfs. This means that writes to the image would no longer be blocked by other writes to disk in the system. Preliminary tests show that this does improve the qemu errors at the expense of sometimes showing qemu startup timeouts as on a loaded system with a large test image, it can take longer than 120s to copy the image to tmpfs. Having a most consistent failure mode for loaded tests is probably desireable though. (From OE-Core rev: fd1c26ab426c3699ffd8082b83d65a84c8eb8bff) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11runqemu: correct forcing of ttyS0Jon Mason
Some platforms do not use ttyS* for their serial consoles (e.g., qemuarm and qemuarm64). The hardcoding of this can cause issues. Modify runqemu to use the serial consoles defined in SERIAL_CONSOLES instead of hardcoding. (From OE-Core rev: 9dea4cd2f9f46ab3a75562639a22d8f56b4d26af) Signed-off-by: Jon Mason <jon.mason@arm.com> Change-Id: I746d56de5669c955c5e29d3ded70c0a4d3171f17 Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-10Fix up bitbake logging compatibilityRichard Purdie
Bitbake changed the debug() logging call to make it compatible with standard python logging by no longer including a debug level as the first argument. Fix up the few places this was being used. Tweaked version of a patch from Joshua Watt. (From OE-Core rev: 5aecb6df67b876aa12eec54998f209d084579599) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-21oeqa/commands: Ensure sync can be found regardless of PATHRichard Purdie
Avoid command not found errors shown in selftest logs due to changes to PATH settings which also risks intermittent problems due to IO load. (From OE-Core rev: 40bcae01b0be2f293dea9ab42c6b7f8f47827cf5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-24oeqa/commands: Fix compatibility with python 3.9Richard Purdie
Python 3.9 dropped isAlive() so use the preferred is_alive(). (From OE-Core rev: 9bb06428cbb2ac0f3d98a1696f050d3393385503) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-20oeqa: Add sync call to command executionRichard Purdie
We previously put a sync call into devtool to try and combat the bitbake timeout issues on the autobuilder. It isn't enough as the timeouts occur mid test. They are also occurring on non-devtool tests. Add in sync calls around command execution instead. (From OE-Core rev: ceca5ed121e2b54415a7ab3a217882e4ea86923a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-20oeqa/qemurunner: Increase serial timeoutRichard Purdie
Increase the serial login timeout from 60 to 120s. This seems like a long time, however for a qemumips image with systemd+PAM and openssh, (e.g. core-image-sato-sdk + DISTRO=poky-altcfg), the getty connects to systemd's pam module which waits on logind and 45s for all this to happen at the same time as things like ssh key generation happens is not unknown. Increase the timeout to match the longer times we know these things can take in the worst case scenarios since we're tired of intermittent issues related to the serial login affecting the autobuilder. (From OE-Core rev: d8b4292db741de660f756dfb766210814d587b7a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27deb: Move package manager to its own dirFredrik Gustafsson
This is a part of a refactor that will split the package manager code so that it's possible to use other package managers in other layers. RP: Fixes to parse/build (From OE-Core rev: 510d5c48c0496f23a3d7aede76ea8735da2d371d) Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27ipk: Move package manager to its own dirFredrik Gustafsson
This is part of a refactor that will split the package manager code so that it's possible to use other package managers in other layers. RP: Fixes to parse/build (From OE-Core rev: 3ef5a3c885e1010cddfe7eba1cd3728f15270d78) Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-27rpm: Move package manager to its own dirFredrik Gustafsson
This is part of a refactor that will split the package manager code so that it's possible to use other package managers in other layers. RP: Fixes to parse/build (From OE-Core rev: 8b776ed9ed291dd8e112621561762449c7eb5ee2) Signed-off-by: Fredrik Gustafsson <fredrigu@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-22oeqa/qemurunner: Add priority/nice information for running processesRichard Purdie
We're seeing failures due to system load. In theory we've set process nice levels which should compensate for this. Add debugging so we can find out if they're being correctly applied. (From OE-Core rev: 1e4e345bba8216b9b5623682206a7dae7cad261c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-14oeqa/utils/qemurunner: Fix missing pid file tracebacksRichard Purdie
One element of the error message guarded against None as a value but I missed the other, fix this. (From OE-Core rev: dbce6baec68d7658453b8c44159e1d1fef746151) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-13qemurunner: Add extra debug info when qemu fails to startRichard Purdie
When qemu fails to start we're struggling to work out why. Add more debug info which can at least confirm/rule out various things. This code is only on the error handling path and more info shoudl help us debug issues. (From OE-Core rev: 3001d0d8f3429e5ff0c37ea7192e85e7001cdb32) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-13qemurunner: Ensure pid location is deterministicRichard Purdie
The pid location could vary due to changes in cwd as only a filename is specified, not a full path. This in theory could be resulting in some of our autobuilder failures. Whilst its difficult to know if this is causing a problem, Using a full path removes any question of such an issue. (From OE-Core rev: 55c186ff410c99570242478b99ac24ebc40aa6bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-23oeqa/utils/command: Improve stdin handling in runCmdRichard Purdie
Occasionally we've been seeing leftover threads from runCmd. The stdin test assumes we clean up all threads but the code assumes that the daemonic thread can be left behind. The issue can be reproduced by adding a time.sleep(10) to the end of writeThread() which will mean it stays resident past the end of the command. We may as well add it to the threads list and clean it up properly, hopefully removing the race in the tests from the autobuilder. [YOCTO #13055] (From OE-Core rev: 9b251dcaffe52d32c1faf41ab57ab414fbc29722) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04oeqa/targetcontrol: Rework exception handling to avoid warningsRichard Purdie
We're seeing: WARNING: bitbake/lib/bb/cookerdata.py:136: ResourceWarning: unclosed file <_io.FileIO name='tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/testimage/qemurunner_log.20200601181912' mode='ab' closefd=True which can only be caused by the qemu.stop() method not being called. Tweak the error handling to fix the blanket exception handler which is likely meaning this function isn't getting called. (From OE-Core rev: ee707090848d793e3b2d82dd3861ae22222682c0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-05-30qemurunner: fix ip fallback detectionKonrad Weihmann
When falling back from detecting ip from /proc/./cmdline the output of runqemu is acutally 'Network configuration: ip=192.168.7.2::192.168.7.1::255.255.255.0' which doesn't match the given regex and leading to run failure, although IP is detectable. Fix regex by inserting an optional 'ip=' prefix to first IP (From OE-Core rev: 75f2471d15fab024775c59cb70c54e3f25f9ae72) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26oeqa/qemurunner: Clean up failure handlingRichard Purdie
If you fail to setup the tap devices, runqemu will error quickly however stdout/stderr are not shown to the user, instead a SystemExit traceback is shown. This could explain some long since unexplained failures on the autobuilder. Rework the error handling so SystemExit isn't used and the standard log failure messages can be shown. The code could likely ultimatley need some restructuring to work effectively. (From OE-Core rev: 57ccf1e3bb320bd28a2d106c98f4706434c3075a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-24testimage: enable ovmf supportKonrad Weihmann
Add support for running wic images with EFI as testimage. Introduces a variable called QEMU_USE_OVMF for configuration. (From OE-Core rev: 3af8aaff68ed332d812ea7dc184d392700ad7882) Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24oeqa/commands: Fix runqemu after tinfoil data connector changesRichard Purdie
Poking changes into config_data and expecting them to appear in the recipe is a bad idea, place the data in recipedata directly instead. (From OE-Core rev: 191dd811900ace0e0af2e97221e10461fae0d9bd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-03-24oeqa/qemurunner: Don't print a warning for harmless exceptionRichard Purdie
Printing a warning for something which is harmless just causes the people monitoring the autobuilder more work. Silently ignore this race. (From OE-Core rev: 2246c8d2466d3876fd1b27fd1943aa4bbf28e14e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-22oeqa/logparser: correctly set test result markersAlexander Kanavin
I spotted this by noticing ptests started to unexpectedly pass, which shouldn't happen yet. (From OE-Core rev: 1cd6e9a8fdfef927916c6cea2371fc0430b1ffdf) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-21oeqa/logparser: Fix performance issues with ptest log parsingRichard Purdie
On the autobuilder a ptest log with 2.1 million lines took around 18 hours to process. This is clearly crazy. We can tweak the processing code to: a) Stop repeatedly joining large strings together (append to a list instead) b) Use one startswith expression instead of multiple re.search() operations With this change it takes 18 hours down to around 12s. [YOCTO #13696] (From OE-Core rev: 01c9f40b979e36a53e789a6bedd89b6d9557dce3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-13testimage: Extend runtime testing infrastructure to allow unconventional ↵Alejandro Hernandez Samaniego
booting processes to be tested The current runtime infrastructure contains hardcoded values which Ill refer to as patterns, these patterns are either searched through or sent via the serial terminal to communicate between HOST and TARGET. These patterns are required since they allow us to check when a device has finished booting, to log in, and to check whether a command sent from our tests has returned, this way we are able to check both the status of the commands that were sent along with its output. The testing process goes somewhat as follows: 1. Launch QEMU and start booting. 2. Check when the device has booted by looking for the pattern login:. 3. Log in as the root user (default for our images). 4. Check that we were able to log in succesfully. 5. Start running the runtime test cases defined by TEST_SUITES. 6. One of such test cases could send a command to the QEMU target. 7. Check whether that command returned. 8. Check its output and status, return whether the test case passed or failed. This patch allows this set of patterns to be defined instead of being hardcoded, but it also automatically sets the defaults that we have been using in the past if they have not been manually defined, for this reason, the patch is less invasive and should not affect in any way how tests are currently being run. Cases that can be enabled with this patch: - A customized image that does not use the root user (or maybe we want to check what happens if we dont use the root user). - An image where the PS1 env variable has been modified, and the prompt pattern wouldnt match the default. - Baremetal applications, which do not follow the conventional way of booting Linux and would probably not show a prompt for a user to log in, same applies for testing bootloaders. - poky-tiny: Using DISTRO=poky-tiny and an image such as the core-image-tiny from meta-intel, which boots directly to RAM, and does not show a log in prompt since it does not contain a conventional init process. The code itself contains comments that should be self explanatory but here is an example on how these patterns can be defined in a hypothetical case where we want to run test cases as the webserver user instead: TESTIMAGE_BOOT_PATTERNS = "send_login_user search_login_succeeded" TESTIMAGE_BOOT_PATTERNS[send_login_user] = "webserver\n" TESTIMAGE_BOOT_PATTERNS[search_login_succeeded] = "webserver@[a-zA-Z0-9\-]+:~#" The variable TESTIMAGE_BOOT_PATTERNS defines which patterns to override when used to communicate with the target when booting, anyone familiar with the PACKAGECONFIG syntax should have no trouble setting these. Other patterns would still be set up as default, e.g. search_reached_prompt would still be login: The accepted flags for TESTIMAGE_BOOT_PATTERNS are the following: search_reached_prompt, send_login_user, search_login_succeeded, search_cmd_finished. They are prefixed with either search/send, to differentiate if the pattern is meant to be sent or searched to/from the target terminal. A working example of this code that falls under the baremetal case mentioned above along with a test case is present on the meta-freertos layer, which tests an RTOS image built with OpenEmbedded and automatically runs a test case on it after booting such image: As usual, INHERIT += "testimage" needs to be present on local.conf $ bitbake freertos-demo -c testimage RESULTS: RESULTS - freertos_echo.FreeRTOSTest.test_freertos_echo: PASSED (2.00s) SUMMARY: freertos-demo () - Ran 1 test in 2.006s freertos-demo - OK - All required tests passed (successes=1, skipped=0, failures=0, errors=0) (From OE-Core rev: 3ab2cbfeff371e8791b031a2852eeef80101a831) Signed-off-by: Alejandro Hernandez Samaniego <aehs29@gmail.com> Signed-off-by: Alejandro Hernandez Samaniego <alejandro@enedino.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-08qemurunner.py: add try/except for pid handling raceTrevor Gamblin
In some instances, attempts to remove the qemu pidfile within the stop() method fail despite the os.path.exists() call immediately before implying that the file is present. Add a try/except block to log a warning if this occurs, rather than failing outright, since the process simply appears to be exiting at an inconvenient time. [YOCTO #13675] (From OE-Core rev: eadb899e23b18eb9eaff145c3bf5b20fb417c3e8) Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-02-04oeqa: modernise subprocess: preexec_fn=os.setsid -> start_new_session=TrueAndré Draszik
start_new_session was added to python3 subprocess in v3.2 and is meant to take the place of the common use of preexec_fn to call os.setsid() in the child - as done here. Update to use the new equivalent. (From OE-Core rev: c3bd502f31a621394a2f6ea3ae90443ff4ac98a4) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-16oeqa/utils/httpserver: allow to pass in listening portAndré Draszik
Being able to specify the listening port is useful when running OEQA from within a docker container, e.g. crops or any other solution. In that case, a port on the outside must be mapped to a specific port inside the container. If no port is specified for the http server module in this case, the http server would choose a random port, which is unlikely to be mapped and thus won't be reachable from the outside. (From OE-Core rev: e54391ca9068661693275fbfcfded9bc0b2606f3) Signed-off-by: André Draszik <git@andred.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-15oeqa: qemu: ensure we print runqemu output in the event of failurePaul Eggleton
If we get here in the code it's because runqemu has failed, this is not a debug situation - we need to see the output, so print it as an error. Fixes [YOCTO #13681]. (From OE-Core rev: c1def38c680d64c992839166bbf9bec51eb0788a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-15oeqa: qemu: fix width of top output in host dumpPaul Eggleton
If runqemu fails, there is some logic to run a set of commands to dump various bits of information that might help debug the issue (particularly in a busy situation such as on the autobuilder). One of those commands is "top -bn1", however top restricts the output to the width of the calling terminal, and for whatever reason this is a little restrictive when called from inside testimage, so set COLUMNS in the environment to a high value to fix it. (Another way is to use the -w option, but that is not supported by our default busybox configuration so it will then fail when this same list of commands is used on the target). (From OE-Core rev: 07da998f5aa6c3f5ffeee5150fc579e8a1051ba4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-15oeqa: qemu: ensure that host dump commands can be run properlyPaul Eggleton
If runqemu fails, there is some logic to run a set of commands to dump various bits of information that might help debug the issue (particularly in a busy situation such as on the autobuilder). However, when we try to run these we are inside the normal build environment which restricts commands to be run on the host to those specified in HOSTTOOLS. Since this isn't a place where host contamination is going to be a problem, override PATH to a reasonable default so that we run the actual host tools directly to avoid the issue. (Logically we would want to use the original PATH value here, but it is not easily accessible.) (From OE-Core rev: 3f6dcf67bccb4f0fc1a9a64e02bd0e2da620ad0e) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-12-06oeqa/qemurunner: Fix cmdline variable use before referenceRichard Purdie
This avoids some tracebacks we've seen on failed autobuilder builds which would allow the real error to be seen. (From OE-Core rev: a4031935a7b8ea4f61b9020c1aa5598e186e7ad7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-27oeqa/utils/qemurunner.py: Handle QEMU machines with a single serialNathan Rossi
Not all QEMU machines are capable of having more than one serial port, this is due to the machine emulating a physical device/board. Rework QemuRunner to handle machines that only have 1 serial port, where the serial port shares output of the kernel log buffer and a login console. In this case the output is mixed but enables the machine to boot and have QemuRunner detect the login prompt. QemuTarget uses SERIAL_CONSOLES to determine the number of available serial ports. (From OE-Core rev: 333897c4a00b41681ffe34312a08cae09274327e) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-30oeqa: Set LD_LIBRARY_PATH when executing native commandsJoshua Watt
Some commands like to look for libraries at runtime manually (e.g. Python's ctype.utils.find_library() function). For this to work properly, the libraries in the native sysroot must be findable. To accomplish this, set LD_LIBRARY_PATH to search library paths in the native sysroot. (From OE-Core rev: 3a7305bdfded3c8988484c3f430110cc121123b5) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-28oeqa/utils/nfs: Add unfs_server function to setup a userspace NFS serverNathan Rossi
Add a nfs module into oeqa utils. This module provides unfs_server which allows a test case to build unfs3-native and setup the unfs server on a target directory of the host. This directory is then shared and can be mounted by the host or a target device attached to the host (e.g. qemu via tap or slirp). The nfs server is setup over UDP and automatically assigns user privileged ports. The function provides the UDP ports for the server as part of a returned python contextmanager which handles cleanup of the server process on completion or exception. Also add a 'udp' arg to get_free_port to get a free UDP port. Note: unfs3 still requires the host to have rpcbind or portmap running. (From OE-Core rev: c754fd85be85ad0a381b642365eca17cea8eb627) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-28commands.py: fix typoChen Qi
(From OE-Core rev: 9b1c150573ffd0e68d37772d5f47482d86ddde6d) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>