summaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2021-04-06oeqa/concurrencytest: Fix display of test stdout/stderrRichard Purdie
If oe-selftest is run with -j, the output to stdout/stderr is being lost at present. Capture this and display it upon test failure. We have code that previously tried to enable this but it wasn't functioning correctly. This should give more usable error reports on the autobuilder. This code will mix stdout and stderr as the output is streamed from the test server without markup. This is most in keeping with subunit/testools though and the easiest way to handle the various challenges here as far as I can see. (From OE-Core rev: 6a954ce5834c8026adecff8478c3d827640bc647) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-04-06oeqa/concurrencytest: Rename variables to improve the codeRichard Purdie
Each time I look at this code I get confused about what the different variables represent. Rename a few of them to better indicate what they represent. (From OE-Core rev: e39d97c0b191add9281bac463ca059685288c81a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-20selftest/wic: Fix dependency issue in rawcopy testRichard Purdie
The current test code contains a subtle race. It is building an ext4, then trying to reuse that image in a wic build for the same recipe. Whilst this happened to work, the code is within its rights to clean up the output before the wic code runs. Avoid this by using separate image targets instead. (From OE-Core rev: 590398080e81fb5e2f81e12b8900858837dfe25f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-20cve-check: CVE_VERSION_SUFFIX to work with patched releaseLee Chee Yang
CVE_VERSION_SUFFIX in "patch" to treat version string with suffix "pX" or "patchX" as patched release. also update testcases to cover this changes and set CVE_VERSION_SUFFIX for sudo. (From OE-Core rev: 8076815fc2ffc8f632e73527ce2b7d158a29e9ea) Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-20selftest/reproducible: Drop ovmf exclusionRichard Purdie
The hardcoded path issue was fixed, drop the exclusion. (From OE-Core rev: 58e02c83dcf4d6c40c206a0371ef5d7f8db2b272) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-16meson: Work around determinism issueRichard Purdie
The meson determinism issue is the same frozenset issue we encountered with python itself. Remove the problematic pyc file until upstream work out the best way to address the issue (as with core python). (From OE-Core rev: ad00107dc02b3211f5d1a7fe889f538ee2bc064d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-15distrodata.py: Test to track recipes without HOMEPAGE or DESCRIPTIONDorinda Bassey
This test is to keep track of recipes that do not have a HOMEPAGE or DESCRIPTION, with exceptions to recipes whose SRC_URI are not linked to external sources. Results of this test: it take about 2 to 3 minutes for this test to complete execution depending on the laptop used, Recipes that didn't have a homepage or description were found, and using list data structure instead of set() returned an ordered list of recipes that didn't have a homepage. Hence it's easier to view recipes with missing homepage and description. [YOCTO #13471] (From OE-Core rev: 5e0cfce7a126c194bfb7855952355c62eae3831a) Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-15go-dep: RemoveKhem Raj
go-dep was an effort for dependency management before go modules, which since 2020 has been deprecated in favor of go modules. Since its not developed any longer and go mdules is officially supported, this should be retired from OE-core as well. (From OE-Core rev: 1e7ed44d87034446f1d07692c9378c3b0a8a9dd3) Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Otavio Salvador <otavio.salvador@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-14reproducibile: remove perf from exclusionsBruce Ashfield
We have fixes for perf reproducibility, so we can drop it from the exclusion list. (From OE-Core rev: 2989779fa244ec2c2c714c27f67753eefeeaf4af) Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11runqemu: correct forcing of ttyS0Jon Mason
Some platforms do not use ttyS* for their serial consoles (e.g., qemuarm and qemuarm64). The hardcoding of this can cause issues. Modify runqemu to use the serial consoles defined in SERIAL_CONSOLES instead of hardcoding. (From OE-Core rev: 9dea4cd2f9f46ab3a75562639a22d8f56b4d26af) Signed-off-by: Jon Mason <jon.mason@arm.com> Change-Id: I746d56de5669c955c5e29d3ded70c0a4d3171f17 Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-11parselogs: Allow expected kernel messages for qemuppc64Khem Raj
pseries machine emulation ends up with these three messages in kernel logs which are harmless, so ignore them for now https://www.mail-archive.com/ubuntu-bugs@lists.ubuntu.com/msg5600561.html https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1827162 (From OE-Core rev: f9a5dc7eeb99d211d728f8f7395aa22d4749e6a7) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-10oeqa/selftest: add test to verify that poisoned sysroots are detectedRoss Burton
Add a recipe that explicitly searches /usr/include, and use that in oe-selftest to verify that host include paths are correctly causing build failures. (From OE-Core rev: b3e3eba796b843021b264f0e98dc30f983775d58) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-06oeqa/weston: correctly run wayland-info when systemd is in useAlexander Kanavin
Under systemd weston's socket is in /run, not in /run/user/0. (From OE-Core rev: ea7dfd857ce3779745dccda0cbfd93b5e5de1a36) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02python3-cython: Remove build paths from debug sourcesRichard Purdie
There are build paths which are in generated sources injected into ${PN}-src. These are in generated files and don't influence the binary output. As such we can correct them at do_package time by tweaking in an appropriately injected function. This fixes the reproducbility of the recipe. (From OE-Core rev: cc1de818247c9ae329fdc0536658b3968a4caea2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02ltp: Fixing determinism issuesRichard Purdie
Add a patch adding sorting to a couple of points in the Makefiles which removes most of the determinism issues in ltp. Build swapon before the main build to ensure libswapon.o is built deterministically as it races with swapoff. All issues reported on the upstream mailing list. (From OE-Core rev: 0f51f9a37e5d058bce28cfe7b9a32a895f83c091) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02Add core-image-weston to reproducible build testsJan-Simon Moeller
This change adds core-image-weston to the reproducible build test. For this we also need to update the static-group and static-passwd. Besides the world package build which includes the wayland package, this adds a wayland/weston-based image to the reproducible build test. (From OE-Core rev: ffc4de47988ccf7568eecc8a27e8964beeaaacfb) Signed-off-by: Jan-Simon Moeller <jsmoeller@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02lttng: Fix reproducibility issuesRichard Purdie
Add a hack to hardcode in specific rpaths which we then remove, allowing the build to be reproducible. Strip build patches out of one of the test scripts too. (From OE-Core rev: 18299a114c66280ba2d00becc3fae235d3a6cbd7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02qemu: Determinism fixesRichard Purdie
Add a patch to address two determinism issues and allow reproducible builds. Also strip full paths we don't need out of some ptest files, we can use the installed binaries. (From OE-Core rev: 14e7cc6c6040b983024bec56fbd32d9101507ff0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02pinentry: update 1.1.0 -> 1.1.1Alexander Kanavin
Refresh xxx-config->pkg-config patches. (From OE-Core rev: 0995cd6083a42469aa2913c4d696b0b46e336854) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02selftest/reproducible: Add ability to pull some objects from sstateRichard Purdie
When debugging why a single recipe doesn't reproduce, its a pain to wait for the world to rebuild from scratch. Update the selftest to allow this to be configured, for example you could set targets as ['perf'] and sstate_targets as ['virtual/kernel'] and then it should only be rebuilding perf in the test rather than things like the toolchain (parts of the kernel may be unavoiable as they're not in sstate). Can be run as: OEQA_DEBUGGING_SAVED_OUTPUT=/tmp/perf-diffoscope oe-selftest -r reproducible.ReproducibleTests.test_reproducible_builds to save diffoscope output. (From OE-Core rev: 132a17d02f29711572e14a2f38a841323fbb6df6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01systemd-bootchart: Disable LTO to fix reproducibilityRichard Purdie
LTO likely doesn't buy us much here, disable it to allow the binaries to be reproducible. (From OE-Core rev: b4b5e52b5bd98deb0855700e4aae6228d52e5a83) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01efivar: Fix reproducibility issueRichard Purdie
Add sorting to the globbing within the Makefile to make the output reproducible. (From OE-Core rev: c43ae151f572786818fe048233b4bbfd6b0ba2cf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01swig: Fix reproducibility issueRichard Purdie
Remove hardcoding the build configuration into the help/version output from swig to make the binaries reproducible. (From OE-Core rev: 7ed28ae9717ea9dad4e131012186d5f08e8f0bec) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01syslinux: Fix reproducibility issuesRichard Purdie
Add sorting to wildcard expansion in the makefile to make builds reproducible. (From OE-Core rev: 5541ba76ccc0c416f315bc0dc14a20a33059bd5f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01gstreamer1.0-python: Set internal python library path correctyRichard Purdie
The library hardcodes paths to the python library internally and currently these are build paths. Fix this to use the correct target path and fix reproducibility in the process. (From OE-Core rev: 4c190e8c9a2b53de3def1065fb6e479b72d040dc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-01apr-util: Fix CFLAGS used in buildRichard Purdie
We need to use CFLAGS with the correct WORKDIR in them, replace those in the sysroot file with the ones appropriate to the current recipe. (From OE-Core rev: 45edf189961aff1858be9bb7b63116073c0a0c10) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-27libid3tag: Fix reproducibility issueRichard Purdie
Configure was swallowing our cflags meaning the resulting binaries were not reproducible. Tweak configure not to do that and fix reproducibility. (From OE-Core rev: 1395269a6fae5cec01d721e2ad52bd7cb8fc38ce) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-27igt-gpu-tools: Fix reproducibility issueRichard Purdie
Add a configuration option to pass in srcdir, removing hard coded build paths from the binaries. (From OE-Core rev: d7e92e3a22f0c87aff4f452b51f50ec417d2949b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-27bootchart2: Fix manpage reproducibility issueRichard Purdie
The compressed docs contained a timestamp of the original file which meant the SDE clamping during package creation didn't work. The benefits of compression are minor, decompress the files to avoid the reproducibility issues. (From OE-Core rev: a3b59a67bea16899b57a0d187120c988495da4d5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-27oeqa/pam: Need shadow installed for the testsKhem Raj
Distros may not use shadow for login manager secondly login --help returns 1 only with shadow provided login utility, with busybox it returns normal 0, the testcase clearly assumes shadow being part of image, therefore add needed checks to ensure the pre-requisites are met (From OE-Core rev: 0820aeba74869bf6e85b200f39bc610a43ab9a7a) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26selftest/reproducible: Don't call sync between each file compareRichard Purdie
Calling sync between each file compare is horrible performance wise as we compare thousands of files. We don't care about IO latency here so disable. (From OE-Core rev: 25f78abd8bbeb201fd9452e7983e015027954948) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26libhandy: Fix reproducibility issueRichard Purdie
(From OE-Core rev: ba559d5e07cbe92ee22308a1b5b8afc558c662dd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26libsecret: Improve determimismRichard Purdie
(From OE-Core rev: 28175c44e7402397a24a00a4b056eef39b0a03fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26parted: Fix reproducibility issueRichard Purdie
We don't use tbe BUILDINFO line of host information in the Makefile so remove it for reproducibility. (From OE-Core rev: a9742595fa90d4977fdd8129a4fe4932ddb96a18) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26gst-devtools: Fix reproducibility issueRichard Purdie
(From OE-Core rev: d221abc411b49b219b89cb38a56b2901d0bd5529) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26epiphany: Fix reproducibility issueRichard Purdie
We don't want to encide BUILD_ROOT into target packages. This is used for build time tests but in our case those would be on target anyway do use the target paths. (From OE-Core rev: 2818dc90bd62487879e7da2c68357b39282415b0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26gtk-doc: Fix reproducibility issueRichard Purdie
config_data was including a build system path to pkg-config, fix this. (From OE-Core rev: 7b8f56224c12a40f4f0aca3f612198deb933af4d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26gcr: Fix reproducibility issueRichard Purdie
(From OE-Core rev: b53133572de9e5bdf77dd44831a86ec34bf41b2c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26cups: Fix reproducibility issuesRichard Purdie
configure inspects the host's /etc/group for these configuration options, fix this to the correct values by using configure options. (From OE-Core rev: f16f9c727569414cd52862dcba18d8e423f4e961) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26oe-selftests: add rpm to reproducible build selftestJan-Simon Möller
rpm packaging has shown good numbers wrt reproducible builds as well when testing core-image-minimal locally: Reproducibility summary for rpm: same=4671 different=0 missing=0 total=4671 Thus enable it in the autobuilder to be more widely tested. (From OE-Core rev: 84d3a90557444d8cd83d780b1c3f2b278ab07af2) (From OE-Core rev: 85b9dd01f2e1d5429f8da0c48a25ec034acebf67) Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26reproduce: Fix exclusion list for rpmRichard Purdie
rpm uses "XXX-VERSION" compared to deb/ipk which use "XXX_VERSION" which breaks the go package exclusion. Work around with both go_ and go- for now. (From OE-Core rev: 2af535282833dddb58f24ec5b360230c47e6f6b1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26selftest: Replace building dep tool with direnvKhem Raj
dep tool is now deprecated and its replaced with go modules so using this as sample for testing is also getting arcane. Replace it with another project direnv[1] which is quite active and uptodate [1] https://github.com/direnv/direnv (From OE-Core rev: bb510037beaaaa723b9158d1f4e11a3547f6fb73) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-24selftest/reproducible: Remove exclusions for recipes which now reproduceRichard Purdie
git, libproxy, libcap-ng(-python), babeltrace2, rsync and groff have their reproducibility issues fixed, remove from the exclusion list. Also fix whitespace. (From OE-Core rev: 044eda9c70ddda195b1c0ce1d228904392431582) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-23kernel-fitimage: Don't use unit addresses on FITKlaus Heinrich Kiwi
Das U-Boot 2021.4-rc1 has the following commit: commit 3f04db891a353f4b127ed57279279f851c6b4917 Author: Simon Glass <sjg@chromium.org> Date: Mon Feb 15 17:08:12 2021 -0700 image: Check for unit addresses in FITs Using unit addresses in a FIT is a security risk. Add a check for this and disallow it. CVE-2021-27138 Adjust the kernel-fitimage.bbclass accordingly to not use unit addresses. This changte is required before we can bump U-Boot to 2021.4. (From OE-Core rev: 6047be9f8f0f5d616fda11d83b682c1b8aeaa0ae) Signed-off-by: Klaus Heinrich Kiwi <klaus@linux.vnet.ibm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-21selftest/incompatible_lic: Update the tests after the 'or-later' license ↵Richard Purdie
handling changes With the separate of the "-only" and "-or-later" licenses, we need to update the tests to match the messages now given in the output. Also use a mix of canonicalised and non-canonlised names in the reference recipes to help test those cases and ensure coverage. (From OE-Core rev: b7f38af7ac9449178c603c5349808b8c0dd84d35) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-19df.py: Add feature check for read-only-rootfsWes Lindauer
If the target is using a read-only rootfs, the available space on '/' will be zero. This will cause the test to incorrectly fail and skipping seems appropriate in this case. (From OE-Core rev: af1dbea3c9b9b42a3e6803b231e425423d70e210) Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-19oeqa/runlevel : add test for runlevelsTeoh Jay Shen
This test mimic the boot_from_runlevel_3 and boot_from_runlevel_5 test cases from oeqa/manual/bsp-hw.json. The boot_from_runlevel_3 and boot_from_runlevel_5 manual test cases should be remove from oeqa/manual/bsp-hw.json if this patch get merged. (From OE-Core rev: 6effd30f0e3726bc1f2eb7768c57c6b95eddb079) Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-17Revert "oe-selftests: add rpm to reproducible build selftest"Richard Purdie
This reverts commit 84d3a90557444d8cd83d780b1c3f2b278ab07af2. This shouldn't have been merged, its not ready yet. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-17oe-selftests: add rpm to reproducible build selftestJan-Simon Möller
rpm packaging has shown good numbers wrt reproducible builds as well when testing core-image-minimal locally: Reproducibility summary for rpm: same=4671 different=0 missing=0 total=4671 Thus enable it in the autobuilder to be more widely tested. (From OE-Core rev: 84d3a90557444d8cd83d780b1c3f2b278ab07af2) Signed-off-by: Jan-Simon Möller <jsmoeller@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-17acpica: Fix reproducibility issuesJoshua Watt
Fixes reproducibility when build acpica yacc sources by instructing bison on how to do the file prefix mapping (From OE-Core rev: d13a1df7bd173fca3fc2cdf889d5eb487e35d6fd) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>