summaryrefslogtreecommitdiffstats
path: root/meta/classes
AgeCommit message (Collapse)Author
2023-12-29testimage: drop target_dumper and host_dumperSteve Sakoman
The target_dumper code is basically broken. It has been reading binary files over the text base serial communication and runs at every command failure which makes no sense. Each run might overwrite files from the previous run and the output appears corrupted due to confusion from the binary data. It isn't possible to cherry-pick "testimage: Drop target_dumper and most of monitor_dumper" from master, so just make target_dumper and host_dumper empty functions. For further details see: https://lists.openembedded.org/g/openembedded-architecture/message/1888 (From OE-Core rev: 94e9019d2f170a26206c2774381a1d183313ecaa) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-12-21testimage: Exclude wtmp from target-dumper commandsRichard Purdie
wtmp is filled with binary data which the run_serial command can't cope with. Catting this results in confusion of the serial interface and potentially large backlogs of data in the buffers which can hang qemu. Exclude the problematic files from the command. (From OE-Core rev: 05d9f9c6b27c0216fa4e349109ef42cf91bb4084) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 599ac08a6f6fb3f6a89a897c8e06367c63c2f979) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-11-17cve-check: sort the package list in the JSON reportRoss Burton
The JSON report generated by the cve-check class is basically a huge list of packages. This list of packages is, however, unsorted. To make things easier for people comparing the JSON, or more specifically for git when archiving the JSON over time in a git repository, we can sort the list by package name. (From OE-Core rev: 5a509bc6f26247cc7561189d582c91816042fd91) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e9861be0e5020830c2ecc24fd091f4f5b05da036) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-10-20kernel.bbclass: Add force flag to rm callsRyan Eatmon
The latest 6.5 kernels do not appear to create the source file in ${D}${nonarch_base_libdir}/modules/${KERNEL_VERSION}/source so the recipe errors out when trying to remove it. Simple fix is to add the -f (force) flag to the call. (From OE-Core rev: 2e669bf797b15d803e7d6a700e449bdc467a4bcc) (From OE-Core rev: 844faa7c51ae8ec0966e9c5c3f70a1dbf2222c21) Signed-off-by: Ryan Eatmon <reatmon@ti.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Alexander Sverdlin <alexander.sverdlin@siemens.com> Signed-off-by: Steve Sakoman <steve@sakoman.com> Backported from kirkstone commit 7e177848f97e. Signed-off-by: Paul Barker <paul.barker.ct@bp.renesas.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-09-15kernel: Fix path comparison in kernel staging dir symlinkingStaffan Rydén
Due to an oversight in the do_symlink_kernsrc function, the path comparison between "S" and "STAGING_KERNEL_DIR" is broken. The code obtains both variables, but modifies the local copy of "S" before comparing them, causing the comparison to always return false. This can cause the build to fail when the EXTERNALSRC flag is enabled, since the code will try to create a symlink even if one already exists. This patch resolves the issue by comparing the variables before they are modified. (From OE-Core rev: 27a982807caa7ffbdf2d4ef02bc0b037150b1b3b) Signed-off-by: Staffan Rydén <staffan.ryden@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit afd2038ef8a66a5e6433be31a14e1eb0d9f9a1d3) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-09-15rootfs-post: remove traling blanks from tasksPriyal Doshi
remove the traling blanks before the ;-delimiter, so one could use "_remove" to avoid running tasks like 'rootfs_update_timestamp', which are currently hardcoded and not bound to any configurable feature flag (From OE-Core rev: 18246f0bfedb5c729a0fc5b515f25a1ed0cde191) Signed-off-by: Priyal Doshi <pdoshi@mvista.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-08-16kernel: skip installing fitImage when using Initramfs bundlesAbdellatif El Khlifi
When including an initramfs bundle inside a FIT image, the fitImage is created after the install task by do_assemble_fitimage_initramfs. This happens after the generation of the initramfs bundle (done by do_bundle_initramfs). So, at the level of the install task we should not try to install the fitImage. The fitImage is still not generated yet. After the generation of the fitImage, the deploy task copies the fitImage from the build directory to the deploy folder. Change-Id: I3eaa6bba1412f388f710fa0f389f66631c1c4826 (From OE-Core rev: b70a8333a7467162b9d148b99f5970c0af2a531f) Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1b67fd9ac74935fa41e960478c54e45422339138) Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-22kernel-fitimage: fix dtbo support for fit imagesAnthony Bagwell
8a2f4e143 added support for u-boot boot script but missed adding the extra parameter to fitimage_emit_section_config on the dtbo branch (From OE-Core rev: d1b6c34d33704f05374154e4ea7d8acdea7b8018) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 22bac8aea0d5d28cc5a3bf20edf638225cce2f88) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-12useradd-staticids.bbclass: improve error messageMikko Rapeli
Current error message is difficult to read: ERROR: Nothing PROVIDES 'image' trs-image was skipped: image - image: normal username test does not have a static ID defined. Add test to one of these files It's not clear that first "image" is recipe name, second "image" is binary package name and that "test" is the user account which does not have a static ID defined. Improve the error message so that these are more explicit. Now the error message looks like: image was skipped: Recipe image, package image: normal username "test" does not have a static ID defined. (From OE-Core rev: ea997ec788a5397598e24301e40d1c30ffa68c04) Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 07898218f3908a83e07178b6530dfa48d55d4ec2) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-12go.bbclass: don't use test to check output from lsMartin Jansa
* avoids possibly confusing error message in log.do_install like: ls: cannot access 'etcd/3.5.7-r0/build/bin/linux_arm64/': No such file or directory (From OE-Core rev: 236a3e0bf753669b4fddbd91be5fdb10c2e6093f) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 2f1777e6ac5269a71203b6a2c562a43503be95ae) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-27kernel-fitimage: use correct kernel imageAndrej Valek
Even if initramfs_bundle_path was used, a wrong compression was reflected in output its template file. Use linux.bin as universal kernel image. The linux.bin file covers both cases because it's beying created from vmlinux. We know, that vmlinux is created inside compressed directory already, so no external compression will be used. (From OE-Core rev: 8280c089b0df97368476b8bc54ed758e809f42f5) Signed-off-by: Andrej Valek <andrej.valek@siemens.com> Signed-off-by: Walter Schweizer <walter.schweizer@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Backport of 27fbbeaf972bb7e2535c1b23375cfa9d66b69db6 in master branch Signed-off-by: Remi Peuvergne <remi.peuvergne@non.se.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-27kernel-fitimage: adding support for Initramfs bundle and u-boot scriptAbdellatif El Khlifi
This commit adds Initramfs bundle support to the FIT image in addition to u-boot boot script capability. These new features are selectable. In case of Initramfs, the kernel is configured to be bundled with the rootfs in the same binary (ie: zImage-initramfs-<machine>.bin). When the kernel is copied to RAM and executed, it unpacks the Initramfs rootfs. For more information about Initramfs please read: https://www.kernel.org/doc/Documentation/filesystems/ramfs-rootfs-initramfs.txt For more details about the Initramfs bundle and boot script implementation please check the kernel-fitimage.bbclass paragraph in Yocto reference or mega manual. Current limitations: - Initramfs bundle FIT support has been tested on ARM 32-bit - The kernel image type in case of ARM 32-bit is zImage Change-Id: I901bfd899e8d733c5b9a2b6645b1d4748f4b1fda (From OE-Core rev: e3eb79c0a11e9f67af3161361fb88b00123ecbac) Signed-off-by: Abdellatif El Khlifi <abdellatif.elkhlifi@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Backport of 19fa415c8769a67b52babd80f71d68bf36a21db2 in master branch Signed-off-by: Remi Peuvergne <remi.peuvergne@non.se.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-27uninative.bbclass: handle read only files outside of patchelfSteve Sakoman
We are seeing autobuilder failures with the latest uninative: patchelf: open: Permission denied See upstream discussion which suggests handling read-only files explicitly outside of patchelf: https://github.com/NixOS/patchelf/pull/89 (From OE-Core rev: 3680fcbf9ba327aedb32bb4eb647dd56fcccfd51) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-27uninative: Ensure uninative is enabled in all cases for BuildStarted eventRichard Purdie
Recent changes in bitbake mean the datastore is not always reset between ConfigParsed and BuildStarted. This means in a fresh buiild, with memory resident bitbake active, uninative may end up disabled. Update the code so the enable code is always run at BuildStarted if needed. (From OE-Core rev: 24897c70980f934cf59a28a0d3b6e8fddcb15a7f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit db743cc78fe5172bb4a4dac9c1dad4f5aa9e1491) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-27cve-update-nvd2-native: new CVE database fetcherMarta Rybczynska
Add new fetcher for the NVD database using the 2.0 API [1]. The implementation changes as little as possible, keeping the current database format (but using a different database file for the transition period), with a notable exception of not using the META table. Minor changes that could be visible: - the database starts in 1999 instead of 2002 - the complete fetch is longer (30 minutes typically) [1] https://nvd.nist.gov/developers/vulnerabilities (From OE-Core rev: 191e2c6d7009999f049a0cf8460259f8f91a829d) Signed-off-by: Marta Rybczynska <marta.rybczynska@syslinbit.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit fb62c4c3dbca4e58f7ce6cf29d4b630a06411a97) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-05-25populate_sdk_ext.bbclass: set METADATA_REVISION with an DISTRO overrideMartin Jansa
* otherwise it ends '<unknown>' inside esdk, because of parsing order: # $METADATA_REVISION [3 operations] # set /OE/build/test-D/conf/local.conf:43 # "f2da54ef432eac89b0f18eaad68e602b6990b5de" # immediate /OE/build/test-D/layers/poky/meta/classes/metadata_scm.bbclass:9 # "${@oe.buildcfg.detect_revision(d)}" # set /OE/build/test-D/layers/poky/meta/classes/metadata_scm.bbclass:10 # [vardepvalue] "${METADATA_REVISION}" # pre-expansion value: # "<unknown>" METADATA_REVISION="<unknown>" * This causes base-files.do_install and following tasks to have different signatures between esdk and the build directory where this esdk was created: bitbake-diffsigs {test-D,poky/build-uninative-disabled}/tmp/stamps/qemux86_64-poky-linux/base-files/*do_install*sigdata* NOTE: Starting bitbake server... basehash changed from 5b6981cf58bfd57d416b0e31611b73a26baae635dd1ac31c08d46f95064c3ffc to dbdce042da4d7813d632b6d1cc87a16f728ad20e55fecbc392830e6acf72babd Variable METADATA_REVISION value changed from '<unknown>' to 'f2da54ef432eac89b0f18eaad68e602b6990b5de' and an warning from "python3 /OE/build/test-D/ext-sdk-prepare.py" when eSDK is being prepared for use: WARNING: The base-files:do_install sig is computed to be 83b9c9a6ef1145baac5a1e0d08814b9156af239c58fc42df95c25a9cd8a7f201, but the sig is locked to 3dc22233059075978e5503691e98e79e7cc60db94259dfcd886bca2291c0add7 in SIGGEN_LOCKEDSIGS_t-qemux86-64 [RP: Add commit about why we need the override for future reference] (From OE-Core rev: 8f0e2e505d11697d0a1280b0a77a3fe8d72a4d49) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> (cherry picked from commit 675ea7281c17f77bf5dea17cfd4d9da0928382a0) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-05-25pypi.bbclass: Set CVE_PRODUCT to PYPI_PACKAGEAlex Kiernan
The CVE product name for PyPI packages is (usually) the same as the PyPI package name (and not our recipe name), so use that as the default. (From OE-Core rev: 1c37b96cd4fdfad21bf24b8b883e371c3bca56de) Signed-off-by: Alex Kiernan <alexk@zuma.ai> Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 61f6b0ad09bf87cdc2d3f08770b7c44cad1d0e58) Signed-off-by: Sanjay Chitroda <schitrod@cisco.com> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-05-03populate_sdk_base: add zip optionsChristoph Lauer
Add SDK_ZIP_OPTIONS to remove symbolic link creation in zip archive or add options, e.g. for encryption of the zip archive. (From OE-Core rev: 04b62f9459b401c276255f166d0738b6f902a576) (From OE-Core rev: b9e0c3ced645cab74b2488a26b8f656a94b2a6f5) Signed-off-by: Christoph Lauer <christoph.lauer@xtronic.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-04-14cve-check: Fix false negative version issueGeoffrey GIRY
NVD DB store version and update in the same value, separated by '_'. The proposed patch check if the version from NVD DB contains a "_", ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison. [YOCTO #14127] Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr> (From OE-Core rev: eb439b1283b60e6665694ff28c89fbd633eda6b0) Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 7d00f6ec578084a0a0e5caf36241d53036d996c4) Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-04-01classes/package: Use gzip for extended package dataJoshua Watt
The master version of extended package data uses zstd for efficient compression, but it relies on the zstd tool to be present on the host system. Since dunfell supports older distros, we don't want to add this tool as an additional requirement so switch to using gzip instead. (From OE-Core rev: 1c7d555379c4b0962bccd018870989050d87675f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-01create-spdx: Use gzip for compressionJoshua Watt
The master version of the SPDX classes uses zstd for efficient compression, but it relies on the zstd tool to be present on the host system. Since dunfell supports older distros, we don't want to add this tool as an additional requirement so switch to using gzip instead. (From OE-Core rev: 55b08fd641767a3d2e080727828ca37d9e610109) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-01classes/package: Add extended packaged dataJoshua Watt
Adds extended package data which is encoded as JSON which allows it to encode more structure than the "flat" package data files. The extended data might be much larger than the standard package data, so it is not read by default and instead requires oe.packagedata.read_subpkgdata_extended() to be called Currently, the file sizes and ELF debug sources are saved off into the extended package data (cherry picked from commit db9cf430e54ae68da80fbc3fba80ce88d8df164d) (From OE-Core rev: 2318cb79ef1652fb19d696a8d43826e39c22834e) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Reviewed-by: Saul Wold <saul.wold@windriver.com> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-01classes/create-spdx: BackportJoshua Watt
Backports the create-spdx classes from the latest versions on master. This backport is a simple copy with no modifications, as its too complex to cherry-pick all the corresponding changes. This will give an appropriate base commit for subsequent changes and if necessary additional backport cherry-picks from master in the future. (From OE-Core rev: 48fbddf32ffa3ec44a788f42895c1730a84b5a91) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-25populate_sdk_ext: Handle spaces within user $PATHKenfe-Mickael Laventure
The script generated by the sdk_ext_postinst function was not quoting the user existing PATH when updating it causing the export command to fail. Add necessary double quotes around $PATH. (From OE-Core rev: b8298401c710146ae35f282048b66d5773a513ed) Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 00e96bf250eaaded839caf465dbc0af5b604aed7) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-25toolchain-scripts: Handle spaces within user $PATHKenfe-Mickael Laventure
The environment-setup script generated by the recipe was not quoting the user existing PATH when updating it causing the export command to fail. Add necessary double quotes around $PATH. (From OE-Core rev: ad6aaa77f1d08b4ca66f46e318f1a09b645d9e23) Signed-off-by: Kenfe-Mickael Laventure <mickael.laventure@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 42177ff2d45ee70ad00917bb6fbabca49dae4f59) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-25staging/multilib: Fix manifest corruptionRichard Purdie
The previous fix wasn't enough to address all the possible ways the manifests might be ordered. Rework the previous fix so it is tied to the multilib cross-canadian code which is causing the problem. RECIPE_SYSROOT_MANIFEST_SUBDIR is not documented as I'd hope nobody ever needs to use this outside the core multilib code. (From OE-Core rev: 271ac31e15d00d636e888a1f3b7d4e22e0b8267a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit beab42e00713880cd95a04729c892f8662fbcbed) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-25staging: Separate out different multiconfig manifestsRichard Purdie
""" require conf/multilib.conf MACHINE = "qemuarm64" MULTILIBS = "multilib:lib32" DEFAULTTUNE:virtclass-multilib-lib32 = "armv7athf-neon" bitbake gcc-cross-canadian-arm """ and then inspecting the lib32* manifest files under recipe-sysroot-native shows them referencing lib32-recipe-sysroot instead of recipe-sysroot as used by gcc-cross-canadian recipes. To fix this separate out the manifest by multilib. It is caching mechanism to optimise disk usage so this doesn't break anything, just separated out some files. (From OE-Core rev: eb9c6f7ac91431a6296789c23900f2a3016ef619) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 891d3faa3ed3d1cc231da58e5fa1325f05d5ade5) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-15linux: inherit pkgconfig in kernel.bbclassMing Liu
pkgconfig is being required to find dependencies for building kernel native tools, move "inherit pkgconfig" to kernel.bbclass so BSP kernel recipes can also benefit from it. (From OE-Core rev: a5e62888768be641d5da00fd847e0cac96820c75) Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 8a84bd98e3fbc16c782f83064801e469d086911e) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14devshell: Do not add scripts/git-intercept to PATHPeter Kjellerstedt
The use of scripts/git-intercept was introduced in commit 3266c327df (install/devshell: Introduce git intercept script due to fakeroot issues) and later reverted in commit af27c81eaf (scripts: Make git intercept global). (From OE-Core rev: 51424b9955374196307aaf73cf4b6c184ce4fb6d) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> (cherry picked from commit f6c260c8e2a33e282a35afc99de4ef8cc1791b08) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14image.bbclass: print all QA functions exceptionsMauro Queiros
For the QA checks in `image.bbclass`, all exceptions other than `oe.utils.ImageQAFailed` always print the following generic message: "Image QA function func_name failed" This can be very misleading, as it may hide python syntax errors and other kind of issues that are hard to detect without more explicit error messages. This change makes sure that the error message of all exceptions are displayed. Before this change: "Image QA function func_name failed" After this change: "Image QA function func_name failed: f-string: empty expression not allowed (<string>, line 13)" (From OE-Core rev: c4e3b86f2bcb2b445efc72bd8e06b1b89d88daa2) Signed-off-by: Mauro Queiros <maurofrqueiros@gmail.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 3d85b30d8704d38b86f5b006748cebc74bd2a4fa) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-14kernel-yocto: fix kernel-meta data detectionUlrich Ölmann
Fixes: 7ef7af5c03ba ("kernel-yocto: restore kernel-meta data detection for SRC_URI elements") (From OE-Core rev: 4533e8363549f87a0484d7c0a43a162c918f33bc) Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c77754f23e3fb49a62602a6c6a04d5525d1cf457) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24libc-locale: Fix on target locale generationRichard Purdie
If on target locale generation is used, it fails at first boot showing errors about a missing directory. Ensure the directory exists. (From OE-Core rev: 63141bd8c19285bbef33f3d32a89ca127da0c658) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit f2844c9f1bbb729562063d96a3d1cc9d44dafa0a) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24meta: remove True option to getVar and getVarFlag calls (again)Martin Jansa
* True is default since 2016 and most layers were already updated not to pass this parameter where not necessary, e.g. oe-core was updated couple times, first in: https://git.openembedded.org/openembedded-core/commit/?id=7c552996597faaee2fbee185b250c0ee30ea3b5f Updated with the same regexp as later oe-core update: https://git.openembedded.org/openembedded-core/commit/?id=9f551d588693328e4d99d33be94f26684eafcaba with small modification to replace not only d.getVar, but also data.getVar as in e.g.: e.data.getVar('ERR_REPORT_USERNAME', True) and for getVarFlag: sed -e 's|\(d\.getVarFlag \?\)( \?\([^,()]*, \?[^,()]*\), \?True)|\1(\2)|g' \ -i $(git grep -E 'getVarFlag ?\( ?([^,()]*), ?([^,()]*), ?True\)' \ | cut -d':' -f1 \ | sort -u) (From OE-Core rev: 4ec04d14899cb7725ce908e3ef6302838275f0a8) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 26c74fd10614582e177437608908eb43688ab510) Signed-off-by: Steve Sakoman <steve@sakoman.com> (cherry picked from commit 24a86d0c55ee89ae0dc77975e1d0ee02898d2289) Signed-off-by: Steve Sakoman <steve@sakoman.com> (cherry picked from commit de7bf6689a19dc614ce4b39c84ffd825bee1b962) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24classes/fs-uuid: Fix command output decoding issuePawel Zalewski
The default return value from subprocess.check_output is an encoded byte. The applied fix will decode the value to a string. (From OE-Core rev: 622fea338a0c43f95185469b54cc72c545b73100) Signed-off-by: Pawel Zalewski <pzalewski@thegoodpenguin.co.uk> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 046769fa952a511865c416b80d10af6287147fb7) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-24nativesdk: Handle chown/chgrp calls in nativesdk do_install tasksRichard Purdie
We disable the useradd code for nativesdk targets since we don't support postinstalls or multiple users in those cases. This means any usage of chown/chgrp inside do_install tasks won't work and would have to be conditional. Rather than require all recipes to do that, add intercepts of the calls and map those to root/root user/groups. We can't just ignore them as some calls are used to remove host contamination from the host user ID so they need to be made, just as root. (From OE-Core rev: a05c116c517e0700837c335d0691c3fcc4462dda) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit e1f40670c438e33cae87678425de72ca03566888) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-13Fix missing leading whitespace with ':append'Niko Mauno
Mitigate occurences where ':append' operator is used and leading whitespace character is obviously missing, risking inadvertent string concatenation. (From OE-Core rev: fcd340ec53ff8352b8cae0eb351810072b025a08) (From OE-Core rev: b21e5524e3326c77598d30f6fe9268a5fdac91dd) Signed-off-by: Niko Mauno <niko.mauno@vaisala.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-13devtool: fix devtool finish when gitmodules file is emptyThomas Roos
When a .gitmodules file exists but is empty then devtool finish fails. Add an additional check for this. [YOCTO #14999] (From OE-Core rev: 077edd3992683985f0779afc73d4207a795ced39) Signed-off-by: Thomas Roos <throos@amazon.de> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit b4f0f7c4934bade9e4d4a1086f9d8b29d8e9ad45) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-13toolchain-scripts: compatibility with unbound variable protectionJan Kircher
Fixed an error when Bash's unbound variable protection is enabled (set -u) and variable "LD_LIBRARY_PATH" does not exist. (From OE-Core rev: 2c7309eea616ec88338d508bc9f284e3bd44f5a1) Signed-off-by: Jan Kircher <openembedded@hetsh.de> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 85685370b0ad93291cda59fb091a15eeecf5e0d5) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-13cve-check: write the cve manifest to IMGDEPLOYDIRJermain Horsman
When building an image cve_check_write_rootfs_manifest() would sometimes fail with a FileNotFoundError when writing the manifest.cve due to the parent directory (DEPLOY_DIR_IMAGE) not (yet) existing. The image task will provide the manifest in the deploy directory afterwards, so other recipes depending on the manifest being in DEPLOY_DIR_IMAGE should continue to function properly. (From OE-Core rev: c25a5c5f6fede29893a2ac1502216e9deccfa6c3) Signed-off-by: Jermain Horsman <jermain.horsman@nedap.com> Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 00fb2aae22ce0d7ff5f3f8766fa770eeb4e73483) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-13rm_work.bbclass: use HOSTTOOLS 'rm' binary exclusivelyLuis
The do_rm_work() task is using the first available 'rm' binary available in PATH to remove files and folders. However, depending on the PATH setup and RECIPE_SYSROOT_NATIVE contents, the function can be using the 'rm' binary available in RECIPE_SYSROOT_NATIVE, a folder that will get removed. This causes a sporadic race-condition when trying to access the 'rm' binary of a folder already deleted. Solve this by exclusively using the HOSTTOOLS 'rm' binary, as this folder will not get removed. (From OE-Core rev: 75b7e86c9d9931c9e4e114af026b51710f1920a2) Signed-off-by: Luis Martins <luis.pinto.martins@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit edcd9ad333bc4e504594e8af83e8cb7007d2e35c) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-13base.bbclass: Fix way to check ccache pathChangqing Li
The previous code had 2 issues: 1. make hosttools/ccache always link to host's ccache (/usr/bin/ccache) even we have one buildtools 2. make hosttools/gcc etc, link to host's gcc event we have one buildtools when keyword ccache in buildtools's path, eg: /mnt/ccache/bin/buildtools This patch is for fix above issues. (From OE-Core rev: f2f70bf8d93b33b65875828c0402a98e943f660a) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 1b7c81414cf252a7203d95703810a770184d7e4d) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-06qemuboot.bbclass: make sure runqemu boots bundled initramfs kernel imageJagadeesh Krishnanjanappa
The QB_DEFAULT_KERNEL is set to pick bundled initramfs kernel image if the Linux kernel image is generated with INITRAMFS_IMAGE_BUNDLE="1". This makes runqemu to automatically pick bundled initramfs kernel image instead of explicitly mentioning bundled initramfs kernel image in runqemu. [YOCTO #14748] (From OE-Core rev: 0c63018ef3843cfefd2be31c0a6693181037410d) Signed-off-by: Jagadeesh Krishnanjanappa <workjagadeesh@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit 52371624313184e1a825519160c3833e282df8b9) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-01-06externalsrc: fix lookup for .gitmodulesPeter Marko
Commit 0533edac277080e1bd130c14df0cbac61ba01a0c broke bitbake parsing when bitbake is executed from directory with existing .gitmodules and the recipe in externalsrc does not have .gitmodules The check needs to search for .gitmodules in sources path, not cwd. iParsing recipes...ERROR: ExpansionError during parsing <path to recipe> ... bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception CalledProcessError: Command '['git', 'config', '--file', '.gitmodules', '--get-regexp', 'path']' returned non-zero exit status 1. (From OE-Core rev: c58d82b98348b167b60ec3c8b9651d73b1bdfbdc) Signed-off-by: Peter Marko <peter.marko@siemens.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 66ff3d1f65cd2e7f5319e98fa41f47a59b714c72) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-23classes: make TOOLCHAIN more permissive for kernelAlexey Smirnov
Currently TOOLCHAIN is strictly set to gcc in kernel-arch.bbclass. And this prevents any TOOLCHAIN changes for any kernel recipe. This change makes TOOLCHAIN configurable as usual. (From OE-Core rev: 7d45d4696970c45534b905dc7a65418073108f97) Signed-off-by: Alexey Smirnov <pyih.soft@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit be1634fc35dcc81f0301d942064a6eed584e0704) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-07kernel.bbclass: make KERNEL_DEBUG_TIMESTAMPS work at rebuildChen Qi
Currently, the KERNEL_DEBUG_TIMESTAMPS is not working as expected at rebuild. That is, even if we set it to "1", the kernel build time is not changed. The problem could be reproduced by the following steps. 1. bitbake core-image-minimal; start image and check `uname -a` output. 2. set in local.conf: KERNEL_DEBUG_TIMESTAMPS = "1" 3. bitbake core-image-minimal; start image and check `uname -a` output. It's expected that after enabling KERNEL_DEBUG_TIMESTAMPS, the kernel build time will be set to current date. But it's not. This is because the compile.h was not re-generated when do_compile task was re-executed. In mkcompile_h, we have: """ # Only replace the real compile.h if the new one is different, # in order to preserve the timestamp and avoid unnecessary # recompilations. # We don't consider the file changed if only the date/time changed, # unless KBUILD_BUILD_TIMESTAMP was explicitly set (e.g. for # reproducible builds with that value referring to a commit timestamp). # A kernel config change will increase the generation number, thus # causing compile.h to be updated (including date/time) due to the # changed comment in the # first line. """ It has made it very clear that it will not be re-generated unless we have KBUILD_BUILD_TIMESTAMP set explicitly. So we set this variable explicitly in do_compile to fix this issue. (From OE-Core rev: e44f0cda8176186d42a752631810c1cb5f1971eb) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 1b68c2d2d385013a1c535ef81172494302a36d74) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-07kernel: improve transformation from KERNEL_IMAGETYPE_FOR_MAKEMike Crowe
In 526bdd88ccd758204452579333ba188e29270bde the imageType loop in kernel_do_deploy was changed to use KERNEL_IMAGETYPE_FOR_MAKE rather than KERNEL_IMAGETYPES. This broke the special handling for fitImage immediately below because KERNEL_IMAGETYPE_FOR_MAKE never contains fitImage. It has always been my understanding that KERNEL_IMAGETYPE_FOR_MAKE controlled what was passed to make, but KERNEL_IMAGETYPE controlled what was installed/deployed. When the two are different then it's the responsibility of whoever set KERNEL_IMAGETYPE_FOR_MAKE to ensure that whatever comes out of the kernel build system has been transformed in to the requested form by the time of installation. This is what happens for kernel.bbclass's own support for vmlinux.gz. I think this means that for KERNEL_IMAGETYPE vmlinux.gz, kernel.bbclass is responsible for generating vmlinux.gz.initramfs[1] so that kernel_do_deploy can deploy it. This means that the change in 526bdd88ccd758204452579333ba188e29270bde can be reverted, fixing KERNEL_IMAGETYPE = "fitImage". In addition, it ought to be possible for recipes and other classes that use kernel.bbclass to hook into this mechanism by setting KERNEL_IMAGETYPE_FOR_MAKE and performing their own transformations. do_bundle_initramfs calls kernel_do_compile and we don't want it to transform vmlinux to vmlinux.gz at that point, since it will fight against the careful renaming and preserving that do_bundle_initramfs does. Let's separate the transformation out of kernel_do_compile to a new do_transform_kernel task that can be run at the right time. This means that it's also logical to perform the equivalent translation for the kernel with the initramfs in a separate do_transform_bundled_initramfs task too. This leaves two clear customisation points for recipes and other classes to hook into the process and perform their transformations: do_transform_kernel and do_transform_bundled_initramfs. (I care about this because our recipes that use kernel.bbclass also set KERNEL_IMAGETYPE_FOR_MAKE and transform vmlinux into a form suitable for our bootloader after do_compile and do_bundle_initramfs into the format matching KERNEL_IMAGETYPE. I'm unable to successfully bundle an initramfs after 526bdd88ccd758204452579333ba188e29270bde, but I didn't want to just revert that change to reintroduce the bug that it was fixing.) I can't say that I'm entirely happy with this change, but I'm unsure what to do to improve it. I find the way that both the bare kernel and the one with the initramfs both get deployed to be confusing, and a waste of build time. I would like to not actually generate a publishable kernel image at all during do_compile when an initramfs is in use, but I suspect that this would affect valid use cases that I'm not aware of. (From OE-Core rev: d9e9653616638f2b187d5e04540071ac34d99f56) Signed-off-by: Mike Crowe <mac@mcrowe.com> [1] It could be argued that this should be vmlinux.initramfs.gz, but that would require another special case in kernel_do_deploy and the filename is only visible within this class and the recipes that use it anyway. Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 10a4a132e87e835726bf5da81a60f6f509b90765) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-12-07sstate: Account for reserved characters when shortening sstate filenamesManuel Leonhardt
Previously, when shortening sstate filenames, the reserved characters for .siginfo were not considered, when siginfo=False, resulting in differently shortend filenames for the sstate and siginfo files. With this change, the filenames of the truncated sstate and siginfo files have the same basename, just as is already the case for untruncated filenames. Making sure that the .siginfo files always have the filename of the corresponding sstate file plus its .siginfo suffix, also when being truncated, makes it easier to manage the sstate cache and an sstate mirror outside of Bitbake/Yocto. (From OE-Core rev: 408bf1b4bb4f4ed126c17fb3676f9fa0513065ba) Signed-off-by: Manuel Leonhardt <mleonhardt@arri.de> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> (cherry picked from commit c2e0e43b7123cf5149833e0072c8edaea3629112) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-20sanity: check for GNU tar specificallyRoss Burton
We need the system tar to be GNU tar, as we reply on --xattrs. Some distributions may be using libarchive's tar binary, which is definitely not as featureful, so check for this and abort early with a clear message instead of later with mysterious errors. (From OE-Core rev: 8f852648fe730615c99bcdaace8a4748ef4e96a5) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 7dd2b1cd1bb10e67485dab8600c0787df6c2eee7) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-20externalsrc.bbclass: Remove a trailing slash from ${B}Peter Kjellerstedt
The trailing slash in ${B} caused -fdebug-prefix-map=${B}=... to not match as intended, resulting in ${TMPDIR} ending up in files in ${PN}-dbg when externalsrc was in use, which in turn triggered buildpath QA warnings. (From OE-Core rev: fa89e048e223ae2e96d0f55979f93f29904b5229) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 9b5031ed5a0d102905fa75acc418246c23df6eef) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2022-11-20externalsrc.bbclass: fix git repo detectionMartin Jansa
* fix issue introduced in: https://git.openembedded.org/openembedded-core/commit/?id=95fbac8dcad6c93f4c9737e9fe13e92ab6befa09 * it added check for s_dir + git-dir (typically '.git') isn't the same as ${TOPDIR} + git-dir, but due to copy-paste issue it was just comparing it with s_dir + git-dir again, resulting in most external repos (where git-dir is '.git') to be processed as regular directory (not taking advantage of git write-tree). * normally this wouldn't be an issue, but for big repo with a lot of files this added a lot of checksums in: d.setVarFlag('do_compile', 'file-checksums', '${@srctree_hash_files(d)}') and I mean *a lot, e.g. in chromium build it was 380227 paths which still wouldn't that bad, but the checksum processing in siggen.py isn't trivial and just looping through all these checksums takes very long time (over 1000sec on fast NVME drive with warm cache) and then https://git.openembedded.org/bitbake/commit/?id=b4975d2ecf615ac4c240808fbc5a3f879a93846b made the processing a bit more complicated and the loop in get_taskhash() function took 6448sec and to make things worse there was no output from bitbake during that time, so even with -DDD it looks like this: DEBUG: virtual/libgles2 resolved to: mesa (langdale/oe-core/meta/recipes-graphics/mesa/mesa_22.2.0.bb) Bitbake still alive (no events for 600s). Active tasks: Bitbake still alive (no events for 1200s). Active tasks: Bitbake still alive (no events for 1800s). Active tasks: Bitbake still alive (no events for 2400s). Active tasks: Bitbake still alive (no events for 3000s). Active tasks: Bitbake still alive (no events for 3600s). Active tasks: Bitbake still alive (no events for 4200s). Active tasks: Bitbake still alive (no events for 4800s). Active tasks: Bitbake still alive (no events for 5400s). Active tasks: Bitbake still alive (no events for 6000s). Active tasks: DEBUG: Starting bitbake-worker without -DDD it will get stuck for almost 2 hours in: "Initialising tasks..." before it finally writes sstate summary like: "Sstate summary: Wanted 3102 Local 0 Mirrors 0 Missed 3102 Current 1483 (0% match, 32% complete)" * fix the copy&paste typo to use git work-tree in most cases, but be aware that this issue still exists for huge local source trees not in git [YOCTO #14942] (From OE-Core rev: 1f0e4de8d92edd7438d462c779d917ac0ccd5499) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> (cherry picked from commit 9102e5a94b8146cb1da27afbe41d3db999a914ff) Signed-off-by: Steve Sakoman <steve@sakoman.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>