summaryrefslogtreecommitdiffstats
path: root/bitbake
AgeCommit message (Collapse)Author
2021-10-11bitbake: bitbake: Bump to version 1.52.0Richard Purdie
(Bitbake rev: c78ebac71ec976fdf27ea24767057882870f5c60) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-11bitbake: hashserv: Improve behaviour for better determinism/sstate reuseRichard Purdie
We have a choice of policy with hashequivalence - whether to reduce sstate duplication in the sstate feed to a minimum or have maximal sstate reuse from the user's perspective. The challenge is that non-matching outhashes are generated due to determinism issues, or due to differences in host gcc version, architecture and so on and the question is how to reconcile then. The approach before this patch is that any new match is added and matches can update. This has the side effect that a queried value from the server can change due to the replacement and you may not always get the same value from the server. With the client side caching bitbake has, this can be suboptimal and when using the autobuilder sstate feed, it results in poor artefact reuse. This patch switches to the other possible behaviour, once a hash is assigned, it doesn't change. This means some sstate artefacts may be duplicated but dependency chains aren't invalidated which I suspect may give better overall performance. Update the tests to match the new behaviour. (Bitbake rev: 20d6ac753efa364349100cdc863e5eabec8e5b78) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-11bitbake: hashserv: Fix diverging report race conditionJoshua Watt
Fixes the hashequivalence server to resolve the diverging report race error. This error occurs when the same task(hash) is run simultaneous on two different builders, and then the results are reported back but the hashes diverge (e.g. have different outhashes), and one outhash is equivalent to a hash and another is not. If taskhash was not originally in the database, the client will fallback to using the taskhash as the suggested unihash and the server will see reports come in like: taskhash: A unihash: A outhash: B taskhash: C unihash: C outhash: B taskhash: C unihash: C outhash: D Note that the second and third reports are the same taskhash, with diverging outhashes. Taskhash C should be equivalent to taskhash (and unihash) A because they share an outhash B, but the server would not do this when tasks were reported in the order shown. It became clear while trying to fix this that single large table to store all reported hashes was going to make these updates difficult since updating the unihash of all entries would be complex and time consuming. Instead, it makes more sense to split apart the database into two tables: One that maps taskhashes to unihashes and one that maps outhashes to taskhashes. This should hopefully improve the parsing query times as well since they only care about the taskhashes to unihashes table, at the cost of more complex INNER JOIN queries on the lesser used API. Note this change does delete existing hash equivlance data and starts a new database table rather than converting existing data. (Bitbake rev: dff5a17558e2476064e85f35bad1fd65fec23600) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-11bitbake: hashserv: Add tests for diverging reportsJoshua Watt
(Bitbake rev: 953c8d622c9d1bc1eb06bcaf1eaa3aa9f85d0bc2) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-11bitbake: async: Close sync client event loopJoshua Watt
Prevents `ResourceWarning: unclosed event loop` warnings when using the synchronous client and python exits (Bitbake rev: 8b95972bc04ce52a98c7780184af15a5e95f987b) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: bitbake: replace http with https for URLsJon Mason
https has been the preferred way to access websites for many years now. Change all of the URLs with a _working_ https server/certificate to use that URL. (Bitbake rev: da543cdaf88a387675e25d3555765f1146e4105e) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: bitbake:toaster:test: Update SSTATE URLJon Mason
(Bitbake rev: b3c0dbddd7eb3c87e3989977d7640f09b49a460b) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: bitbake-worker: Handle pseudo shutdown in Ctrl+C caseRichard Purdie
If the build is interrupted, handle the shutdown of pseudo even in this case to avoid data corruption inside docker containers. [YOCTO #14555] (Bitbake rev: a2a04c6fe94bc56efcff299c669a151746e35916) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: knotty/uihelper: Show setscene task progress in summary outputRichard Purdie
With the changes to task accounting, bitbake doesn't show progress when executing setscene tasks on the summary console output. Change to show a progress within the setscene tasks and a progress within the main tasks. I can't see any way to display this more easily without confusing users. [YOCTO #14586] (Bitbake rev: 0244acb968eb593d2ad7bc6e52f222c2b1d39aa9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: fetch2: npmsw: Add support for local tarball and link sourcesStefan Herbrechtsmeier
(Bitbake rev: 4f983dc419a1a6f635a5d333f253d49244cec374) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: fetch2: npm: Create config npmrc in environment instantiationStefan Herbrechtsmeier
Create a configuration npmrc per npm environment to avoid repeated creation of the same configuration file. Create the file via python to avoid multiple npm config calls and add the ability to pass a file path instead of a temporary file. Deprecate the npm configs argument of the run function. The configs should be passed to npm environment or as command specific arguments. (Bitbake rev: 2c2df49b06a2bad7a5b8872a9998338a4660498f) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: fetch2: npm: Support npm archives with missing search directory modeStefan Herbrechtsmeier
Delay directory restore and set execute/search directory mode bits in unpack to support npm archives with a missing search directory mode. (Bitbake rev: 60cbd34d3da8f0f523281aad7eec93eec9cd4db8) Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: fetch2/gitsm: remove the 'nugget' SRCREV cachingRoss Burton
The cached revisions which are used to decide if a repository doesn't need to be updated are misleading when used in conjunction with mirror tarballs and can cause partial fetches to happen, resulting in unpack errors as repositories were not fetched. A concrete example: edk2-firmware in meta-arm is at version 202102 (ef91b0). This is built on the autobuilder so the source mirror contains the repository as a mirror tarball. If I build edk2-firmware 202102 the gitsm fetcher will initially download the top-level repository and then iterate into the submodules to also fetch those repositories, including cmocka from cryptomilk.org. edk2-firmware will then unpack and build successfully. I then update edk2-firmware to 202105 (e1999b) and build it. Gitsm.needs_update() starts by calling Git.needs_update() which returns False, as the mirror tarball contains this revision. It then looks at the "nuggets" which are SRCREVs it has fetched before. The mirror tarball itself contains the nugget for e1999b as this has been built on the autobuilder, so needs_update return False, no more fetching is done, and the build proceeds to unpack. However, as part of the 202105 upgrade the URL of the cmocka submodule changed, and this new repository was never fetched. This means that unpack fails as one of the required git repositories isn't available. The nugget codepaths appear to be an attempt at optimising the fetch process, but have demonstratable failure cases. Just removing them entirely solves the edk2-firmware example, and all of the fetcher test cases still pass. (Bitbake rev: 51212507ce3f670ace9efb691c92887d66f7aaf8) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: fetch2: clarify the command-no-found error messageRoss Burton
If runfetchcmd() fails with bb.process.NotFoundError, the message output is simply "Fetch command" which doesn't really explain what the problem is. Add "not found" to clarify what happened. (Bitbake rev: 8de9dc02ed6a73b47f2ab10be30d1aed7954bc72) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: tests/fetch: prefix the FetcherTest temporary directoryRoss Burton
Set a prefix so the temporary directories are identifable. (Bitbake rev: c3440b82cbe9c317f9961d61e12ea37fc9541ce0) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-08bitbake: tests/utils: mark a regex as a raw stringRoss Burton
Avoids a warning from Python as \s isn't a valid escape. (Bitbake rev: be39c2e926beebe78030fbe26c6737f08f960fcb) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-03bitbake: data: Ensure functions are defined in a deterministic orderRichard Purdie
When writing functions into shell scripts, write then in a deterministic order. This is unlikely to affect anything at runtime but it does change the signatures of the generated useradd postinst scripts in OE-Core and is a good thing to be consistent about in general. (Bitbake rev: 8a181dc8f3c8c9c9885ea3011cb234321a296d92) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-26bitbake: cooker/command: Add a dummy event for tinfoil testingRichard Purdie
We need a command genetating an event to test through the tinfoil API. The current test has IO load issues so add a dummy version which won't have the IO constraints. (Bitbake rev: a144178584394ea0700ffc2f2bfac94e8690effc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-26bitbake: siggen: Fix sorting in diff outputRichard Purdie
The diff output isn't deterministic at the moment as the sets can have differing ordering. Sort the output so it is consistent. (Bitbake rev: 117830c1d7ef3e53052fa326e1ca62c5c3946c45) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-26bitbake: bitbake-worker: Allow shutdown/database flush of pseudo server at ↵Richard Purdie
task exit We have a problem where pseudo server processes exist after bitbake exits and hold the pseudo database in memory. In a docker container, the processes will be killed as the container is destroyed with no warning and no opportunity to write the data to disk. This leads to permissions/inode corruptions and data loss. Send a shutdown message to pseudo which in new versions of pseudo will flush the database, thereby fixing some of the issues people using docker containers see. (Bitbake rev: a07a971b40acd3eee12e203d2cfa3e49f56109f6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-23bitbake: build: Fix log flushing raceRichard Purdie
There is a race between sending the TaskFailed event which could trigger the UI to look at the logs and flushing and closing the log files. Reverse the order of the finally clause and the exception handling to ensure we've handled the logfiles before sending the task events. This should fix a race seen in bblogging.BitBakeLogging.test_python_exit_logging (Bitbake rev: 032190aac31604d37740d8aecf6e74a5448de358) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-23bitbake: build: Make exception printing clearerRichard Purdie
Shows: ERROR: SystemExit(1) instead of: ERROR: 1 (Bitbake rev: df89e37c33e4398a5f8ece9a8b973be3fe2ff361) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-23bitbake: build: Ensure python stdout/stderr is logged correctlyRichard Purdie
Currently we see things like: Log data follows: | DEBUG: Executing python function do_pythontest_exit | DEBUG: Python function do_pythontest_exit finished | ERROR: 1 | This is python stdout Whilst after the change we see things like: Log data follows: | DEBUG: Executing python function do_pythontest_exit | This is python stdout | DEBUG: Python function do_pythontest_exit finished | ERROR: 1 since the output is now correctly mixed with the log messages. In some cases the logging tests indicate the output is being lost entirely which is bad for debugging and makes things rather confusing. (Bitbake rev: 8966b43761500e0505333d8c9a3f0f2c3dbe7559) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-22bitbake: tests/fetch2: Use our own git server for dtc test repoRichard Purdie
(Bitbake rev: 36f6dce1b21a7d9f39a73f081395c71045960318) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-21bitbake: fetch2/svn: Allow peg-revision functionality to be disabledRichard Purdie
Sometimes the peg revision functionality we use in the svn fetcher is not the correct option. Add a parameter nopegrevision which can be added to disable this behaviour. [YOCTO #6258] (Bitbake rev: ea26682b706f655a8e418f56bebe742e2d130a1d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-21bitbake: data_smart: Don't add None to ExpansionError varlistRichard Purdie
If a "None" value gets into the varlist, it doesn't display properly. Ensure we don't add one to have the exception display properly. (Bitbake rev: ee26e258888114143e66330c256b5bfe7d071c53) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-21bitbake: tests/fetch2: Fix quoting warningRichard Purdie
Fix: lib/bb/tests/fetch.py:1288: DeprecationWarning: invalid escape sequence for several lines of the fetch tests. (Bitbake rev: 9d84fd557a3fcbae2cdd70b24e69325ad737a01e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-21bitbake: fetch2/git: Use os.rename instead of mvRichard Purdie
os.rename will overwrite the destination file if present so we can use this instead of the process call overhead. (Bitbake rev: b3cccaa6a896c41d8c9be5eebc327f726542d16b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-19bitbake: cookerdata: Show a readable error for invalid multiconfig nameRichard Purdie
If a multiconfig starts with a digit, users would see pages of errors as we use the multiconfig as a python function name prefix and python functions cannot start with a digit. We could avoid doing that but it is easier just to ask users to name multiconfigs not starting with digits. This tweak ensures the user sees an easier to understand error. (Bitbake rev: f9cddaeef35b2ea0dadf717101ed896f6b857abd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-19bitbake: bitbake: enable python warnings at the first opportunityAlexander Kanavin
We really do want to see those, as they tend to turn into hard errors eventually, as what happened with collections vs collections.abc in python 3.10. (Bitbake rev: bc43fbb86361a21dc2d5deb910810c5a77fdabe8) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: fetch2/s3: allow to switch profile from environment variableValentin Danaila
Make usage of Bitbake's s3 fetcher more flexible with different AWS profiles and switch between profiles as export of the AWS_PROFILE environment variable (Bitbake rev: 0f35dc4dfc829fe9c51c239d15567f86c5c14e58) Signed-off-by: Valentin Danaila <drlv85@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: bitbake: correct deprecation warning in process.pyAlexander Kanavin
(Bitbake rev: aff52fe21a0b27f6302555c1e52a864550eb46ce) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: bitbake: adjust parser error check for python 3.10 compatibilityAlexander Kanavin
The change was introduced in https://github.com/python/cpython/commit/a698d52c3975c80b45b139b2f08402ec514dce75 (Bitbake rev: 8d3c6cbbe6ee734495713ae3b99c609527842506) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: bitbake: do not import imp in layerindexlibAlexander Kanavin
The module is deprecated and unused. (Bitbake rev: 76c9030d6c91cd776a1aa732fb92b7cd4539b503) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: bitbake: fix regexp deprecation warningsAlexander Kanavin
See here for details: https://docs.python.org/3/library/re.html (Bitbake rev: 660e6ad4abb77c6f3c1d48bd64777dd76c05d7e2) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: bitbake: correct the collections vs collections.abc deprecationAlexander Kanavin
This becomes a hard error in python 3.10. (Bitbake rev: ae219e1f7460077f4492b31ac91cef4cf9b17277) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: bitbake: drop old rules for python warningsAlexander Kanavin
These no longer even work, and it's much better to just see all warnings and fix them as they happen. (Bitbake rev: 62d96ea2afbb0cfdb0d1f932dc96643e7bcd7f26) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: fetch2/git: Avoid races over mirror tarball creationRichard Purdie
There is a potential race over the mirror tarballs where a partial git repo could be extracted causing fetcher failures if the tarball is being rewritten whilst another build accesses it. Create the mirror tarball atomically to avoid this. [YOCTO #14441] (Bitbake rev: 3250bc950c56bd7dd2114df26e5a8e13b04ceac8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-17bitbake: runqueue/knotty: Improve UI handling of setscene task countingRichard Purdie
The recent fixes to merge setscene and normal task accounting in runqueue fixed some display issues but broke the task numbering of setscene tasks. Add new accounting methods to the stats structure specifically designed for setscene. This accounts for the fact that setscene tasks can rerun multiple times in the build. Then use the new data in the UI to correctly display the numbers the user wants to see to understand progress. (Bitbake rev: ed7e2da88bf4b7bfc7ebfc12b9bd6c0fb7d8c1aa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-16bitbake: bitbake-worker: Improve error handlingRichard Purdie
If bitbake-worker fails, return an error code showing that. Also make the thread cleanup code explict in a finally clause as it would otherwise hang. [YOCTO #14393] (Bitbake rev: 7e0af70fb53fb13f824ca954b8cc1dffee730233) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-16bitbake: fetch2/wget: Enable ftpsDaniel Ammann
The fetcher would fail with: Could not find a fetcher which supports the URL: ftps://... (Bitbake rev: 9e56710c7203b1ec6cbefa758c81b69b697fe1a4) Signed-off-by: Daniel Ammann <daniel.ammann@bytesatwork.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-14bitbake: bitbake.conf: fix vars_from_file() callMartin Jansa
The reference to this function was dropped from BBHandler.py in: https://git.openembedded.org/bitbake/commit/?id=aaa5292ef96ea27f505bc5c5a4b1eb4f497ed061 (Bitbake rev: b4fe8507079b6464287549f3eac1f1fc8d0d9be2) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11bitbake: cooker: Allow upstream for local hash equivalence serverJoshua Watt
The hash equivalence server has had the option to support a read-only upstream server for some time now when launched as a standalone program, but there was no way to set the upstream when using a locally started server. Add a new variable called BB_HASHSERVE_UPSTREAM that can be used to specify an upstream server when a local hash equivalence server is used (e.g. BB_HASHSERVE is "auto") (Bitbake rev: 250fa17f1391ff1ee01ab9b51d2a4f9aa35c1d1e) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11bitbake: runqueue: Clean up task stats handlingRichard Purdie
When we parallelised normal and setscene tasks, the task stats handling was left separate pending further thought. We had to remove handling of the setscene tasks from the UI in order to maintain consistent task numbering. Currently, "0 of 0" tasks can be shown as setscene tasks execute until the first normal task runs. The only use left for sq_stats is in the active task numbers which we can use the length of sq_ive for instead. We can therefore drop it and return stats in all cases. This removes the "0 of 0" task problem since the stats in all normal and setscene tasks matches. [YOCTO #14479] (Bitbake rev: eae6e947e37e18cded053814bd2a268b44fb25cd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11bitbake: cookerdata: Show error for no BBLAYERS in bblayers.confRichard Purdie
If there is no BBLAYERS set in bblayers.conf show a more helpful error and exit. [YOCTO #14340] (Bitbake rev: 97183e10faf9862b5d9489d6e2c27ac77c3b697d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11bitbake: cookerdata: Improve missing core layer error messageRichard Purdie
If the core layer is missing from bblayers.conf, the message the user sees is hard to understand. Improve it. [YOCTO #14340] (Bitbake rev: 5815a7258ebb8a989e0c6f5798853559d9413f02) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11bitbake: fetch2: Add recursion guardRichard Purdie
Users sometimes put ${S} references in ${SRC_URI} without realising this can be problematic. Improve the error messages if they accidentally do. [YOCTO #11593] (Bitbake rev: 89e0b19ec0b245a6cd414088904c91808e8814ab) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11bitbake: data_smart: Improve error display for handled exceptionsRichard Purdie
We don't need tracebacks for BBHandledException. Reduces confusing output like: ERROR: /meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio ERROR: ExpansionError during parsing /meta/recipes-core/images/core-image-tiny-initramfs.bb Traceback (most recent call last): File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_qa', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1050, in follow_chain(task='do_image_cpio', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in deps: > follow_chain(othertask, endtask, chain) chain.pop() File "/bitbake/lib/bb/build.py", line 1038, in follow_chain(task='do_image_complete', endtask='do_build', chain=['do_image_complete', 'do_packageswu', 'do_image_qa', 'do_image', 'do_image_cpio']): if task in chain: > bb.fatal("Circular task dependencies as %s depends itself via the chain %s?!" % (task, " -> ".join(chain))) chain.append(task) File "/bitbake/lib/bb/__init__.py", line 165, in fatal: mainlogger.critical(''.join(args), extra=kwargs) > raise BBHandledException() to the real error: ERROR: /media/build1/poky/meta/recipes-core/images/core-image-tiny-initramfs.bb: Circular task dependencies as do_image_complete depends itself via the chain do_image_complete -> do_packageswu -> do_image_qa -> do_image -> do_image_cpio (Bitbake rev: 551d4c0576a0a0c3406000029df9238b312f2263) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11bitbake: build: Catch and error upon circular task referencesRichard Purdie
If there are circular task references, error on them rather than show a recursion error. A simple reproducer is: """ do_packageswu () { : } addtask do_packageswu after do_image_complete before do_image_qa """ into image_types.bbclass. There is code in runqueue to detect these but we never get that far with the current codebase. [YOCTO #13140] (Bitbake rev: 339d4d6be515a71311b81fb9e99742af0d8a5130) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-11bitbake: data_smart: Make ExpansionErrors more readableRichard Purdie
This adds context to ExpansionError messages which show the variable chain for which expansion is being attempted. This should allow users to debug the issues more easily than the current message (the first line alone below). Example output from a SRC_URI which references ${S}: bb.data_smart.ExpansionError: Failure expanding variable PV, expression was 0.1+git${SRCPV} which triggered exception RecursionError: maximum recursion depth exceeded while calling a Python object The variable dependency chain for the failure is: PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> WORKDIR -> S -> SRC_URI -> SRCPV -> PV -> BP -> FILESPATH which is more useful that no output. We could truncate at repetition but I suspect this makes this clearer as it stands so there is little value in complicating the code. (Bitbake rev: 699634bec47964fa7ab18689dc23db6f0bc22fb3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>