summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb
AgeCommit message (Collapse)Author
2019-11-21bitbake: utils: also use mmap for SHA256 and SHA1, for performanceRoss Burton
md5_file() uses a mmap() window to improve performance when hashing files, so refactor the code and do the same for SHA1 and SHA256. (Bitbake rev: ecf87437ff796e17c3e4f210b5803b0136a9e8a4) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-21bitbake: tests: add test for the hashing functionsRoss Burton
Add a basic test for bb.utils.md5_file() etc. (Bitbake rev: e944d02fe678f7c6b05c62419f8bceb0709f3037) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-14bitbake: cooker: Remove a left-over comment about expanded_dataPeter Kjellerstedt
This should have been removed together with expanded_data in commit e3694e73 (cooker/command: Drop expanded_data). (Bitbake rev: 33197db8abf82be240d7c1c5c9d2484a08a90849) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-07bitbake: fetch2/hg: Fix various runtime issuesVolker Vogelhuber
Fix mercurial fetching after breakage from changes to the core fetcher. Fix username and password usage and setting moddir needed by setup_revisions. (Bitbake rev: 8962b27abc68427eae085624f26300108ae88aa1) Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-07bitbake: runqueue: Improve sstate rehashing outputRichard Purdie
Bibake is currently too 'chatty' when hash equivalence is enabled. Fix this by only printing the log output if a rehash happens and it matches an sstate object. Also, pass a summary option to the hash checking function. This was already changed to a mechanism which allows addition of new parameters so this should be backwards and forwards compatible. (Bitbake rev: 0c4515603ad08775e3b0404cba5374367e49f236) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-11-05bitbake: fetch2: Ensure cached url data is matched to a datastoreRichard Purdie
There was a weird error in OE-Core where "devtool modify virtual/kernel" was showing basehash mismatch errors. This was due to SRCPV sometimes being: AUTOINC+b867b78b50_47b80ef7bd and sometimes AUTOINC+b867b78b50_255a750d28. The latter hash comes from KBRANCH and meant sometimes the correct branch was seen, sometimes it was not. The issue was complicated by the execution using a remote datastore over tinfoil. The problem turns out to be a fetcher caching error. If the datastore changes, the cached url data may not be valid. We therefore ensure we match cached url data against the datastore that generated it, which appears to fix this issue. (Bitbake rev: 0540abd338ed2ffd822edbd2947cab2f18873422) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-28bitbake: tests/fetch: add test for fetching shallow revsChristopher Larson
[YOCTO #13586] (Bitbake rev: f7a973604fbfd1059875ff1fabb3f885df9c5b95) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-28bitbake: fetch2/git: fetch shallow revs when neededChristopher Larson
When bitbake determines if a git clone needs updating, it only checks for the needed srcrevs, not the revs listed in BB_GIT_SHALLOW_REVS, which will fail if using shallow and the needed rev was added to the upstream git repo after a previous fetch. Ensure that we also check for shallow revs. [YOCTO #13586] (Bitbake rev: 93b65e3c6ca64f644946953980595c44fbbcc748) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-28bitbake: bitbake: ConfHandler: Use with to manage filehandle lifetimeOla x Nilsson
(Bitbake rev: 459ad524756a3f9b50feeedf31e33502dceae8d5) Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-09bitbake: bitbake: Update to version 1.44.0Richard Purdie
(Bitbake rev: 5d83d828cacb58ccb7c464e799c85fd2d2a50ccc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-02bitbake: bitbake: Bump verison 1.43.1 -> 1.43.2uninative-2.7Richard Purdie
This allows metadata to depend on SignatureGeneratorUniHashMixIn which was recently added. (Bitbake rev: f0f814407fdd2fffa7071c36c011b489bfcd53da) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-02bitbake: toaster: improve warnings when adding dependency to packagesDavid Reyna
Some of the objects that bitbake reports to Toaster as dependencies to packages are known objects that are not packages, for example library files and kernel modules. In the Toaster logs, mark these as "Info" instead of "Warning". [YOCTO #13386] (Bitbake rev: 0d66f644d647900e8f5afa526a6d9cee687c41cc) Signed-off-by: David Reyna <David.Reyna@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-10-02bitbake: tests/runqueue: Fix hashserve shutdown raceRichard Purdie
The hashserve can delete its socket whilst the cleanup us happening leading to backtraces and test failures. Add code to avoid this race condition. [YOCTO #13542] (Bitbake rev: efd7b025cee25d0ee668c09476395d08fcf5ae1a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30bitbake: siggen: Remove full path from unitaskhashes keysRichard Purdie
The full paths make the cache useless in the sdk. They also bloat the cache size. They're for human debugging benefit only so compromise and reduce this to the filename. (Bitbake rev: 3b275c4083eae1d3781f0862919af9de83932b0f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30bitbake: runqueue: Small performance optimisationRichard Purdie
A minor performance optmisation to keep lists smaller when running large builds. We can do this since once a task has been built, we don't need to worry about it. This improves a major bottleneck that shows up on performance profile charts in dryruns. (Bitbake rev: cd6b89230823707c3c9bb9e6883bf5a971916581) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30bitbake: runqueue: Save unihashes more frequentlyRichard Purdie
There are some runqueue code paths where the unihash cache would not be saved where for example only parsing or an occurred. Save the cache at the end of runqueue generation to ensure entries are cached. (Bitbake rev: 9eee0d36870c11dd303894a6151c33a83bd3a1bc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30bitbake: siggen: Avoid writing misleading sigdata filesRichard Purdie
Use the unihash in the output filename of sigdata files else the contents of stamp directories is misleading. Write the unihash into the singature to make it clear what happened. (Bitbake rev: feb01ee54d3706fe93768f332054c7532f7209e4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30bitbake: siggen/runqueue: Fix signature mismatch issuesRichard Purdie
We need to set the setscene tasklist before we call into the taskhash/unihash code else the behaviour is inconsistent. Avoid reporting hashes for non setscene tasks since we'd never query that. (Bitbake rev: 419a7840b8627278db694029c25df00214d01d96) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30bitbake: runqueue: Change task migration behaviour for rerunning setscene tasksRichard Purdie
Currently runqueue will rerun setscene tasks multiple times as hashes change. This has caused numerous problems since a setscene task may become "unavailable" for some future signature combination and the code then can't easily "unskip" tasks its already passed into the execution queue. At least for now, only run setscene once and assume they're equivalent at that point. In practise that has been much more stable in testing. Tweak the test to match the change in behaviour. (Bitbake rev: 4205a3ef23834f317642bba155d67cd772176fb6) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30bitbake: siggen: Ensure setscenetasks list is available to worker contextRichard Purdie
The setscenetasks list needs to be available in the worker contexts else the signature behaviour there mismatches what the server does. Add the data to get/set_taskdata to ensure this happens. (Bitbake rev: 632980ef90fe126b7ba3d138f4d574ae05914779) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-30bitbake: runqueue: Fix task migration problemsRichard Purdie
Tasks were not migrating consistently, particularly: * if a task was rehashed which had already run * if a task which was valid became invalid due to a rehash We need to always run the migration code for rehashed tasks and then reprocess them for hash validity. This means rearranging the code. It also means several tests are no longer correct and can't be written correctly to work on all possible workflows so those are removed. (Bitbake rev: 8443989ee41e9b162972935513e437b5c66ea74d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27bitbake: hashserv: Don't daemonize server processJoshua Watt
The hash server process is terminated and waited on with join(), so it should not be a daemon. Daemonizing it cause races with the server cleanup, especially in the selftest because the process may not have terminated and cleanup up its socket before the test cleanup runs and tries to do it. [YOCTO #13542] (Bitbake rev: 7c829675581818f92d57056b57fbd3880829b6bd) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27bitbake: siggen: Fix attribute error when hashserver failsJoshua Watt
The HashConnectionError class was moved to the client module and needs to be updated. [YOCTO #13537] (Bitbake rev: 9fb862685e5e5a2aa534bc25cab1e4158d708b40) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27bitbake: cookerdata: Add mc conffiles hashes to cache hashJoshua Watt
The variable values that result from parsing multiconfig should be included in the cooker data hash, otherwise changes to these files won't be detected, which will allow the parsing cache to be loaded with the old values for the multiconfigs. This can either manifest as the variable values simply not updating, or getting basehash changed errors when building. This bug was previously undetected because all of the multiconfig base files were a direct file dependency in all parsed recipes. This was fixed in 34137a00f60 ("bitbake: bitbake: cooker: Rename __depends in all multiconfigs"), exposing this bug. [YOCTO #13541] (Bitbake rev: c74481aa15226e1bff9d53e4ee4b702ebfa1ad32) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27bitbake: tests/data: Test combinations of _append together with overrideJacob Kroon
(Bitbake rev: f31f35e8527c60a95931a4a8311a4cd237770b42) Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27bitbake: tests/fetch: add test case for git-lfs handlingRoss Burton
Add a test case to exercise the detection of git-lfs repositories and the behaviour of the lfs parameter. (Bitbake rev: a7cf4fc72cce357c425084dc2c5f35b5ed1a4b7b) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-27bitbake: fetch2/git: refactor check for git-lfs commandRoss Burton
Refactor the git-lfs checking: this means both clearer code in the download() function and allows unit testing to monkeypatch the functionality. (Bitbake rev: 33cf9172ded50a869f7201ba463ab9ecc69b8252) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-19bitbake: utils: Add ionice option to prunedirRichard Purdie
Autobuilder type infrastructure can benefit from deletion of certain files as background IO due to the way Linux filesystem priority works. We have problems where build directories as part of oe-selftest being delete starves the running tasks of IO to the point builds take much longer to compelte. Having this option of running the deletion at "idle" helps a lot with that. (Bitbake rev: 797354d285f6d624d9adb52bab65823572da0e39) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-19bitbake: runqueue: validate_hashes(): currentcount should be a numberRobert Yang
According to sstate_checkhashes which is defined in sstate.bbclass, the currentcoun should be a number (0, not None). Fixed: $ bitbake base-files -Sprintdiff > bb.plain("Sstate summary: Wanted %d Found %d Missed %d Current %d (%d%% match, %d%% complete)" % (total, len(found), len(missed), currentcount, match, complete)) TypeError: %d format: a number is required, not NoneType (Bitbake rev: 45cb73e2846eaffe8964a573875f54808e8f3633) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-19bitbake: fetch2/git: add git-lfs toggle optionRoss Burton
Add a new 'lfs' option to the git fetcher so that we can optionally not fetch git-lfs content, for repositories that contain LFS data that we don't actually need for building. By default lfs is set to 1, so if the repository has LFS content then git-lfs is required. Setting lfs to 0 will mean that git-lfs won't be required to fetch, and some files will be missing. (Bitbake rev: be0b78ccfc5ede98041bc0545a15092494b12b26) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18bitbake: svn fetcher: allow "svn propget svn:externals" to failMikko Rapeli
Not all servers and repositories have this property set which results in failures like this when actual svn checkout command succeeded: svn: warning: W200017: Property 'svn:externals' not found on '' svn: E200000: A problem occurred; see other errors for details (Bitbake rev: 238636f033cbf18e5741f0ea0e64db40e84f5838) Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-18bitbake: bitbake: Rework hash equivalenceJoshua Watt
Reworks the hash equivalence server to address performance issues that were encountered with the REST mechanism used previously, particularly during the heavy request load encountered during signature generation. Notable changes are: 1) The server protocol is no longer HTTP based. Instead, it uses a simpler JSON over a streaming protocol link. This protocol has much lower overhead than HTTP since it eliminates the HTTP headers. 2) The hash equivalence server can either bind to a TCP port, or a Unix domain socket. Unix domain sockets are more efficient for local communication, and so are preferred if the user enables hash equivalence only for the local build. The arguments to the 'bitbake-hashserve' command have been updated accordingly. 3) The value to which BB_HASHSERVE should be set to enable a local hash equivalence server is changed to "auto" instead of "localhost:0". The latter didn't make sense when the local server was using a Unix domain socket. 4) Clients are expected to keep a persistent connection to the server instead of creating a new connection each time a request is made for optimal performance. 5) Most of the client logic has been moved to the hashserve module in bitbake. This makes it easier to share the client code. 6) A new bitbake command has been added called 'bitbake-hashclient'. This command can be used to query a hash equivalence server, including fetching the statistics and running a performance stress test. 7) The table indexes in the SQLite database have been updated to optimize hash lookups. This change is backward compatible, as the database will delete the old indexes first if they exist. 8) The server has been reworked to use python async to maximize performance with persistently connected clients. This requires Python 3.5 or later. (Bitbake rev: 2124eec3a5830afe8e07ffb6f2a0df6a417ac973) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-16bitbake: runqueue/siggen: Optimise hash equiv queriesRichard Purdie
We only have hash equivalence for setscene tasks so only query the server for those, reducing the number of connections needed. (Bitbake rev: 22082c7b3ca0cffcedb7d1d8c6681d35286376db) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-16bitbake: Revert "bitbake: cooker: Ensure bbappends are found in stable order"Martin Jansa
This reverts commit 94c0c7f15c7a6244a8576ed948ffc21afb96ba82. This ignores the layer priority, making the issue much worse. E.g. I'm seeing a lot of failures caused by missing users, because base-passwd bbappends applied in unexpected order caused different passwd.master to be found in re-ordered FILESPATH. (Bitbake rev: 2dc862237dba82da37c8ac9289e0a21409b1305c) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-07bitbake: bitbake: cooker: Ensure bbappends are found in stable orderWes Lindauer
Thanks to wildcards in bbappend filenames, it's possible to have multiple bbappends that apply to the same recipe in the same directory. In order to get sstate hits between different workspaces, we want to apply those bbappend files in a consistent order. Since readdir() returns files in a non-deterministic order between workspaces (based on inode number and/or time of creation), we'll need to sort its result in order to have any consistency. (Bitbake rev: 94c0c7f15c7a6244a8576ed948ffc21afb96ba82) Signed-off-by: Wes Lindauer <wesley.lindauer@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-03bitbake: fetch2: show warning when renaming the archive with bad checksum failedMartin Jansa
* noticed on read-only sshfs premirror * it was showing the warning about renaming the file: WARNING: laser-geometry-1.6.4-r0 do_fetch: Renaming /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz to /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441 and then failed because of movefile() issue with python3 (fixed in previous commit): ERROR: laser-geometry-1.6.4-r0 do_fetch: Error executing a python function in exec_python_func() autogenerated: with movefile() fixed, it let do_fetch continue and re-fetch locally with the right checksum, but still the renamed file didn't exist, because of movefile failure - add another warning when the movefile fails - for whatever reason - unfortunately movefile prints error messages with just print() so the real error is hidden only in log.do_fetch in this case: movefile: Failed to move /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz to /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441 [Errno 30] Read-only file system: '/jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz' -> '/jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441' (Bitbake rev: 9a1bf4ba9ec00c2a222d820f8f83d1f056b021d6) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-03bitbake: utils: Fix movefile() exception handling with python3Martin Jansa
* with python3 this fails with: File: 'bitbake/lib/bb/utils.py', lineno: 799, function: movefile 0795: try: 0796: os.rename(src, destpath) 0797: renamefailed = 0 0798: except Exception as e: *** 0799: if e[0] != errno.EXDEV: 0800: # Some random error. 0801: print("movefile: Failed to move", src, "to", dest, e) 0802: return None 0803: # Invalid cross-device-link 'bind' mounted or actually Cross-Device Exception: TypeError: 'OSError' object is not subscriptable (Bitbake rev: d6e43c443ddbbe467c4380c48d2bc28ae18504a1) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-01bitbake: tests/fetch: Resolve fetch error in bitbake-selftestArmin Kuster
FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0]) AssertionError: '' is not true : Could not find upstream version for db [YOCTO #13496] The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget. Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the recipe. Also change the version being checked. (Bitbake rev: 4cf5bb761c561ddea86f2875be35d05abc8486e1) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-28bitbake: cooker.py: remove generation of recipe-depends.dotChen Qi
The information of recipe-depends.dot is misleading. e.g. $ grep xz recipe-depends.dot | grep bzip2 "bzip2" -> "xz" "xz" -> "bzip2" Users would wonder why they get some circular dependency. The information is derived from removing the task names of task-depends.dot. It's not giving people any additonal information, and it's misleading. So we remove the generation of this file. (Bitbake rev: 4c484cc01e3eee7ab2ab0359fd680b4dbd31dc30) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-28bitbake: bitbake: respect force flag in runall and runonlyJoshua Watt
Specifying the force flag will now cause runall and runonly to invalidate the tasks before running them. This allows a --runall or --runonly to force the tasks to run, even if they would have otherwise been skipped, e.g.: bitbake -f --runall fetch Will run all do_fetch tasks even if they wouldn't be necessary (for example, skipped by setscene) (Bitbake rev: 71e52d3822016027106f2a2e74b8dfdf20f5dc1e) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-21bitbake: runqueue: Optimise build_taskdepdata slightlyRichard Purdie
Rather than repeatedly calling mc_from_tid() do this in the parent, removing around a million function calls. Takes time spent in this function from 40s to 36s. (Bitbake rev: 28b3f0d8867804799420689c314ac4a8f01efb8c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-21bitbake: runqueue: Further optimise holdoff tasksRichard Purdie
There are other data structures which can be reprocessed at the same time as holdoff_tasks, further improving build efficiency in various places. (Bitbake rev: 02090b3456b7a2de12e72dfeaabfd3b631609924) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-21bitbake: runqueue: Optimise holdoff task handlingRichard Purdie
We don't need to process the holdoff task list until we're executing tasks which saves some data manipulation, at the cost of some data structures not being correct at all times. This saves significant amounts of time in various profile charts of larger builds. (Bitbake rev: 270f076111b12eab358417b0c4cf9c70d7cc787a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-21bitbake: runqueue: Small but critical fixRichard Purdie
We've observed do_package and do_package_setscene running in parallel. The reason is that holdoff_tasks wasn't getting updated. Looking at the code, it would seem the reason is that the task was in pending_migrations and hence changed wasn't set and holdoff_tasks wasn't updated. Fix this. It only affects builds with rehashing enabled. (Bitbake rev: e26e61e84575669bd223f6ab316798097ed95ec8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-21bitbake: cookerdata: Delay the setup of the siggen slightly to allow ↵Richard Purdie
metadata defined siggens If we define a metadata siggen it can fail due to the early init here. Move slightly later to avoid those failures which allows fixes in OE to the check-layer script related to the hash equiv siggen. (Bitbake rev: fdf5c341f3393173876a753c46c9bd067eb2b353) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-16bitbake: runqueue: Ensure target_tids is filteredRichard Purdie
bitbake <target> --runonly=fetch failed as the target_tids list included entries which were no longer targeted task ids. Fix this. (Bitbake rev: 94e848ae6544e628a19cb97115279b0b1678967c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-16bitbake: bitbake: server/process: Handle BBHandledException to avoid ↵Robert Yang
unexpected exceptions The parseBaseConfiguration() raises bb.BBHandledException(), but BitBakeServer() didn't handle it, so we always got unexpected exceptions when there were errors. For example: === Case 1: * Add "print "hello"' in base.bbclass' def oe_import() function def oe_import(d): print "hello" [snip] $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Last 60 lines of server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): File "/buildarea1/lyang1/poky/meta/classes/base.bbclass", line 21 print "hello" ^ SyntaxError: Missing parentheses in call to 'print' <The first exception> During handling of the above exception, another exception occurred: <Tracebacks> <The second exception> During handling of the above exception, another exception occurred: <Tracebacks> <The third exception> During handling of the above exception, another exception occurred: <Tracebacks> [snip] Now it looks like: $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): ERROR: Error in compiling python function in /buildarea1/lyang1/poky/meta/classes/base.bbclass, line 21: The code lines resulting in this error were: 0001:def oe_import(d): *** 0002: print "hello" 0003: import sys 0004: 0005: bbpath = d.getVar("BBPATH").split(":") 0006: sys.path[0:0] = [os.path.join(dir, "lib") for dir in bbpath] SyntaxError: Missing parentheses in call to 'print' (base.bbclass, line 21) === Case 2: * Add 'HOSTTOOLS += "hello"' to conf/local.conf: $ bitbake -p ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): <Tracebacks> [snip] During handling of the above exception, another exception occurred: [snip] <Tracebacks> ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed: hello The error message is printed by bb.fatal() which raises bb.BBHandledException(), but BitBakeServer() doesn't handle it, so we got it. Now it looks like: ERROR: Unable to start bitbake server (None) ERROR: Server log for this session (/buildarea1/lyang1/test_hy/bitbake-cookerdaemon.log): ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed: hello No unexpected exceptions anymore. [YOCTO #13267] (Bitbake rev: 6e6865e6371dbd31a136eae64cc5b1fa5f5bee33) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-15bitbake: runqueue: Ensure data is handled correctlyRichard Purdie
This doesn't appear to have ill effects right now but there is a correctness issue which this so fix it. (Bitbake rev: a5e084a266f63c2fd370122327615e49beaeb94e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-15bitbake: runqueue: Fix data corruption problemRichard Purdie
This was overwriting data in the parent which caused all kinds of odd/weird failures. (Bitbake rev: 4c5aeb424247a9d0c907524ffacd9c61fcdc0852) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-15bitbake: tests/runqueue: Fix testsRichard Purdie
There were paths being accidentally included in some of the hashserv tests. Remove that and update the hashes so the tests work independently of paths. (Bitbake rev: 6ddb9f09cb60c2354fa6a67cce412c4dc1e7dc2d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-06-02 00:42:48 +0000