aboutsummaryrefslogtreecommitdiffstats
path: root/lib
AgeCommit message (Collapse)Author
2012-08-02cooker: fix OVERRIDES in BB_SIGNATURE_HANDLER (to override value from ↵Martin Jansa
DISTRO.conf from local.conf) We really need to pass the finalised data store into the parser init function (and hence the siggen init function). This ensures any value changes get passed into the correct code. Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02fetch2/local: Add search paths to the debug log to improve log usefullnessRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02fetch2/local.py: Provide better debug output when fetch of a local file failsRichard Purdie
When a fetch failure occurs for a local file, this patch ensures we print the locations searched making it easier for the user to debug the problem. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02runqueue.py: Clean up runqueue exception catching in the normal task failure ↵Richard Purdie
case Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02Hob: change the Build failure scenario as ui designLiming An
As ui design 'build-fail-spec.pdf' to change the GUI [YOCTO #2183] Signed-off-by: Liming An <limingx.l.an@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02Hob: log for Hob and allow users to show logs after successful buildLiming An
If users build images in Hob, record logs and allow users to retrieve the lo after successful build. The logs are generated if and only if: - users do "just bake" - users do "build image" after "build packages" - users do "build packages" only [YOCTO #1991] Signed-off-by: Shane Wang <shane.wang@intel.com> Signed-off-by: Liming An <limingx.l.an@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02fetch2: print checksums when they are different then expectedMartin Jansa
* in form which can be copied into a recipe without modification * like oe-classic did since: http://git.openembedded.org/openembedded/commit/?id=68abc465559a68e9201c9525be3c4acc6297eaed * it shows them in right form when they are missing completely, but in more verbose form when different * it needs to print that only when checksums were requested, e.g. fetching from sstate mirror sets both md5mismatch and sha256mismatch, but your checksums shouldn't be shown Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02fetch2: handle broken symlinks in local mirror handlingChristopher Larson
If a file:// mirror is being used, the fetcher will create a symlink to the local file. However, if the local file gets removed, that link will be dead, and os.path.exists() returns False in that case, so it tries and fails to recreate the link. Now we unlink such a dead link if it exists. Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02knotty: ensure that directory for BB_CONSOLELOG exists before using itMartin Jansa
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02fetch2/__init__.py: Add NoChecksumError exceptionMark Hatle
Without the new exception, when the system is configured to use premirrors, but not allow network access (via BB_NO_NETWORK), when a recipe was lacking a checksum the wrong error message(s) were being generated. Instead of complaining about trying to perform network access, if the system was able to find the item in the premirror, it should inform the user of the lack of checksums, and the two SRC_URI fields they should use to update their recipe. Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02hob: deal event DiskFullKang Kai
Part of [Yocto #2168] When bitbake runqueue is teminated by disk monitor, it will send event DiskFull. Update to handle it. Signed-off-by: Kang Kai <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-08-02monitordisk: fire event DISKFULL when terminate buildKang Kai
Part of [Yocto #2168] Add a event DiskFull to descript the termination by disk monitor. Update check() to fire the event DiskFull when terminates the build. This could help UIs to deal this scenario and show more information to end user. Signed-off-by: Kang Kai <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26fetch/local: Allow preservation of path components in relative file:// urlsRichard Purdie
This enhances the fetcher to allow preservation of the path component in urls like: file://xxx/yyy/somefile.patch. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26runqueue.py: Wipe out the stamp cache between setscene and main task executionRichard Purdie
The stamp files can change during setscene and the cache should be cleared to account for this. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26runqueue.py: Allow the setsceneverify function to have a list of tasks that ↵Richard Purdie
are invalid and need to run There was some odd behaviour if some task was run from setcene whilst there were existing valid stamps for a depepdency. For example, do_populate_sysroot might be installed at setscene time but if there were other tasks not installed from setscene such as do_populate_lic which depend on do_configure, the setsceneverify function would think that do_configure needed to be rerun and would hence void the do_populate_sysroot and force that to rerun too. The setsceneverify function needs to know which tasks are going to be rerun, not just what the overall task list is and what setscene functions have run. This patch adds that information and maintains backwards compatibility in a slightly ugly but effective way. The metadata needs updating to take advantage of this change. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26runqueue.py: Ensure one setscene function doesn't mask out another which ↵Richard Purdie
needs to run The scenequeue code could result in one setscene function masking out another which had been marked as notcovered. This change ensures the notcovered list is taken into account when deciding which tasks need to be run, ensuring that tasks that should be run do get run. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26runqueue.py: Optimise the hashvalidate call to only process tasks we may ↵Richard Purdie
actually want Currently we call the hashvalidate for anything without a valid setscene stamp. This improves the code to account for existing stamp files so that we only process setscene hashes for things we might actually end up using. This avoids hash processing and makes things slightly more efficient and the logs less confusing. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26runqueue.py: Gracefully exit if an exception occurs in the runqueue ↵Richard Purdie
execution code There was a bug where an exception in the runqueue code would cause an infinite loop of debug messages. The exception would get reported but would loop since runqueue was still registered as an idle handler. This patch adds an exception handler to ensure in the case of errors, the system more gracefully shuts down and doesn't loop. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-26runqueue.py: Fix a stamp comparision bugRichard Purdie
We check the stamp cache before comparing t2 and t3 which means that we can miss a level in the stamp file chains. The result of this is that a stamp can be accepted as valid when in fact it isn't. Some weird behaviour alerted me to this in a local build. This patch also fixes to only uses the cache in recurse mode, there was a corner case where stamps not in recurse mode could get added to the cache which could cause an issue potentially. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-24bitbake wget fetcher: add parameter: downloadfilenameNitin A Kamble
this allows wget fetcher to store the downloaded file in a specified custom filename in ${DL_DIR} Exmaple: SRC_URI = "https://edc.intel.com/Download.aspx?id=6190;downloadfilename=LIN_IEMGD_1_14_GOLD_2443.tgz" This fixes bug: [YOCTO #2570] Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-24Change bzr fetcher to use branch instead of co. Fixes: bzr: ERROR: No pull ↵Martin Ertsaas
location known or specified. This problem occurs when fetching a different revision of the same source. Which mean every time you update a bzr package. Using branch sets the pull location, and are the preferred way of cloning/branching a repository in bzr. Signed-off-by: Martin Ertsaas <mertsas@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-24cooker.py: Ensure we track parser errors and only show bbappend issues on ↵Richard Purdie
complete parse Currently if any parsing failure occurs, there can be a long list of bbappends that are "dangling" based on the fact that recipes were not parsed. This change firstly ensures the error counter is incremented and secondly that the bbappends list is only shown on a completed parse list. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-19Update version to 1.15.3Richard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-19bitbake: Warn upon finding tab indentation in python functionsRichard Purdie
Mixed spacing in python functions can cause subtle issues and generally confuses users. We've standardised on 4 space indentation, adding this warning helps ensure consistency and avoid bugs. It also makes _prepend and _append operations on python functions slightly less risky. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-18fetch2/svn.py: Use protocol parameter to get the protocolMartin Jansa
* it was send in v1 of proto -> protocol changes but then wasn't in V2 http://patchwork.openembedded.org/patch/31617/ where warning about proto= was moved to shared __init__ Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-17hob2: add elf to hcc supported image typesKang Kai
[Yocto #2709] OE Core Commit c9b01af84b64edba056f959c349895e0698324e6 add elf image type. This will cause hob build image failed. Add elf to hcc supoorted image types to fix this issue. Signed-off-by: Kang Kai <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-17siggen.py: Insure .siginfo files writes into shared sstate cache are atomicJeffrey C Honig
Use tempfile.mkstemp to create a temporary file in the sstate dir and move it into place after closing. The previous code would fail in the chmod() if two users were running jobs that touched the same signature file. Signed-off-by: Jeffrey C Honig <jeffrey.honig@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-17runqueue.py: Improve error output to be more useful when non-existent tasks ↵Richard Purdie
are found Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-11fetch2/__init__.py: Warn user if SRC_URI is using "proto" and not "protocol"Andrei Gherzan
As well, if "proto" is used, get the associated value as "protocol" Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-11fetch2/bzr.py: Use "protocol" parameter to get the protocolAndrei Gherzan
Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-11fetch2/hg.py: Use "protocol" parameter to get the protocolAndrei Gherzan
Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-11fetch2/osc.py: Use "protocol" parameter to get the protocolAndrei Gherzan
Signed-off-by: Andrei Gherzan <andrei@gherzan.ro> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-11bitbake: Abort build if runtime dependency conflictWenzong Fan
Currently if there are multiple preferred providers available for a runtime dependency, bitbake will print an Error message and let the build go on. Anyways the build should abort while any Errors occured. [YOCTO #2734] Signed-off-by: Wenzong Fan <wenzong.fan@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-10fetch2/wget: Use FETCHCMD_wget to provide the commandline and optionsRichard Purdie
This also changes to use -t 2 -T 30 as the defaults which are more sane for a modern fetcher and already specified in OpenEmbedded metadata. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-10fetch2/cvs.py: Switch to use FETCHCMD_cvs and handle parameters in the ↵Richard Purdie
fetcher itself This brings the cvs fetcher more into line with the others and allows consistent usage of the FETCHCMD variable and option handling. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-09bitbake/cooker: Print which pkgs would be built in -g outputRobert Yang
This is for giving the user a clear list to show which pkg would be built, we have the "bitbake -g", but it is not easy to read for people, it is for "dot". Improve the "bitbake -g" to also save a pn-buildlist: $ bitbake -g core-image-sato ... NOTE: PN build list saved to 'pn-buildlist' [snip] The contents of pn-buildlist: busybox shadow-native pth sysfsutils qemu-helper-native curl-native ncurses-native gdbm xserver-xorg linux-libc-headers [snip] [YOCTO #2404] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-05data_smart: Fix multiple override interaction with append and prepend operatorsRichard Purdie
Variables which used multiple overrides and the append/prepend operators were not functioning correctly. This change fixes that. This fixes the testcase: OVERRIDES = "linux:x86" TESTVAR = "original" TESTVAR_append_x86 = " x86" TESTVAR_append_x86_linux = " x86+linux" TESTVAR_append_linux_x86 = " linux+x86" [YOCTO #2672] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-04runqueue.py: Fix recursive task pruning to only prune self referencing tasksRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-04fetch2/svn: Enhance to cope with subversion 1.7 upgradeRichard Purdie
svn changed working checkout formats between 1.6 and 1.7. Its convoluted to detect what format a given working copy is in so the simplest solution is simply to run "svn upgrade" within the working copy. The base svn command variable is relocated slightly to enable this new code to work effectively. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-04runqueue.py: Allow recrdeptasks that have self referencesRichard Purdie
In some cases we want to pull in DEPENDS and RDEPENDS of recrdeptask dependencies but we need a way to trigger or avoid this behaviour depending on context. The logical syntax to trigger such behaviour would be a self referencing recrdeptask: do_a[recrdeptask] = "do_a do_b" The dependency chains already recurse this kind of expression correctly, the missing piece is to avoid any circular reference errors. Since the dependencies have already been recursively resolved, simply removing any recrdeptask references is enough to break the circular references. This patch therefore removes any circular references using the set difference_update() operator. There will be metadata tweaks required to add any references needed to the extra taskname. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-04Hob: fixed some variables not clean bug in detail pageLiming An
[YOCTO #2679] Signed-off-by: Liming An <limingx.l.an@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-07-04git: do not execute 'git remote prune'Enrico Scholz
'git remote prune' at this location does not make much sense because the following 'git remote rm' will prune stale and non-stale branches. The 'prune' can cause trouble because it will access the network bypassing the no-network code in bitbake. When this operation fails and throws an exception, the next command (--> 'git remote rm') will be skipped. This in turn, will make all the following operations fail, because they assume that the remote does not exist yet. Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-29Hob: change 'run image' and 'deploy' Gui and work flow for building detail pageLiming An
In build detail page, the response action of clicked the 'run image' or 'deploy' button will accroding to below as ui design: 1)if there has one file in building result, it will responsed the 'run image'(now, we only support the qemu) or 'deploy' directly 2)if there has more than one file, it will popup a dialog with listed created files type, they are has same action attributes 'deploy' or 'runnable'. Note: because the qemu image (runnable file) can't be deployed and we can't generated a image that has the two attributes now, can be run or can be deployed, so the code will not deal with this case. [YOCTO #2155] Signed-off-by: Liming An <limingx.l.an@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-28runqueue.py: Handle multiple rdeptask entriesRichard Purdie
I'm not sure why we don't currently allow multiple entries in rdeptask when we do in deptask. This makes the handling match between the two since its trivial to fix. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-28fetch2: Revert the regexp removal for the type field and instead anchor regexpRichard Purdie
People are using regexps in the url type field so we need to preserve this bitbake behaviour. To address the issues with https:// urls mapping badly to file:// urls we anchor the regexp if its not already anchored. There should be no expressions in the wild which would break with this change. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-28test/fetch: Switch the comparision order to make test failures slightly clearerRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-28test/fetch: Ensure cwd is valud for the git cloneRichard Purdie
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-28runqueue: Reimplement recrdepends so it works more correctlyRichard Purdie
Currently, recrdepends is extremely greedy. For example: do_foo[rdepends] = "somedep:sometask" addtask foo which adds foo with *no* dependencies, will suddenly start appearing as a dependency in every task which uses recrdepends. So far this has been mildy annoying but we now have use cases where this makes no sense at all. This reworks the recrdepends code to avoid this problem. To do this we can no longer collapse things into lists just based on file ID. The problem is this code is extremely performance sensitive. The "preparing runqueue" phase spends a lot of time in these recursive dependency calculations so any change here could negatively impact the user experience. As such, this code has been carefully tested on convoluted dependency trees with operations like "time bitbake world -g". The net result of this change and the preceeding changes combined is a net speed up of these operations in all cases measured. Tests were made comparing "bitbake world -g" task-depends.dot before and after this patch. There *are* differences for example -nativesdk do_build dependencies on -native recipes are no longer present. All removed dependencies appear to be sensible improvements to the system. The "rdepends" cross contamination issue above is also fixed. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-28runqueue.py: Convert depends variable to use setsRichard Purdie
This gives some small performance gains and sets the scene for other improvements by removing the need for duplicate detection code. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-28taskdata: Add gettask_id_fromfnid helper functionRichard Purdie
This is like gettask_id but doesn't require translation of fnid -> fn first which the function then translates back. This gives a sizeable performance improvement since a significant number of lookups are avoided. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>