summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/ui/knotty.py
AgeCommit message (Collapse)Author
2019-08-08bitbake: knotty: Fix for the Second Keyboard InterruptRobert Yang
Fixed: $ rm -fr tmp-glibc/cache/default-glibc/qemux86/x86_64/bb_cache.dat* ; bitbake -p Press the first Ctrl-C when the parsing process is at about 50%: Keyboard Interrupt, closing down... Then presss the second Ctrl-C: File "/path/to/bitbake/bitbake/lib/bb/ui/knotty.py", line 619, in main event = eventHandler.waitEvent(0.25) File "/path/to/bitbake/lib/bb/server/process.py", line 591, in waitEvent self.eventQueueNotify.wait(delay) File "/usr/lib/python3.5/threading.py", line 549, in wait signaled = self._cond.wait(timeout) File "/usr/lib/python3.5/threading.py", line 297, in wait gotit = waiter.acquire(True, timeout) KeyboardInterrupt Capture the second KeyboardInterrupt during stateShutdown is running can fix the problem. There may be still tracebacks for the third KeyboardInterrupt, but I'm leaning to not fix it since we aimed for supporting 2 KeyboardInterrupts only. (Bitbake rev: 8c26b451f22193ef1c544e2017cc84515566c1b8) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-15bitbake: event/runqueue: Drop StampUpdate event, its pointless/unusedRichard Purdie
Whilst this class has existed for years, it doesn't have any users and has a questionable interface. Drop it to allow for further simplification and changes. (Bitbake rev: 3ab51764f7965d696bb2c5a872bf161473df4289) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-11bitbake: knotty: allow progress rate for indeterminate barsChris Laplante
(Bitbake rev: 85f0b443b7ab1848abc6eb658be489fc1718004c) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04bitbake: bitbake: Drop duplicate license boilerplace textRichard Purdie
With the introduction of SPDX-License-Identifier headers, we don't need a ton of header boilerplate in every file. Simplify the files and rely on the top level for the full licence text. (Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-04bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie
This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-01bitbake: knotty: Implement console 'keepalive' outputRichard Purdie
CI systems like jenkins and buildbot will timeout applications which haven't had console output in some period of time. Add 'keepalive' output to knotty which gives output every 5000s if not other output was made and tasks are still running. This reduces some problems encountered with our CI testing. (Bitbake rev: aa4f31e5741dd98acec73f16f6028e52f4c22d6f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-30bitbake: knotty: Pretty print task elapsed timeJacob Kroon
A task's runtime is currently printed in seconds. Change it to include minutes and hours for easier reading. (Bitbake rev: c593ae5ec9fecd4bde823948024e4d56314a60ce) Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31bitbake: tinfoil: ensure log lines get printed when tasks failPaul Eggleton
If a task fails during build_targets(), we need to print out the log lines as knotty does or the user will be missing information about the failure. (This should get some deeper refactoring, but now isn't the time for that.) (Bitbake rev: 24879df071d4803db3d39ae1d5cad852daa92f28) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-08-31bitbake: ui/knotty: Send updateConfig earlyRichard Purdie
If for example you run: bitbake -r somefile.inc rm somefile.inc bitbake -e bitbake will crash with an error about not being able to find somefile.inc. This is because it tries to reparse the base config for the early getVariable requests before it sees the updated missing -r option. Send the updateConfig command earlier to avoid this. (Bitbake rev: a38164620ebdc770690c5f39ff9ed69d3f82719e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: knotty: make it possible to use termfilter without either consolePaul Eggleton
This isn't useful for knotty itself, but for use from tinfoil in case we can't get access to either the console or errconsole, allow either to be unspecified (None). (Bitbake rev: 7544de437fc66b81502ecdb5db859182c45827cb) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-21bitbake: lib/bb/event: refactor printing eventsPaul Eggleton
We really ought to have just one place where the string representation of these events is produced. This doesn't take any real control away from the UI - if an alternative representation is desired, that can still be made. (Bitbake rev: cb15db2a799be6d8eab9a2a43a9a573f89229cff) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-18bitbake: knotty: fix indentationPaul Eggleton
(Bitbake rev: 06a96d267dec5b542725defb8f01403f505c4f44) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-08bitbake: knotty: Drop task prefix of PLAIN log messagesRichard Purdie
To quote Paul: Not that long ago we added a prefix to logged messages to allow us to see where the message has been generated (recipe / task). This is undoubtedly useful for errors and warnings, however, I'm not sure it's really appropriate for bb.plain(). As an example, see the output for -c listtasks now: ... NOTE: Executing RunQueue Tasks nodejs-native-4.5.0-r0 do_listtasks: do_addto_recipe_sysroot nodejs-native-4.5.0-r0 do_listtasks: do_build Default task for a recipe - depends on all other normal tasks required to 'build' a recipe nodejs-native-4.5.0-r0 do_listtasks: do_checklicense nodejs-native-4.5.0-r0 do_listtasks: do_checklicenseall nodejs-native-4.5.0-r0 do_listtasks: do_checkpkg ... This patch excludes PLAIN messages from this prefixing making the log output neater. [YOCTO #11457] (Bitbake rev: 4a14b44b3e4fad3a3e5e53461aa8ba9929a515b8) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-07-08bitbake: bitbake: Add MultiConfigParsed eventRichard Purdie
There are some cases where the metadata needs to be aware a multiconfig build is happening and have access to the multiconfig data stores to merge data into the common build. This adds such an event allowing access to these datastores. (Bitbake rev: 160e47f5df90850e64dcb857c81a5039abc9235f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12bitbake: knotty: prevent extra logger from being enabled for tinfoilPaul Eggleton
tinfoil sets up its own logger by default, but if and when we initialise the UI (by default knotty) will also set one up, leading to duplicated messages specifically from tasks. To avoid this, rather than adding some kind of parameter, just check if there is already a logger outputting to stdout/stderr and if so, skip adding our own. Part of the fix for [YOCTO #11275]. (Bitbake rev: 66d866745f35468d1540a793d07e3a401298b84b) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-13bitbake: knotty: Remove extra whitespace after task progress barsPeter Kjellerstedt
For some reason, BBProgress.setextra() always adds a space at the end. This prevents the task progress bars from filling the entire width of the terminal, making them stop one character short. This looks odd when shown together with the main progress bar, which does fill the entire terminal width. (Bitbake rev: 6540e98090d6a88607489a23c063be338bcc3b57) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14bitbake: knotty: fix --observe-only optionPaul Eggleton
If we're in observe-only mode then we cannot run commands that would affect the server's state, including getSetVariable, so prevent that from being called in observe-only mode. (Bitbake rev: 2c5a8661430edebff67ab4a108995033d182b5d6) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-14bitbake: knotty: make quiet option a level optionPaul Eggleton
Allow you to specify -q / --quiet more than once to reduce the messages even further. It will now operate as follows: Level Option Result ----- ------ ---------------------------------------- 0 Print usual output 1 -q Only show progress and warnings or above 2 -qq Only show warnings or above 3+ -qqq Only show errors (Bitbake rev: 6cf2582e17c28ca04f5cfb59858c4a9778c700d4) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-07bitbake: cooker process: fire heartbeat event at regular time intervalsPatrick Ohly
The intended usage is for recording current system statistics from /proc in buildstats.bbclass during a build and for improving the BB_DISKMON_DIRS implementation. All other existing hooks are less suitable because they trigger at unpredictable rates: too often can be handled by doing rate-limiting in the event handler, but not often enough (for example, when there is only one long-running task) cannot because the handler does not get called at all. The implementation of the new heartbeat event hooks into the cooker process event queue. The process already wakes up every 0.1s, which is often enough for the intentionally coarse 1s delay between heartbeats. That value was chosen to keep the overhead low while still being frequent enough for the intended usage. If necessary, BB_HEARTBEAT_EVENT can be set to a float specifying the delay in seconds between these heartbeat events. (Bitbake rev: 7cf22ea057d28c54bd98dc1ab7a43402a29ff1f5) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-09bitbake: ui/knotty.py: Fix signal handling of SIGWINCH in BBProgressAníbal Limón
Add the ability to pass default signal handler for SIGWINCH in BBProgress because with multiple instace of BBProgress the original signal handler set by TerminalFilter (sigwinch_handle) is lost. This is a fix for stack trace due to multiple async calls of ProgressBar _handle_resize (ioctl to terminal fd), see: NOTE: Executing SetScene Tasks Fatal Python error: Cannot recover from stack overflow. Current thread 0x00007f70a4793700 (most recent call first): File "/home/alimonb/repos/poky/bitbake/lib/progressbar/progressbar.py", line 183 in _handle_resize File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 58 in _handle_resize File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 60 in _handle_resize ... File "/home/alimonb/repos/poky/bitbake/lib/bb/ui/knotty.py", line 60 in _handle_resize ... Aborted (Bitbake rev: 812bd49cb569379ee90d5be28a4b6e60645f1e54) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23bitbake: knotty: ensure progress bar output is accounted for in displayPaul Eggleton
When calculating how many lines we'd printed we weren't properly taking the progress bars into account, with the result that sometimes if the last line printed on the terminal wrapped to the next line (which is possible) we backed up less lines than we should have. Additionally, we should always print a newline after updating the progress bar - there's no need to check if there wasn't output (there always will be courtesy of our overridden _need_update()) and we now allow the line to wrap so we don't need to check the other condition either. Hopefully this will fix [YOCTO #10046]. (Bitbake rev: 326d18d96faf02675ba34ad3c3a20cd424b39b91) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-23bitbake: knotty: Show task elapsed timeRichard Purdie
Its often useful to know how long a task has been running for. This patch adds that information to the task display, updating every 5s if there were no other updates so the user can see how long tasks have been running for. [YOCTO #9737] (Bitbake rev: 6c42025e5dd7761213be3f82f3252a7892d2239d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-20bitbake: toasterui, knotty: don't print taskid followed by taskstring which ↵Martin Jansa
are now in most cases identical * unify the format how the task is described * don't show taskid followed by taskstring as the taskstring is different only for setscene tasks (by _setscene suffix) * the duplicated output was introduced by: 2c88afb taskdata/runqueue: Rewrite without use of ID indirection as reported and confirmed as a bug here: http://lists.openembedded.org/pipermail/openembedded-core/2016-June/123148.html * show: NOTE: Running task 541 of 548 (/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package) instead of much longer: NOTE: Running task 541 of 548 (ID: /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package, /OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_package) and similarly for failed tasks: ERROR: Task (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1' instead of much longer: ERROR: Task virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install (virtual:native:/OE/build/oe-core/openembedded-core/meta/recipes-core/zlib/zlib_1.2.8.bb:do_install) failed with exit code '1' (Bitbake rev: 696693d45f5eff1226866ed79dbfb67161d8cd3f) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17bitbake: knotty: don't show number of running tasks in quiet modePaul Eggleton
There's not a whole lot of point showing how many tasks are running when we're in quiet mode, it just looks a bit strange particularly when it's not running any tasks. (Bitbake rev: 5317200d9cd73c6f971bc1b0cfe8692749e27e3a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-17bitbake: knotty: fix task progress bar not starting at 0%Paul Eggleton
If we have the task number here we need to subtract 1 to get the number of tasks completed. (Bitbake rev: 7c78a1cd3f0638ae76f7c7a469b7f667c7c58090) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: knotty: don't display ETA for tasks with progressPaul Eggleton
It turns out that progress information we can extract from a task is rarely apportioned closely enough to the time taken for the ETA to be accurate, so showing it is going to be misleading most of the time for anything but the most basic of examples. Let's just remove it and avoid misleading (or worse, annoying) the user. Fixes [YOCTO #9986]. (Bitbake rev: 235db4870b11db97250979e647b54cdb5ce4fbb6) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-26bitbake: knotty: fix some minor bugs in BBProgressPaul Eggleton
If you specify custom widgets then we don't want to assume where the "extra" position is - you should have to specify it, and if it isn't specified it shouldn't just wipe out the last widget or you can start to see odd behaviour if you're modifying the code. (Bitbake rev: 19e33c10feb1637589ceb05b5e8d58b1e012ccb8) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08bitbake: knotty: Handle process indicators more gracefullyRichard Purdie
Mistakes can happen with the generation of the progress events, change knotty to be more tolerant of this rather than crashing, reporting to the user when something unexpected happens. I haven't debugged why multiple finish events appear to be triggered. (Bitbake rev: 7dd06b1016b36420a9c55a45ff29dd64ae1dbcda) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08bitbake: knotty: add quiet output modePaul Eggleton
Quiet output mode disables printing most messages (below warnings) to the console; however these messages still go to the console log file. This is primarily for cases where bitbake is being launched interactively from some other process, but where full console output is not needed. Because of the need to keep logging all normal events to the console log, this functionality was implemented within the knotty UI rather than in bb.msg (where verbose mode is implemented). We don't currently have a means of registering command line options from the UI end, thus the option actually has to be registered in main.py regardless of the UI, however I didn't feel like it was worth setting up such a mechanism just for this option. (Bitbake rev: db95cdef08e339dec7462bfde3ad7d75c1c60dd8) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08bitbake: knotty: show task progress barPaul Eggleton
In addition to the "currently running n tasks (x of y)" message, show a progress bar for another view on how much of the build is left. We have to take care to reset it when moving from the scenequeue to the runqueue, and explicitly don't include an ETA since not all tasks take equal time and thus it isn't possible to estimate the time remaining with the information available. (Bitbake rev: de682015a3fefeff36ddc4197641a700f3fb558d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08bitbake: knotty: add code to support showing progress for sstate object queryingPaul Eggleton
Add support code on the BitBake side to allow sstate.bbclass in OpenEmbedded to report progress when it is checking for availability of artifacts from shared state mirrors. Part of the implementation for [YOCTO #5853]. (Bitbake rev: 070ae856da0715dbaf4c560c837ea796ffc29f00) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08bitbake: lib: implement basic task progress supportPaul Eggleton
For long-running tasks where we have some output from the task that gives us some idea of the progress of the task (such as a percentage complete), provide the means to scrape the output for that progress information and show it to the user in the default knotty terminal output in the form of a progress bar. This is implemented using a new TaskProgress event as well as some code we can insert to do output scanning/filtering. Any task can fire TaskProgress events; however, if you have a shell task whose output you wish to scan for progress information, you just need to set the "progress" varflag on the task. This can be set to: * "percent" to just look for a number followed by a % sign * "percent:<regex>" to specify your own regex matching a percentage value (must have a single group which matches the percentage number) * "outof:<regex>" to look for the specified regex matching x out of y items completed (must have two groups - first group needs to be x, second y). We can potentially extend this in future but this should be a good start. Part of the implementation for [YOCTO #5383]. (Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08bitbake: knotty: import latest python-progressbarPaul Eggleton
Since we're going to make some minor extensions to it, it makes sense to bring in the latest version of python-progressbar. Its structure has changed a little but the API hasn't; however we do need to ensure our overridden _needs_update() function's signature in BBProgress() matches properly. (Bitbake rev: c3e51d71b36cbc9e9ed1b35fb93d0978e24bc98a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08bitbake: knotty: provide a symlink to the latest console logPaul Eggleton
If you're looking to find the latest console log repeatedly it can be a bit tedious - let's just create a symlink just as we do with other logs to make it easy to find. (Bitbake rev: e9f41c0507a6527bf2ed86506813d4d4a89f8ebf) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02bitbake: bitbake: Convert to python 3Richard Purdie
Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-17bitbake: knotty: Fix output buffering issuesRichard Purdie
We need to flush the footer removal, else it may not be outputted until the buffer is flushed as part of StreamHandler and this would lead to it removing the ERROR output just printed which is extremely confusing. Also ensure the footer is cleared before printing a summary as in some cases it wasn't being removed, also leading to user confusion. (Bitbake rev: 0e030c4d074c41859608dab5f3ad26b05f56b306) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-13bitbake: knotty: Ensure consolelog file handle is closedRichard Purdie
If we don't close the console log file handle, python prints a warning about unclosed file handles upon exit which is annoying. (Bitbake rev: 624dd92952b2fc736fd86abe5f2390b87b3a7dd3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11bitbake: bitbake: Update logger.warn() -> logger.warning()Richard Purdie
python deprecated logger.warn() in favour of logger.warning(). This is only used in bitbake code so we may as well just translate everything to avoid warnings under python 3. Its safe for python 2.7. (Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15bitbake: providers: Add PREFERRED_RPROVIDER supportRichard Purdie
Sometimes you can end up in a situation where you need to specify that a specific runtime entity should be provided by a specific entry. An example of this is bluez where you could end up in a situation where for example: NOTE: multiple providers are available for runtime libasound-module-bluez (bluez4, bluez5) NOTE: consider defining a PREFERRED_PROVIDER entry to match libasound-module-bluez NOTE: multiple providers are available for runtime bluez-hcidump (bluez-hcidump, bluez5) NOTE: consider defining a PREFERRED_PROVIDER entry to match bluez-hcidump The only option here is to set something like PREFERRED_PROVIDER_bluez4 = "bluez4" which is clearly not very informative. I've actually held off adding RPROVIDER support for a long while as this does have sigificant potential for misuse. It doesn't for example allow multiple runtime providers of the same name to coexist, that simply isn't supported. It therefore doesn't replace some of the name mappings such as busybox verses coreutils that OE-Core faces as that is a different problem with different constraints. This mechanism is simply to provide bitbake with a hint to decide what the dependency tree should look like. Also, this allows us to stop printing a confusing message telling the user to set PREFERRED_PROVIDER when the setting needed would be rather ambiguous. [YOCTO #5044] (Bitbake rev: 62eb39d1474d024b204634689071700605c6095c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15bitbake: cooker/knotty: Prefix parse logs with filename being parsedRichard Purdie
We now prefix log messages coming from worker task context with the PF and task info, however parsing messages all have to be manually prefixed which is ugly and error prone. This change modifies the log handler filter so this happens automatically, meaning we don't have to change every message to include that information. This makes error messages longer but more usable. (Bitbake rev: 1af0ccaac81e182c4ca520037dda362d180e5605) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16bitbake: bb.ui.knotty: prefix task messages with recipe/taskChristopher Larson
An example prefix: `perl-5.22.1-r0 do_compile:` (Bitbake rev: 792b759e59e31d2e43d525a6e50d866b4f51f072) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15bitbake: knotty: Set exit failure code on runQueueTaskFailed eventsRichard Purdie
If the worker segfaults, we may never see a TaskFailed event from it, only a runQueueTaskFailed event. In this case, return_value isn't getting set leading to an incorrect exit code from bitbake. Fix by setting return_value in both places. (Bitbake rev: e5dd50e0d95d532fe31dde61f8c6b1a7a72321e9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01bitbake: knotty: Don't show errors for universe provider issuesRichard Purdie
When running universe builds, we don't expect an error exit code for provider warnings. Change the error messages to warnings in this case. This deals with errors causing problems on our autobuilders amongst other issues. (Bitbake rev: d4989fb0355476de172169f0698757f7360e9a1f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30bitbake: bitbake: Set process names to be meaninfulRichard Purdie
This means that when you view the process tree, the processes have meaningful names, aiding debugging: $ pstree -p 30021 bash(30021)───KnottyUI(115579)───Cooker(115590)─┬─PRServ(115592)───{PRServ Handler}(115593) ├─Worker(115630)───bash:sleep(115631)───run.do_sleep.11(115633)───sleep(115634) └─{ProcessEQueue}(115591) $ pstree -p 30021 bash(30021)───KnottyUI(117319)───Cooker(117330)─┬─Cooker(117335) ├─PRServ(117332)───{PRServ Handler}(117333) ├─Parser-1:2(117336) └─{ProcessEQueue}(117331) Applies to parse threads, PR Server, cooker, the workers and execution threads, working within the 16 character limit as best we can. Needed to tweak the bitbake-worker magic values to tell the workers apart. (Bitbake rev: 539726a3b2202249a3f148d99e08909cb61902a5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-08bitbake: knotty: Use non-interactive mode as fallback for dumb terminalsRichard Purdie
TERM=dumb bitbake X shows no output for task status which is suboptimal. Use the non-interactive mode if the terminal doesn't support what we need for interactive mode giving a better user experience. Also print a note to the console to say this has happened. [YOCTO #8768] (Bitbake rev: 6f84cf4bd77f35fcd07e0b2f5149f1d6866a414d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18bitbake: knotty: Improve exception error messageRichard Purdie
Instead of: """ can only concatenate tuple (not "int") to tuple """ we now see: """ Traceback (most recent call last): File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 324, in main termfilter.updateFooter() File "/media/build1/poky/bitbake/lib/bb/ui/knotty.py", line 210, in updateFooter lines = 1 + int(len(content) / (self.columns + 1)) TypeError: can only concatenate tuple (not "int") to tuple """ which makes tacking down and fixing the problem much easier. Also ensure we set an error exit code. (Bitbake rev: d965bcae6cfd268406a3bd1ef77c5bb6c6e1c6d7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-18bitbake: knotty: Fix row/column function return value issueRichard Purdie
When the row handling was introduced, one of the callbacks was missed resulting in: TypeError: can only concatenate tuple (not "int") to tuple Fix it. (Bitbake rev: 0b77cea2bf5b5f5704e2650fb0332f5d78037781) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-08bitbake: knotty: Enforce terminal line limit to stop crazy scrollingRichard Purdie
If there are more tasks running than there are lines on the terminal, the data scrolls in ways the UI wasn't designed for. This patch adjusts the UI just to show the processes which fit onto the number of rows in the terminal window. You can see the total number running from the counter in the top left as usual and this makes warning and errors messages scrolling from the top of the window work as designed. Ultimately, scrolling would be nice but is for another time, this fixes the biggest UI issue on highly parallel machines. (Bitbake rev: 67b77658e2bfa849f6f55c9c262cb11d6bfdb399) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-29bitbake: bb/ui: Use getSetVariable command for BB_CONSOLELOGRichard Purdie
Metadata can define BB_CONSOLELOG as containing ${DATETIME} and this can get expanded to a different value each time the variable is read. In the case of BB_CONSOLELOG, this behaviour is not desireable. The values of DATE/TIME are locked down at build time but this is too late for the purposes of ensuring the system can figure out the real value of BB_CONSOLELOG. The best way to do this is to set the variable into the datastore, thereby preserving its value. [YOCTO #8411] (Bitbake rev: 021f2eb55ab5863b57ed1b3f19f1b329bc1ad477) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01bitbake: cooker: properly fix bitbake.lock handlingRichard Purdie
If the PR server or indeed any other child process takes some time to exit (which it sometimes does when saving its database), it can end up holding bitbake.lock after the UI exits, which led to errors if you ran bitbake commands successively - we saw this when running the PR server oe-selftest tests in OE-Core. The recent attempt to fix this wasn't quite right and ended up breaking memory resident bitbake. This time we close the lock file when cooker shuts down (inside the UI process) instead of unlocking it, and this is done in the cooker code rather than the actual UI code so it doesn't matter which UI is in use. Additionally we report that we're waiting for the lock to be released, using lsof or fuser if available to list the processes with the lock open. The 'magic' in the locking is due to all spawned subprocesses of bitbake holding an open file descriptor to the bitbake.lock. It is automatically unlocked when all those fds close the file (as all the processes terminate). We close the UI copy of the lock explicitly, then close the server process copy, any remaining open copy is therefore some proess exiting. (The reproducer for the problem is to set PRSERV_HOST = "localhost:0" and add a call to time.sleep(20) after self.server_close() in lib/prserv/serv.py, then run "bitbake -p; bitbake -p" ). Cleanup work done by Paul Eggleton <paul.eggleton@linux.intel.com>. This reverts bitbake commit 69ecd15aece54753154950c55d7af42f85ad8606 and e97a9f1528d77503b5c93e48e3de9933fbb9f3cd. (Bitbake rev: a29780bd43f74b7326fe788dbd65177b86806fcf) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>