summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/process.py
AgeCommit message (Collapse)Author
2019-05-04bitbake: bitbake: Add initial pass of SPDX license headers to source codeRichard Purdie
This adds the SPDX-License-Identifier license headers to the majority of our source files to make it clearer exactly which license files are under. The bulk of the files are under GPL v2.0 with one found to be under V2.0 or later, some under MIT and some have dual license. There are some files which are potentially harder to classify where we've imported upstream code and those can be handled specifically in later commits. The COPYING file is replaced with LICENSE.X files which contain the full license texts. (Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14bitbake: bitbake:process: flush stderr/stdout to logJuro Bystricky
When a process terminates, some messages may still remain in stdout or stderr and do not make it into the log file. In addition, the messages that do make it to the log file may end up in the log file in incorrect order. This patch flushes all messages into the log file after the process terminates. Some additional log flushing is also needed to keep the various messages showing up in the log file in proper order. [YOCTO#10785] (Bitbake rev: 1f6e6aa8262369eafc3bbf9f01f8d981f90becdf) Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-01bitbake: process: stop bb.process.communicate mixing bytes and str return typesMike Crowe
Python3 regards b"" as False so it is not being converted to a string by d0f904d407f57998419bd9c305ce53e5eaa36b24. This confusingly causes three different potential types for each member of the returned tuple. Let's just assume that everything that's not None is a bytes object and convert it to a string. (Bitbake rev: 0cf5589b7fb3582a6caca5014c4d8152347df545) Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02bitbake: bitbake: Convert to python 3Richard Purdie
Various misc changes to convert bitbake to python3 which don't warrant separation into separate commits. (Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: Fix default function parameter assignment to a listPaul Eggleton
With python you should not assign a list as the default value of a function parameter - because a list is mutable, the result will be that the first time a value is passed it will actually modify the default. Reference: http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments (Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-16bitbake: lib/bb/process: check output of select() before reading extrafilesPaul Eggleton
We're calling select() to find the fds that have data available, so we really ought to check the return to see if the extra file is in there before trying to read from it. This is part of the fix for the performance regression that this code introduced (5-10 minutes extra in a reasonable size OE build); the rest is down to an issue in the way that pseudo currently handles FIFOs and will need to be addressed there, see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=7993 Solution suggested by Richard Purdie <richard.purdie@linuxfoundation.org> (Bitbake rev: 3e2db8d7eaa0f14813213d1c95d3ee9efd97dc9d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-12bitbake: lib/bb: set up infrastructure for shell message functionsPaul Eggleton
Create a fifo under ${T} for each running task that can be used by the task to send events back to BitBake. The purpose of this is to allow OpenEmbedded's logging.bbclass (which provides shell function equivalents for bb.warn(), bb.note() etc.) to have those functions trigger the appropriate events within BitBake so that messages are shown through the BitBake UI rather than just in the task log; to do that we just call the python functions. Part of the fix for [YOCTO #5275]. (Bitbake rev: bae330a1f8a59a912eca71177b3c6ba7c92a2f38) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-10bitbake: process: Add timeout to select callRichard Purdie
On some machines, a hang has been noticed where the system sits in the select call despite the task having completed. The exact reasons for this as unknown but adding a timeout unblocked the builds and resolved the hangs in question. (Bitbake rev: 5223ffb5b6a46d8b3f6ac3362bd2672e2edf2691) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09bitbake: lib: Use modern expcetion syntaxRichard Purdie
(Bitbake rev: a4a37b6a83faa62f61433122c4583e93e64f7372) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-05-09bitbake: lib: Clean up various file access syntaxRichard Purdie
Python 3 is stricter about how files are accessed. Specficially: * Use open(), not file() * Use binary mode for binary files (when checksumming) * Use with statements to ensure files get closed * Add missing file close statements (Bitbake rev: 9f08b901375ba640f47596f1bcf43f98a931550f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2012-06-25bitbake: process: Improve _logged_communicate bufferingRichard Purdie
There are two problems with the _logged_communicate that are both caused as a result of buffering I/O issues: 1) log truncation when python fails 2) While a bitbake task is running it is impossible to see what is going on if it is only writing a small incremental log that is smaller than the buffer, or you get only a partial log, up until the task exists. It is worse in the case that stderr and stdout are separate file handles, because previous code blocks on the read of stdout and then stderr, serially. The right approach is simply to use select() to determine if there is data available and also flush the log before exiting. This is based on a patch from Jason Wessel <jason.wessel@windriver.com> with some changes to flush upon exit, abstract the non blocking file descriptor setup and drop the buffer size parameter. (Bitbake rev: 361fb71e907aa84c28ecec79fefc6ca39c39172f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-15make exception handling syntax consistentScott Garman
Update exception handling syntax to use the modern style: except ExcType as localvar (Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d) Signed-off-by: Scott Garman <scott.a.garman@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-14process.py: Avoid deprecation warningRichard Purdie
bitbake/lib/bb/process.py:15: DeprecationWarning: BaseException.message has been deprecated as of Python 2.6 Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-06parse.ast: avoid code duplication for inheritChris Larson
(Bitbake rev: 0b11a3d4eab84b372fd45b9537cf0327008daf8d) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-05bitbake: Fix process.py to accept None as the log optionRichard Purdie
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04process: fix handling of the input argumentChris Larson
When using a logfile, we weren't sending input to the child process. (Bitbake rev: 5ec4ca7e45bdf6d259503fc67155395e89ba6329) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04process: handle OSErrors other than file not foundChris Larson
(Bitbake rev: 7d80a5355cb540aae8d3082c1efebb72da4f93c6) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04Fix PWD issue with new exec_func_shellChris Larson
The previous attempt was incorrect. The issue isn't that subprocess fails to set PWD, it's that PWD is in the metadata, inherited from the environment, and is re-exported, overwriting the actual accurate one in the shell environment with the old one from the metadata. So, ensure that PWD in the metadata is not exported. We can ditch this when the environment handling is reworked (e.g. poky's commit to do so). (Bitbake rev: 2c8683234acf514706b2b69f5b29405485e664dd) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04build: set PWD in the subprocess preexec hookChris Larson
(Bitbake rev: efa5485c8f9d488ba058c40734cc55296d3de6eb) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04process: add subprocess-based bitsChris Larson
(Bitbake rev: c63e55564a8840083dbd8634b10fe6f76d1f1354) Signed-off-by: Chris Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>