summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb
AgeCommit message (Collapse)Author
2015-09-18bitbake: tests/fetch.py: Fix recursion failure in url mappingMark Hatle
Instead of reproducessing the same line over and over and over, we remove the current line from the mirror list. This permits us to re-evaluate the list while excluding all matches that have previousily occured. Without this fix, adding this test results in a failure: RuntimeError: maximum recursion depth exceeded in cmp (Bitbake rev: 24a8e9a5b0ba145ae589178d74365c986ebca325) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18bitbake: fetch2/__init__.py: uri_replace regex handlingMark Hatle
We should only substitute one time. If we do it without a max count, we can end up matching over and over. Before this change: https://downloads.yoctoproject.org/releases/bitbake/bitbake-1.0.tar.gz with a mirror of https://.*/[^/]* http://AAAA/A/A/A/ would end up either recursing indefinitely or result in: http://AAAA/A/A/A/A/A/A/A/A/A/bitbake-1.0.tar.gz (Bitbake rev: 4d254e02e2867dd9a6663508c8ca9f2733af71a8) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18bitbake: toaster: Read correct cooker log path from toasteruiElliot Smith
The BB_CONSOLELOG variable changes by the time we read it in BuildInfoHelper. This means that the log file location we are using is incorrect, so the links to the cooker logs don't work. Instead, read it at the point when the BuildStarted event occurs in toasterui. The BB_CONSOLELOG variable has the correct value here, so pass that to BuildInfoHelper. [YOCTO #8209] (Bitbake rev: 20609eebee0d2318806cf81913e7ce6dc1005507) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18bitbake: data_smart: Separate out update_overridevars into separate functionRichard Purdie
No code changes, just refactoring to allow for functionality changes by moving things to a separate function. (Bitbake rev: 2eb934814179ccf42e3d424dabe26b17d013a7ed) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-18bitbake: cache: Handle spaces and colons in directory names for file-checksumsRichard Purdie
If there is a space in a directory name containing a file in file-checksums (e.g. from a file:// url), you currently get tracebacks from bitbake. This improves the code to handle colons and spaces in the file-checksums names since it possible to figure out the correct names. [YOCTO #8267] (Bitbake rev: 87282b283921a58426f24fb21151db457c5bca66) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09bitbake: utils: Add signal_on_parent_exit() functionRichard Purdie
Add a new bb.utils.signal_on_parent_exit() function so that a process can register to recieve a signal when the parent dies. There is no POSIX standard for this and the implementation is Linux specific. Alternatives would be having an open pipe or polling os.getppid() for changes but this seems more effective and less invasive to most of bitbake's code structure. We need to be able to determine when parents die to ensure child processes stop running in a variety of circumstances to avoid locks being held and ensure clean shutdown. Roughly based on https://gist.github.com/evansd/2346614 (Bitbake rev: 34974f5e30e9b09c016481e4c81c156a5f379784) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09bitbake: server/process: Handle SIGTERM more gracefullyRichard Purdie
Currently if you send a SIGTERM to the bitbake UI process, the system basically hangs if tasks are executing. This is because the server process doesn't actually try any kind of shutdown before exiting. This patch trys executing a stateForceShutdown command first, which is enough to stop any active tasks before the system exits. I also noticed that terminate can execute multiple times, once at SIGTERM from the handler and once from the real exit. Double execution leads to stack traces and potential hangs (writes to dead pipes), so ensure the code only can run once. With these fixes, bitbake much more correctly deals with SIGTERM to the UI process. (Bitbake rev: 1032ddddbe3241da02ebb3608a1c40f9123b9e80) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: bb.fetch: don't remove the clone when an update failsChristopher Larson
When our clone exists, but is out of date, and the attempt to update it fails, we don't necessarily want to remove the entire clone, particularly if it's a large repository. (Bitbake rev: 19af272ba5256653edeff6acbceeb09e3e478d61) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: bb.fetch: handle checksums consistently for mirrorsChristopher Larson
If the main fetch method doesn't support checksums, the user will not be defining them in the recipe, so we don't want to check them for premirrors/mirrors either. This ensures that we never error due to missing checksums on a git mirror tarball. (Bitbake rev: 24c79bbed361b37f12d3351af13602e3d4386f4c) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: bb.fetch.git: remove leading '.' from gitsrcnameChristopher Larson
When using an absolute file URI, there's no host, and the path starts with '/', the dir under ${DL_DIR}/git2/ ends up starting with '.', so is hidden. Remove any leading '.' to fix this. (Bitbake rev: 8dce6964d56b36a77fb113f2ad496cc992a5ff36) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: bb.cookerdata: include useful traceback for ExpansionError/ParseErrorChristopher Larson
Show the user only the portion of the traceback which was from the metadata, nothing from bitbake's internal calls. (Bitbake rev: c45054aef03393fa0bf70e853ddcfc55988493cf) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: bb.data_smart: retain traceback from ExpansionErrorChristopher Larson
This gives us the needed context of the original ExpansionError, which is invaluable when we have a chain of function calls in the expansion. (Bitbake rev: c514b6fbea77ede1b7871b89592a33ed39b1d71c) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: cooker/runqueue: Allow bitbake commands starting with do_Alex Franco
The output of "bitbake, -c listtasks pkg" lists tasks with their real names (starting with "do_"), but then "bitbake -c do_task" fails, as "do_" always gets unconditionally prepended to task names. This patch handles this error by checking whether a task starts with "do_" prior to prepending it with it when the task runlist is being constructed (and a few other corner cases). [YOCTO #7818] (Bitbake rev: dd3050ceef37ac556546e940aa596ce96ef6c8df) Signed-off-by: Alex Franco <alejandro.franco@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: event/server: Add _uiready flag to handle missing error messagesRichard Purdie
If you start and suspend a bitbake execution so the bitbake lock is held, then try and run "bitbake -w '' X", you will see bitbake return an error exit code but print no message about what happened at all. The reason is that the -w option creates a "UI" which swallows the messages. The code which handles this exit failure mode thinks a UI has printed the messages and therefore doesn't do so. This adds in an extra parameter to the UI registration code so that we can figure out whether its a primary UI or not and base decisions on whether to display information on that instead. This fixes the error shown above and some bizarre failures on the Yocto Project Autobuilder. [YOCTO #8239] (Bitbake rev: d1d60a68c2de40c2984d5040d14251c1be121b0b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-04bitbake: utils: Fix a potential error in movefileBenjamin Esquivel
bitbake utils' movefile is now prone to malform the destination file with duplicated file name strings. Fixing it to force a file name append iff the dest argument is a dir not a file name (Bitbake rev: 38dd27f7191da002a16c561be3790ce487045b01) Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03bitbake: toaster: Improve how default project is identified and fetchedElliot Smith
Command line builds are associated with a "default project" (as we currently require a build to have a project). This acts as a container for builds initiated outside Toaster. Currently, this project is marked as the default by its ID being 0. However, this doesn't work with MySQL, as MySQL won't allow 0 in a foreign key which references an autoincrement field. Instead, use an is_default field to track the default Project for builds initiated outside Toaster. Add a method to fetch this default project, rather than fetching a project with a magic ID. Add this default project in a migration, rather than as a side effect of a get_or_create() style method. Also ensure that builds always have a project explicitly assigned to avoid any magic with a build's project foreign key defaulting to 0 (as it no longer does). [YOCTO #7932] (Bitbake rev: 71b709a1bbc26d89d61873763b467d21e625b274) Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: brian avery <avery.brian@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03bitbake: utils: Specify dest file name in movefile()Benjamin Esquivel
When moving a file via the os.rename function, it was missing the destination file name which caused an OSError [YOCTO#8180] (Bitbake rev: b147ba0341d87e077bd2b09ef4355976ecd2d26b) Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03bitbake: build: delete tasks thoroughlyChristopher Larson
We want addtask to be able to bring back a deleted task, but we don't want its previous dependencies to come back with it, so rather than marking a task as deleted and then skipping tasks marked as such, actually delete the task and its dependency information in deltask. While we're in that part of the code, also fix a couple 'not foo in bar' instances to 'foo not in bar', which is preferred in python. (Bitbake rev: 94b3f3d6bdfbfa47f7eb3c3de64940a145b2ddd1) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-29bitbake: lib/bb/cooker: add interrupted flag to BuildCompleted eventPaul Eggleton
Allow any listeners for this event (such as buildhistory.bbclass in OpenEmbedded) to find out if the build was interrupted rather than completing normally. The value will be 0 if not interrupted, 1 if interrupted waiting for remaining tasks to complete, or 2 if force interrupted (stopping any running tasks immediately). (Bitbake rev: df2b778efd2ecc48f6c5a3ed446f6459f2250035) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-24bitbake: lib/bb/main: avoid importing all server/UI modules on every executionPaul Eggleton
We're importing the server and UI modules in order to check they are valid, but it turns out that that has some nasty side-effects. We don't actually need to do this except when --help is passed or the module doesn't exist, so rearrange the code so that we only do the module listing in those two cases. Additionally, let's just go ahead and catch all errors on import; we really don't care to have them cause a failure now. (Bitbake rev: c9dc6d9c86e8b887821a6d00346bd0b09e1da97c) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-20bitbake: main: Handle RuntimeError exception in list_extension_modulesRandy Witt
This exception was triggered in toaster from recent changes and is completely breaking the whole of bitbake. Add the exception to the list so at least only toaster is affected. (Bitbake rev: f64def7cb6069dc1134fcd546bb59e4030c7376f) Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: toaster logger: fix pylint issuesAlexandru DAMIAN
This patch fixes pylint issues in the toaster build data logger, toasterui. The following types of warnings are touched here: * fixing imports * unused variables are set to _ * logger calls now use lazy evaluation instead of formatting the string * correct whitespace identation * removes unneeded "pass" statements, and extra parantheses * disable specific pylint warnings when decideing to override them (Bitbake rev: 947d47f15048baa967f88e03d80014e88ce152aa) Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: toaster: fix pylint errorsAlexandru Damian
Prompted by issues discovered during running pylint on the toaster sources, this patch fixes: * missing import in toaster ui * improper call of function in toaster ui (logger.debug) * improper function definitions in bbcontroller * invalid references to objects in bldcontrol.models * proper initialization of object fields in ToasterTables Also inhibiting specific pylint errors in files where the problems are mis-identified. (Bitbake rev: 1c71955c416fb68455f7f70669aba4202c411807) Signed-off-by: Alexandru Damian <alexandru.damian@intel.com> Signed-off-by: Michael Wood <michael.g.wood@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: bitbake-layers: fix mapping files to layersPaul Eggleton
bitbake-layers needs to map recipe and class files to the layer they came from within the show-recipes and show-overlayed commands. However, it turns out that mapping a file to the layer it came from is not as trivial as it might seem. To do it properly we need to match the path to an entry in BBFILES then map that to the collection name using BBFILE_PATTERN, then map that to the actual layer using variable history. If it doesn't match any entry in BBFILES, then we can fall back to BBFILE_PATTERN (to handle classes and conf files). This fixes the layer name not showing up properly in the output of the show-recipes and show-overlayed commands for recipes in layers such as meta-intel that have subdirectories in BBFILE_PATTERN. It also fixes the priority not showing up in show-layers for such layers. As part of this I've added a function to VariableHistory which for a space-separated list variable gives you a dict mapping the items added to the files in which they were added. I've also fixed bb.utils.get_file_layer() and reduced some of the duplication by using this function in bitbake-layers. Also fixes the priority not showing up for layers such as meta-intel Fixes [YOCTO #8160]. (Bitbake rev: e852f6cabd7489585477ab567a1afeb2252377ac) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: lib/bb/main: consolidate UI/server extension listing and loadingPaul Eggleton
Provide us with a means of showing the list of UIs / server choices for the command line help, and do the processing in one place for both. (Bitbake rev: 24035c1daad5a904c3372d21d44191ee8182338f) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: lib/bb/parse: properly handle OSError when updating mtime cachePaul Eggleton
If a file no longer exists, drop it from the cache silently instead of generating a traceback. This was visible in some cases when a recipe was deleted when bitbake was resident in memory. (Bitbake rev: fe105b9042bdac4afd9f38fcf92bfdc2c04ec23f) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: cooker: drop appendlistPaul Eggleton
Now we have the bbappends list and all users have been converted over to use it, we don't need this anymore. (Bitbake rev: 279770c42d4c63aa2cebce331b55a92a564b50ac) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: cooker: drop appliedappendlistPaul Eggleton
Whilst collecting this list on the fly may be quicker, doing so within a function that's meant to *query* the list of bbappends is poor practice. (Bitbake rev: 5c12aa3b0010d7d1733e54a0ca7d0af465454210) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: Fix -m handling if cannot connect to serverPaul Eggleton
If we can't connect to the server we should error out, because it might not be that the server is actually dead - it might just be unable to execute commands. (Bitbake rev: d4b921676859d6ba4e1922fa4682ee941652f483) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: cooker: ensure prefile/postfile can work in memory resident modePaul Eggleton
The prefile/postfile options weren't working in memory resident mode because they weren't being passed through to the server, so ensure that they do get passed through and that the server is reset when the values come through. (Bitbake rev: a3f7dc042fc7b1c308bfd248431930eb8ba50326) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: command: ensure sync commands that read configuration see updatesRichard Purdie
Add a means of ensuring that synchronous commands that read the results of the configuration trigger a reparse of the configuration if any underlying files have changed. (Bitbake rev: aaf3cc024315450c1674819edf2a4e5cdf293f35) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: cooker: further limit inotify watchesPaul Eggleton
Unfortunately we were acting on inotify notifications about any files changing within the watched directories, not just the ones we actually care about. In OE the build directory is in BBPATH and hence it gets watched, and we write things like bitbake.lock and bitbake-cookerdaemon.log to that directory, hence effectively notifications were being tripped on every bitbake invocation. To avoid this, record which file/subdirectory we're interested in against each watched directory so we can ignore any events for files/subdirectories we don't care about. Additionally, if we move up to the parent dir, ensure we haven't already seen it before adding a watch on it (we were previously calling watcher.add_watch() on the same directory multiple times before in a typical OE configuration). (Bitbake rev: bc39b8da34c046b629c43fd0a8cac2efbf1c060f) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: command: intercept SystemExit to avoid trashing the serverPaul Eggleton
If sys.exit() is called within a command run over XMLRPC, the XMLRPC server is effectively trashed (apparently listening but no longer able to respond to commands). We need to intercept the SystemExit exception and deal with it as we would any other exception. (Bitbake rev: 95e391acbc3b4efd6c77637a1ce815012ae0f09b) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-19bitbake: Fix default function parameter assignment to a listPaul Eggleton
With python you should not assign a list as the default value of a function parameter - because a list is mutable, the result will be that the first time a value is passed it will actually modify the default. Reference: http://docs.python-guide.org/en/latest/writing/gotchas/#mutable-default-arguments (Bitbake rev: 7859f7388f2e3f675d0e1527cfde18625f36f637) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-10bitbake: runqueue.py: Add provides to taskdepdataMariano Lopez
Currently bitbake is the only one that knows the relation between PN and PROVIDES. In some cases it is needed to have this relation in the data store (the bootloader it's a good case). This adds the PROVIDES to the taskdata, so it would be in the data store as one field of BB_TASKDEPDATA. (Bitbake rev: a660787311d2961c66c0443bf0e2e094c9baef1b) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-03bitbake: cooker: Resolve file monitoring race issues when using memres bitbakeRichard Purdie
If you load memory resident bitbake, then change bblayers.conf or other configuration files, then execute your first command, bitbake will not notice the changes. This is because it adds the file watches during inital parsing, which happens at the time the first command is run. To fix this, we move the addition of the file watches to earlier in the process, so bitbake then does track them correctly. This also avoids some issues Paul was seeing with tinfoil2. (Bitbake rev: b9375c73e736003e66575969c9ea244403e47aeb) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01bitbake: bb.parse: properly error out on filesystem errorsChristopher Larson
We've had a long-standing bug where a legitimate error reading a file (IOError or OSError) is always suppressed as though it was a 'file not found' case. As a concrete example, if you do a `chmod 000 conf/local.conf`, it'll silently not parse local.conf, rather than erroring to let the user know about the problem. Fix this by handling the ENOENT case specifically. (Bitbake rev: e691312a3add222b04e7b2f52f8df6abcb9068bf) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01bitbake: bb.cookerdata: don't show traceback for ParseError/ExpansionErrorChristopher Larson
Tracebacks are of extremely limited usefulness in this context. The exceptions carry the necessary context already, and the user doesn't care about the calls in bitbake internals that led to an expansion or parse failure. (Bitbake rev: 9b95fa94eaae452ac7814f1e67c2f7a6314c52f1) Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01bitbake: bitbake: toaster: Wait for toaster gui to comeEd Bartosh
Set logging level for the Toaster logger to get messages to toaster_ui.log Reverted previous workaround. [YOCTO: #7965] (Bitbake rev: f1d8e0ff7233f17cff3a56db10d2443b392b407d) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01bitbake: wget.py: fix incorrect regexesAlexander Kanavin
[\.-_] means "any character between . and _" What was meant here is certainly "any character from the three characters .-_" (Bitbake rev: af13eaba627f199f91c048c435b9dbe19c79527f) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-01bitbake: cooker: properly fix bitbake.lock handlingRichard Purdie
If the PR server or indeed any other child process takes some time to exit (which it sometimes does when saving its database), it can end up holding bitbake.lock after the UI exits, which led to errors if you ran bitbake commands successively - we saw this when running the PR server oe-selftest tests in OE-Core. The recent attempt to fix this wasn't quite right and ended up breaking memory resident bitbake. This time we close the lock file when cooker shuts down (inside the UI process) instead of unlocking it, and this is done in the cooker code rather than the actual UI code so it doesn't matter which UI is in use. Additionally we report that we're waiting for the lock to be released, using lsof or fuser if available to list the processes with the lock open. The 'magic' in the locking is due to all spawned subprocesses of bitbake holding an open file descriptor to the bitbake.lock. It is automatically unlocked when all those fds close the file (as all the processes terminate). We close the UI copy of the lock explicitly, then close the server process copy, any remaining open copy is therefore some proess exiting. (The reproducer for the problem is to set PRSERV_HOST = "localhost:0" and add a call to time.sleep(20) after self.server_close() in lib/prserv/serv.py, then run "bitbake -p; bitbake -p" ). Cleanup work done by Paul Eggleton <paul.eggleton@linux.intel.com>. This reverts bitbake commit 69ecd15aece54753154950c55d7af42f85ad8606 and e97a9f1528d77503b5c93e48e3de9933fbb9f3cd. (Bitbake rev: a29780bd43f74b7326fe788dbd65177b86806fcf) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-27bitbake: build: Exit scripts with result of last shell functionAndre McCurdy
Since shell scripts run with 'set -e' the final exit at the end of the script can only be returning 0. However, for correctness and to follow the original intention of the 'cleanup' commands, let's fix the typo and return the success of the last shell function rather than the success of unhooking the exit trap handler. (Bitbake rev: bef724057f1ea81571dd3ac5a9cf862f818434b5) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24bitbake: data_smart: Improve override history loggingRichard Purdie
Calling record() for each override alteration is slow. Since we now expand overrides dynamically we don't have to record the log data at each alteration, we can instead print it directly from the existing data stores at variable history print time using the exact same data stores. This massively improves performance of the data store when parsing with bitbake -e for example, it will improve memory overhead as well. The only downside is that VariableHistory has to poke into the datastore for some of its data but that seems an acceptable tradeoff rather than double caching. (Bitbake rev: 100b447a161ef20fa559e39516cd32fa78e38262) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24bitbake: data_smart: Improve performance of infer_caller_details()Richard Purdie
As things stand now, bitbake -e (which turns on all the caller tracking) of OE-Core generates around 9.5 million stat calls which is slow and the largest single thing on the profile data. This is because infer_caller_details() calls traceback.extract_stack() which adds line contents to the traceback. This in turn calls python's internal linecache code which calls stat on every file for every callback. We don't even use that info. We only even want a single frame of the stack. Instead, open code for the pieces of information we need. Also, only obtain the stack once for both halves of the infer_caller_details() code. This reduces the number of stat calls to around 0.5 million and significantly improves parsing with bitbake -e. (Bitbake rev: 7be76d8f79ea92fd4bd36146eb9a4b86551b526d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24bitbake: data: Clean up datastore accesses and True/False valuesRichard Purdie
We should use the d.xxxVar syntax rather than the older function style. Also replace 0/1 with the more pythonic True/False. No functionality changes. (Bitbake rev: 90fdd69cca951f8bd2ff634f3b42fccd4fc03095) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24bitbake: data: Add newline after the last export in emit_funcAndre McCurdy
Minor formatting improvement in generated shell scripts. Add a newline after the last export emitted by emit_func. Also, remove chance of putting an extra newline in the middle of the exports, since we never want to do that. (Bitbake rev: 5117d9b5d32d5d81adf70fa3e3feac9cef654240) Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-24bitbake: tinfoil: Add shutdown methodRichard Purdie
One drawback to tinfoil is that once a cooker is created, it will hold the cooker lock and stop any other bitbake execution against a directory. Add a shutdown method to tinfoil, allowing other users to use the build directory after the tinfoil usage is no longer needed. (Bitbake rev: e44ce85fe551677fc0dcc1da4f789a0c13093ff1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23bitbake: tests/parse: Add BBCLASSEXTEND multiple data store corruption ↵Richard Purdie
reproducer One data store changing a variable poked through into a different data store. This test case replicates that issue where the value 'B' would become unset/disappear. We also enhance parsehelper to generate files with an optional suffix such as bbclass. (Bitbake rev: 5c4179f58a4e04f1c354df5f17d1860eb403f0ac) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-23bitbake: data_smart: Add CoW approach for overridedataRichard Purdie
Using deepcopy() caused a major performance regression. Turns out we can work with a shallow copy as long as we force the recreation of any list that we change, effectively a poor mans CoW. This isn't too invasive and avoids the huge overhead of deepcopy so this seems like the best way to have performance and a working data store. (Bitbake rev: 32dff4749c32f32e947c42c86f8357b8b607354b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-22bitbake: tests/fetch.py: Updated test name FetchMethodTest -> ↵Aníbal Limón
FetchLatestVersionTest Change the test name to be more specific on what is tested. (Bitbake rev: 3e39156bc330c4c726058a5b9c13d33e2daad89f) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>