summaryrefslogtreecommitdiffstats
path: root/bitbake/bin/bitbake-worker
AgeCommit message (Collapse)Author
2015-02-11bitbake: bitbake-worker: Use setsid() rather than setpgid()Richard Purdie
The bug has a long discussion of this. Basically, in some environments, the exact details of which aren't understood, a Ctrl+C signal to the UI is being transmitted to all the process children. Looking at the output of "ps ax -O tpgid", its clear the main process is still the terminal owner of these processes. stty -a on a problematic system shows: "-ignbrk brkint" and on a working system shows: "-ignbrk -brkint" The description of brkint would suggest this is the problem, setting up that terminal environment wasn't able to reproduce the problem though. It was confirmed that using setsid() caused the problem to be resolved and is probably the right thing to be doing anyway, so lets do it. [YOCTO #6949] (Bitbake rev: 81d90389edd4d4778d3aec86e0775ab98dd1496e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-31bitbake: bitbake-worker: exit normally when SIGHUPRobert Yang
Fixed: 1) Run "bitbake recipe" in the terminal 2) Close the terminal while building 3) $ ps aux | grep bitbake-worker There will be many processes, and they will keep the resources (e.g., memory), and won't exit unless kill or kill -9. (Bitbake rev: 72536d4e0cc3379001b730950afa012f7a96a79b) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-22bitbake: bitbake-worker: Fix bitbake -nRichard Purdie
Without this you see: File "bitbake/bin/bitbake-worker", line 201, in fork_off_task os._exit(child()) TypeError: an integer is required (Bitbake rev: cd477b5e77ab0373248b8a8fa30e1c7b8ea984fd) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-11bitbake: siggen/runqueue/bitbake-worker: Improve siggen data transfer interfaceRichard Purdie
We need to transfer some of the siggen data from the core/cooker into the worker instances. There was a partial API created for this but its ugly and its not possible to extend it from the siggen class. This patch completes the interface/abstraction for the data and means the class can extend/customise it in any siggen class. (Bitbake rev: cf2d642052979d236185c5b8ca2c5478c06e62ae) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-28bitbake: bitbake-worker: Extra profiling data dumpRichard Purdie
Currently we get no profiling oversight into either the main bitbake worker process, or the overall parsing before task execution. This adds in extra profiling hooks so we can truly capture all parts of bitbake's execution into the profile data. To do this we modify the 'magic' value passed to bitbake-worker to trigger the profiling, before the configuration data is sent over to the worker. (Bitbake rev: 446e490bf485b712e5cee733dab5805254cdcad0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-08-23bitbake: bitbake-worker: Improve sigterm handlerRichard Purdie
When processes terminate, we really want all of the child processes to terminate too. This was not happening for worker processes which spawned their own multiprocessing pools, leading to build hangs. This change ensures any sigterm gets passed to the whole process group. In local tests, this resolved some hanging process workloads I could generate. It does rely on signals being delivered in a timely fashion and there is a multiprocessing bug we have to work around there. (Bitbake rev: 96f8ea07ace1379380fab2d78eb592fa40c867d4) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-23bitbake: bitbake-worker: Drop BBHASH variablesRichard Purdie
Iterating through and calling setVar on this number of variables has significant overhead in the profiling data. By not setting this, we save 3,000 calls to setVar which gives a noticeable improvement to the speed of task execution. The BBHASH variables have since been replaced by accessing that data through the siggen code and going forward, that is the preferred way work with it. (Bitbake rev: 92526eadd09d19938762290e0492076174367583) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-11bitbake: bitbake-worker: Ensure children have default sigterm handlerRichard Purdie
The children of the worker should have the default SIGTERM handler, else they'll try and do cleanup which should only happen in the parent leading to all kinds of bizarre build failures. (Bitbake rev: a53c8d1f846d94082aa459996c4114f10970b8ef) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-10bitbake: bitbake-worker: Gracefully handle SIGTERMRichard Purdie
Currently if bitbake-worker handles a SIGTERM, it leaves the child processes to complete or hang. It shouldn't do this so hook the SIGTERM event and gracefully shutdown any children. (Bitbake rev: 551406f3f9ee94de09d2da6e16fea054c6dbfdb7) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-26bitbake: bitbake: Share BB_TASKDEPDATA with tasksRichard Purdie
Currently tasks have no knowledge of which other tasks they depend upon. This makes it impossible to do at least two things which would be desirable/interesting: a) Have the ability to create per recipe sysroots b) Allow the aclocal files to be present only for the entries in DEPENDS (directly and indirectly) By exporting task data through this new variable, tasks can inspect their dependencies and then take actions based upon this. (Bitbake rev: 84f1dde717dac22435005b79d03ee0b80a3e8e62) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-24bitbake: runqueue/bitbake-worker: Fix dry run fakeroot issuesRichard Purdie
When using the dry run option (-n), bitbake would still try and fire a specific fakeroot worker. This is doomed to failure since it might well not have been built. Add in some checks to prevent the failures. [YOCTO #5367] (Bitbake rev: f34d0606f87ce9dacadeb78bac35879b74f10559) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06bitbake: bitbake: Ensure ${DATE} and ${TIME} are consistentPeter Kjellerstedt
Due to the worker split the ${DATE} and ${TIME} variables could end up with different values for different workers. E.g., a task like do_rootfs that is run within a fakeroot environment had a slightly different view of the time than another task that was not fakerooted which made it impossible to correctly refer to the image generated by do_rootfs from the other task. (Bitbake rev: 756cc69ebf8bfe8455d0c90f288dd51be2499773) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-04bitbake: bitbake-worker: ensure BUILDNAME is available during executionPaul Eggleton
BUILDNAME is set from cooker by default, so since the worker split it will not be set when executing functions. In OpenEmbedded this results in /etc/version (which is populated from BUILDNAME) not having any content. Pass this variable value through to the worker explicitly to fix the issue. Fixes [YOCTO #4818]. (Bitbake rev: 92940b0427d9b2b3f95e27c230ec1e36638a34bc) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-31bitbake: bitbake-worker: import needed signal moduleValentin Popa
bitbake-worker makes use of the signal module but it doesn't import it. This patch fixes the issue. [YOCTO #4750] (Bitbake rev: c2ed639690f135994199eb24d964e37f57259e3a) Signed-off-by: Valentin Popa <valentin.popa@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14bitbake: prserv: Adapt autostart to bitbake-workerRichard Purdie
With the change to bitbake-worker we need to ensure the workers know how to contact the PR service, the magic 0 port and singleton is no longer enough. (Bitbake rev: c761751e259bb8e940552a28794b45887b5a72d9) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-14bitbake: runqueue: Split runqueue to use bitbake-workerRichard Purdie
This is a pretty fundamental change to the way bitbake operates. It splits out the task execution part of runqueue into a completely separately exec'd process called bitbake-worker. This means that the separate process has to build its own datastore and that configuration needs to be passed from the cooker over to the bitbake worker process. Known issues: * Hob is broken with this patch since it writes to the configuration and that configuration isn't preserved in bitbake-worker. * We create a worker for setscene, then a new worker for the main task execution. This is wasteful but shouldn't be hard to fix. * We probably send too much data over to bitbake-worker, need to see if we can streamline it. These are issues which will be followed up in subsequent patches. This patch sets the groundwork for the removal of the double bitbake execution for psuedo which will be in a follow on patch. (Bitbake rev: b2e26f1db28d74f2dd9df8ab4ed3b472503b9a5c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>