| Age | Commit message (Collapse) | Author |
|---|
|
With the improved exception handling added in an earlier commit bitbake
now stops when recieving a SIGWINCH. This happens frequently when
disconnecting and reconnecting tmux sessions and bitbake didn't survive.
Restore old behaviour of ignoring interrupted system calls but keep
proper exception handling for other errors.
(Bitbake rev: 418358a595c75f45b8d15160ec42bbe569562d91)
Signed-off-by: Dan McGregor <dan.mcgregor@usask.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We add new pages for the layer importing, layer details,
showing project builds and project configuration.
The pages are in read-only mode, but they're needed as
to be able to verify the quality of data in the system.
Write capabilities will be added in a subsequent patch.
[YOCTO #6595]
[YOCTO #6590]
[YOCTO #6591]
[YOCTO #6588]
[YOCTO #6589]
(Bitbake rev: eed9ae5c2a2bd7567e12ae9a4f02a5a966a1e1a3)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We add new pages for the all-machines and all-targets
project-related views.
We update the existing template structure to create
a base project view, similar to a base build view, that includes
a breadcrumb.
Updating existing all layers view to use the new structure.
We update methods in the models to provide corrent
information display.
[YOCTO #6592]
[YOCTO #6593]
(Bitbake rev: 973f582a19441c1ec67061160e4c50ce03ed7b68)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We move the code to calculate build progress as percent
and the ETA of the build to the model, so that they can be
reused across different pages.
(Bitbake rev: c2ced09e7ea4a1762d2788bb12a761734d20fd8e)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 93887dadd5ee35557d320e96059c466d2e541065)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We update pages and queries to match the new project
models.
(Bitbake rev: 8f47ec259106da714260c7388cee75a6c1f4622d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We update the build controller application to make proper
use of the bitbake specification in project settings.
Added heuristic to detect when the meta* layers and bitbake
are checked out from Yocto Project poky, and use a single
git checkout.
Building without a proper oe-init-build-env is not yet supported.
(Bitbake rev: 9eafe14956013f5af39b68fc93e1b03e7ea1f5c2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We enhance the checksettings command to try to automatically
detect settings for running on localhost.
The checksettings will look for a nearby poky layer source,
for a nearby build directory, and will try to import settings
from "toasterconf.json" files found in the local layer.
On new configuration, it will also perform updates from the
layer source.
(Bitbake rev: 2aab77dfccb538e2b09829841ea6c464d40cafb1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We add a page where the user can browse
and import layers from all the layers known
by Toaster.
[YOCTO #6590]
(Bitbake rev: 59f4a9750a6c4f5360a91e3a4d1c03ceb42da086)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We create the navigation structure for the project section.
This includes adding URLs for configuration, builds, all layers,
layer details and all targets pages.
Changes to existing pages to exemplify navigation links.
(Bitbake rev: 6f0cb9d106129eb496a4c009d95b0727378e97c1)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We enable the admin interface in Toaster.
We add admin models for Build Environments (where the
sysadmin can configure where the builds take place)
and for Layer Sources (marking the upstream provider
for layer information).
The admin interface and associated data are enabled
only for the MANAGED version.
(Bitbake rev: 6618613c9210fb44d36d90f5f2404b435f10dfc8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We add a ToasterSettings table that will keep
installation-wide settings.
We update the models for the layer-related data storage to
make them compatible with the layerindex application API.
We add a LayerSource class that can update local data from
a LayerIndex-like compatible API.
Adding a command line option to perform information update
from all upstream layer sources.
Fair warning - there is no backward migration from 0013.
(Bitbake rev: 89e13579e1b44b738f10fadec8454aa0e6f073af)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We fix application discovery, tuples don't have .append().
Other minor fixes.
(Bitbake rev: a6f18aac3e6bb448d89a3425a2f756c6514ee595)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We refactor log saving to go through only one code path.
All logs that happened outside the build (i.e. before build
starting) now will be logged to either toaster_ui.log if the
build command ran in interactive mode, or to the build request
errors if the command ran in managed mode.
This enables proper display of error logs in project page.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the output in the python task/function run files is rather
incomplete and effectively useless. This enhances the code to take
advantage of the bitbake's dependency tracking and extend the output to
include dependencies. This makes the files more usable for debugging
purposes. Since this only happens at python function execution time, the
overhead is minimal in the grand scheme of things.
(Bitbake rev: 02667e048c3e632f857c87177c0022eaf5481802)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently we get no profiling oversight into either the main bitbake worker
process, or the overall parsing before task execution. This adds in extra
profiling hooks so we can truly capture all parts of bitbake's execution
into the profile data.
To do this we modify the 'magic' value passed to bitbake-worker to trigger
the profiling, before the configuration data is sent over to the worker.
(Bitbake rev: 446e490bf485b712e5cee733dab5805254cdcad0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This stream redirection of stdout is horrible. pstats takes a stream argument
so lets use that instead.
(Bitbake rev: 93d155f4766e27e7b004d13569aa03961fe89e3b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We should only add user/password options if they're specified as in the
fetch case. Patch from Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
(Bitbake rev: 303e6256947f4df4f283b75b7ccfdffa72864d67)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, if you have hard dependencies between setscene tasks (like avahi on
base-passwd through useradd.bbclass), other dependencies may not be installed
even if these exist in sstate. For example, avahi -> expat -> pigz-native
(and avahi -> base-passwd) yet if you cleansstate base-passwd:
bitbake gzip-native:do_clean avahi:do_clean expat:do_clean pigz-native:do_clean base-passwd:do_cleansstate
bitbake avahi | tee
you will currently see pigz-native being rebuilt even though it was in
sstate. The fix for this is to continue to iterate dependency chains
around hard blocked dependencies as per this patch.
After this patch is applied, you will see pigz-native installed from sstate.
(Bitbake rev: f787957a224e8c2682a19e5c4a4d9c86bdce52ba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 587b144ee409d444494d8d7f2d1c53ede8f7c953)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This fixes an issue when trying to use depexp without specifying a
recipe. Before change:
$ bitbake -g -udepexp
Traceback (most recent call last):
File ".../bitbake/bin/bitbake", line 382, in <module>
ret = main()
File ".../bitbake/bin/bitbake", line 370, in main
return ui_module.main(server_connection.connection, server_connection.events, configParams)
File ".../bitbake/lib/bb/ui/depexp.py", line 201, in main
logger.error(cmdline['msg'])
NameError: global name 'logger' is not defined
After change:
$ bitbake -g -udepexp
Please specify a package name for dependency graph generation.
(Bitbake rev: 984ad90b2f1e29634dc79803a4a0404ab0534039)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
since python-2.5 string.endswith() takes a tuple
(Bitbake rev: 86a67a1fd4244da9343dbf14deed1ad0d3003f32)
Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With a shared work task like gcc, the task can be run from a variety of
different recipes which may have different virtual extensions in place.
Depending on whether gcc-runtime or nativesdk-gcc-runtime's do_preconfigure
task is called for example will change the sorting of the task hashes due
to the way clean_basename currently works.
The correct thing to do here is sort on the base filename first, then any
extension when ordering the hashes. This means we do account for things
like recipes with both a native and non-native dependency but we also fix
the shared work case where we don't care whether it was a virtual version
or not.
(Bitbake rev: 2e80b5d10a5037ed6f0bc227a1f9b42529c87086)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, if an exception occurs in an event handler, the server shuts
down but the UI simply hangs. This happens in two places, firstly waiting
for events and secondly, sending events to a server which no longer exists.
The latter does time out, the former does not. These patches improve
both code sections to check if the main server process is alive and if not,
trigger things to shut down gracefully. This avoids the timeout in the
command sending case too.
This resolves various cases where the UI would simply hang indefintely.
(Bitbake rev: ac418e1112ff5f9c3157569316902f7a27fba4b4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Our usage of multitprocessing is problematic. In particular, there is a bug
in python 2.7 multiprocessing where signals are not handled until command
completion instead of immediately.
This adds a workaround into our wrapper function to deal with the issue.
(Bitbake rev: a16185e602b39b71475aa7e9ee80ad2b1f28d0f7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If we trigger a shutdown whilst parsing for whatever reason, in some
cases we were not closing down the parser threads. This change
ensures we do so. The function names are not entirely intuitive
but the behaviour is more correct (and commented). The previous bug
with the stdout failure would trigger this one, if there was a cold
cache and parsing was required (but not otherwise).
(Bitbake rev: 25bfa2478f1c3a8eb695e1e5760e06db5be8f2fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, IOErrors are just passed over due to the broken Exception
clause. A command like "bitbake X | <invalid command>" would break stdout
triggering a traceback. With these changes we print the exceptions, shut down
the server gracefully and exit which is a much nicer behaviour and is less
confusion to the user.
(Bitbake rev: 9544108f7b413038d871ce6ca88232de2f2434d9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if an exception occurs, we just run the idle handler again and
again, usually looping indefintely. Chances are the exception that occurred
will keep occuring and this is not a good place to be.
This was breaking the autobuilders with gigabytes of logs.
At least improve things so the cooker shuts down gracefully when this happens.
Some trace of the original problem may still be present on the console too!
(Bitbake rev: 1f28d8d3311262427938180435b68f0a35c2b330)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 000fa81013205dd9bc907ff7a61f06f57637212d)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make OE do_unpack logs slightly prettier.
(Bitbake rev: cec5859dec56529d76a0a685cc67b8da29453c69)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: fd293c5fcc89bc7a541d09e88a6aec5e0b2a1db2)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 4e9aef14d747c37444a4fc683f9641906906afe9)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its useful to have an event emitted when all of the sceneQueue tasks
have completed since the metadata can hook this for processing.
Therefore add such an event.
(Bitbake rev: 38d4f65bf1cbcdd5a2d60dff0e1d2859c34ed62e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The re module was imported at the top, no need to do it again.
(Bitbake rev: c29c2370635075949c75937c237602600f4f10dc)
Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
so that it doesn't check it twice for non-empty events
(Bitbake rev: e304ace6d8f7d0808ee401b7c01146b2798a81b4)
Signed-off-by: Namhyung Kim <namhyung@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It turns out the codeparser cache is the bottleneck I've been observing when running
bitbake commands, particularly as it grows. There are some things we can do about
this:
* We were processing the cache with "intern()" at save time. Its actually much
more memory efficient to do this at creation time.
* Use hashable objects such as frozenset rather than set so that we can
compare objects
* De-duplicate the cache objects, link duplicates to the same object saving
memory and disk usage and improving speed
* Using custom setstate/getstate to avoid the overhead of object attribute names
in the cache file
To make this work, a global cache was needed for the list of set objects as
this was the only way I could find to get the data in at setstate object creation
time :(.
Parsing shows a modest improvement with these changes, cache load time is
significantly better, cache save time is reduced since there is now no need
to reprocess the data and cache is much smaller.
We can drop the compress_keys() code and internSet code from the shared cache
core since its no longer used and replaced by codeparser specific pieces.
(Bitbake rev: 4aaf56bfbad4aa626be8a2f7a5f70834c3311dd3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If there is a corrupt/invalid cache file, we'd keep trying to reopen
it. This is pointless, simplify the code paths and delete the dead
file.
(Bitbake rev: c22441f7025be012ad2e62a51ccb993c3a0e16c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
in memory
If we're writing out merged data to disk, its safe to assume that either
we loaded the data or couldn't. Loading it again is relatively pointless
and time consuming.
(Bitbake rev: 14c31e18e4a4a52ee54056aeefb09ff8c295b475)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch fixes a name collision on the event.data
in the MetadataEvent class. event.data is used in the
event system as a copy of "d" in certain situations,
and this collision triggered a bug leading to data loss.
[YOCTO #6332]
(Bitbake rev: 3f191b7cfe95aea4d4e96babf001d62d45dd3aaa)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch enables the localhost build controller to
properly set the layers before the build runs.
It creates the checkout directories under BuildEnvironment
sourcedir directory, and runs the build in the buildir
directory.
Build launch errors are tracked in the newly added BRError table.
These are different from build errors, in the sense that the
build can't start due to these errors.
(Bitbake rev: 1868d5635b517e0fe1b874674ea7a78910b26e2e)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is the first commit on the project main edit page.
At this point we have:
* the default settings for a newly created project
* the ability to add targets
* the ability to trigger a build command, and have
the build executed
Project layers now have an optional field, allowing for
removal. Default meta, meta-yocto and meta-yocto-bsp
layers cannot be optional.
We add XHR calls for interactivity in the main page.
(Bitbake rev: 4e438854120cbd10319df1b571ec93e334002325)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We move the initial_data fixture to a code migration,
in order to make sure that we don't overwrite modified data.
The data is inserted only if no data is present, instead
of always overwriting the entry.
The check settings returns 0 if the check was ok.
(Bitbake rev: 525c316ac95c06c52fbcfc836ca5599ee9dbfe8b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A layer may live in a subdirectory of a git repository,
so we add a field to track this setting in the Project layers.
We add the Project schedule_build function, which creates
a build request from the current project configuration.
We also fix an import problem with Projects in views.
(Bitbake rev: 1b5835e5d48cbfb7d38e38437c45d161052dfb37)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We add explicit absolute paths for a directory where
the layer sources will be checked out (sourcedir) and
where the build activities will take place.
Adding minimal checking when starting the application in
order to make sure that BuildEnvironment (BE) settings are
usable. This check is ran by the toaster script at startup.
Modify the localhost bbcontroller to use the BE settings
instead of trying to self-configure on checked out sources.
(Bitbake rev: d17500d3f73fdeeef5f11fb3773a65e927be3f02)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 1cd369883469747a8158826bb8d67dcca2a8577f)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The clearcase fetcher is used to retrieve files from a ClearCase
(http://en.wikipedia.org/wiki/Rational_ClearCase) repository.
Usage in the recipe:
SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module"
SRCREV = "EXAMPLE_CLEARCASE_TAG"
PV = "${@d.getVar("SRCREV").replace("/", "+")}"
The fetcher uses the rcleartool or cleartool remote client, depending on which one is available.
Supported SRC_URI options are:
- vob
(required) The name of the clearcase VOB (with prepending "/")
- module
The module in the selected VOB (with prepending "/")
The module and vob parameters are combined to create
the following load rule in the view config spec:
load <vob><module>
- proto
http or https
Related variables:
CCASE_CUSTOM_CONFIG_SPEC
Write a config spec to this variable in your recipe to use it instead
of the default config spec generated by this fetcher.
Please note that the SRCREV loses its functionality if you specify
this variable. SRCREV is still used to label the archive after a fetch,
but it doesn't define what's fetched.
User credentials:
When using cleartool:
The login of cleartool is handled by the system. No special steps needed.
When using rcleartool:
In order to use rcleartool with authenticated users an `rcleartool login` is
necessary before using the fetcher.
(Bitbake rev: 76cff0aaad4cf10b325bceadd8ea90b3d75edcc2)
Signed-off-by: Dennis Meier <meier.dennis@siemens.com>
Reviewed-by: Roger Meier <r.meier@siemens.com>
Reviewed-by: Christian Liechti <christian.liechti@siemens.com>
Reviewed-by: Henrique Mendonca <henrique.mendonca@siemens.com>
Reviewed-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you hit Ctrl+C at the right point, the system processes the request
but merrily continues building. It turns out finish_runqueue() is called
but this doesn't stop the later generation and execution of the
runqueue.
This patch adjusts some of the conditionals to ensure the build really
does stop.
(Bitbake rev: 39b08c604ba713100e174c136f81f18eca6ef33d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patches fixes the build - project identification when
running under managed mode. The build is assigned to the
project from which it was triggered, and to the
build request, as to simplify relationships queries
in the database.
(Bitbake rev: af1d3373706d365f9138caec110fcb20a5966b7b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We add the logic to create a new project. This page
also serves as user registration and silent login
for users.
Once the project is added, the main project page is displayed.
(Bitbake rev: 8855daebe55917c4f5855413d02ae1f3f7f76571)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch is just a whitespace (end-of-line) fix.
(Bitbake rev: 5f33d1a240faa49bd8259a48eb15ef57a6b4ce55)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Dan McGregor
Alexandru DAMIAN
Richard Purdie
Robert P. J. Day
Olof Johansson
Bernhard Reutner-Fischer
Andre McCurdy
Namhyung Kim
Dennis Meier