| Age | Commit message (Collapse) | Author |
|---|
|
Add a few comments at the top of the file explaining what it's for, and
a comment pointing out that you need to increment the cache version when
changing any code that changes the output.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Add some tests to verify that we are extracting "contains" information
from python expressions in the code in the bb.data and bb.codeparser
modules.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
Ensure we handle bb.utils.contains_any() as separate items, rather than
how we handle contains() where every item must be in the list.
Additionally, enable handling bb.utils.filter() which for the purposes
of looking at dependencies is the same as contains_any().
Additionally bump the codeparser cache and recipe cache versions to
invalidate the user's existing caches (ensuring that the changes take
effect and avoiding "taskhash mismatch" errors respectively).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
The code that determines variable dependencies uses the codeparser to
find references to "contains" type operations e.g. bb.utils.contains().
That function can take multiple items to check, and all specified items
have to be present. However this code didn't handle that - it assumed
that only one item would be specified and thus it was treating the
multiple items as a single item with spaces in between. Split the value
and check if all words are present in order to determine whether the
check is "set" or "unset".
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
|
|
When the fetcher retrieves file:// URLs, there is no lock file being
used. This means that in case two separate tasks (typically from two
concurrent invocations of bitbake) want to download the same file://
URL at the same time, there is a very small chance that they also end
up wanting to create a symbolic link to the file at the same time.
This would previously lead to one of the tasks failing as the other
task would have created the link.
Signed-off-by: Peter Kjellerstedt <pkj@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Normally bb.utils.which() is used by the unpack code to find a file in a variety
of places, but it is useful as a slightly more powerful version of os.which().
Support this by allowing it to only return matches which are executable files,
instead of just the first filename that matches.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
URL decoding was improved in the core a while ago and this looks like
a leftover from those times which caused urls needing a user/password to
fail. Use the parameters from the core instead of the broken split
implementation.
[YOCTO #11262]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* Turn reasons from a list into a string (usually there will be only one
reason, but the interface provides for more than one) and state up
front that the recipe is unavailable for clarity
* Use quotes around invalid recipe name
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This function calls cooker.findBestProvider() but didn't handle the fact
that that function returns a tuple (None, None, None, None) when there
is no matching recipe. (This fixes devtool in OpenEmbedded showing a
traceback when an invalid recipe is specified instead of a proper error
message.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This commit looks to see if FETCHCMD_s3 is set and if not, sets
it.
This is needed because I've use cases where I don't use aws, but
s3cmd (due to license).
Signed-off-by: Elizabeth 'pidge' Flanagan <pidge@toganlabs.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
BufferedLogger was sending log records to the target logger without
checking if the logger is enabled for the level of the record - and
handle() doesn't check this either (it's normally checked earlier when
the relevant log function is called e.g. logger.debug()), leading for
example to debug messages from codeparser getting printed when the log
level for the main BitBake logger was set to logging.WARNING.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Imagine you have an sstate mirror accessed over http and an SSTATE_MIRRORS
which maps file:// urls to http:// urls.
File urls set donestampneeded = False, http urls don't. This can result in
races in the try_mirror_url() code since it will trigger new downloads after
aquiring the lockfile as verify_donestamp() doesn't look at origud and there
is no donestamp.
verify_donestamp() already has code to look at origud, we're just missing
some code at the start of the function to do this. Fix it to avoid
these races.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Expanded and clarify documentation for the -v, --verbose and -D, --debug
options.
[YOCTO #9962]
Signed-off-by: Diana Thayer <garbados@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If you had a script that uses tinfoil and it failed to connect to the
BitBake server, you did't see any of the expected messages - this was
because client-side logging wasn't being handled at all. Since you'll
almost always want this when using tinfoil, have it use the new
bb.msg.logger_create() function to enable client-side logging by
default.
Relates to [YOCTO #11185].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We use this code to set up a logger with colour in a number of different
places, so create one function that does this and make some of bitbake's
utility scripts use it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We call git ls-remote to get the latest revision from a git repository,
however by calling runfetchcmd() we can end up recursively running
git ls-remote a number of times with OE e.g. if ${SRCPV} is in PV, ${PV}
is in WORKDIR, and ${WORKDIR} is in PATH (as a result of recipe-specific
sysroots), our call to runfetchcmd() exports PATH so _lsremote() will
get called again - with the end result that we run git ls-remote 30
times in quick succession (!). Prevent that from happening by using a
guard variable and returning a dummy value if it's called recursively.
Fixes [YOCTO #11185].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently shows the name of the signature files that were found when
--task is used.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently shows the name of the signature file that was found when
--task is used.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use --task <recipe> <task> to dump the signature info for a given
recipe and task. This is similar to the --task option of
bitbake-diffsigs.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The dependency graph in the bb.event.DepTreeGenerated and the
corresponding pn-buildlist and task-depends.dot from "bitbake -g"
contained entries like multiconfig:qemuarm.gcc (dot as last separator)
instead of the correct multiconfig:qemuarm:gcc (colon as separator).
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Displaying the environment data in a sorted manner makes it easier
to compare data between different setups and means you can know
where to find specific entries.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
FetchError isn't defined, use bb.fetch2.FetchError in this context.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The touch of .done explicitly specifies the path, so there's no need for
workdir=, and "os.path.join('.')" is identical to just '.'.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
list.index() isn't a particularly efficient operation, so keep track of our
position via enumerate() instead, which is more pythonic as well.
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Various run.do_xxx files export dozens of variables.
Presently they are in random order. Among other things,
this makes it difficult to compare two files for relevant changes.
This patch ensures they are enumerated/printed in a sorted
(alphabetical) order.
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
bb.data.expand(x, d) is deprecated API
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is what was reported when variable expansion in the 'depends'
varflag of a task was broken:
ERROR: Error for .../refkit-image-common.bb, dependency ${@ does not contain exactly one ':' character.
Task 'depends' should be specified in the form 'packagename:task'
It's not clear which task had this broken 'depends' and while one can
guess that variable expansion failed, the full expression isn't
printed either.
This is more useful:
ERROR: Error for .../refkit-image-common.bb:do_stage_swupd_inputs[depends], dependency ${@ in ' virtual/fakeroot-native:do_populate_sysroot ${@ ' '.join(['bundle-refkit-image-common-%s:do_swupd_list_bundle' % x for x in '${SWUPD_BUNDLES}'.split()]) } ' does not contain exactly one ':' character.
Task 'depends' should be specified in the form 'packagename:task'
The 'depends' part gets repeated intentionally, to ensure that it doesn't get overlooked.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The data we read from an HTTPConnection comes in the form of bytes, but
we need it as a string, so in Python 3 we need to decode it (missed in
the Python 3 migration).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Enable the following operations from a remote datastore to affect the
other end:
* setVarFlag()
* delVar()
* delVarFlag()
* renameVar()
In practice I don't expect these to be used much, but they should be
present so that the implementation is at least reasonably filled out
and that the tests pass.
Also add tests for the interface, mostly by subclassing the existing
local test classes so that they are using a remote datastore. (These
don't actually test remote usage via tinfoil, just that the
datastore's interface can be used.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There was a huge gap in the remote datastore code introduced in the
tinfoil2 rework - we weren't handling overrides at all, since these are
stored separately from the actual data in the DataSmart object. Thus,
when a datastore actually represents a remote datastore we need to go
back to that remote datastore to get the override data as well, so
introduce code to do that.
To avoid a second round-trip I had to modify the _findVar() function to
return the override data as well. This will increase the overhead a
little when that data is superfluous, but without making the function
even uglier I don't think there's a way to avoid that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In 2bc4d35fb32defc59cd6ed1fc87e35924c201a5c we fixed cases where an inactive
remove override was influecing the value of a variable. This adds a test
case for this issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When the regular expression for matching a variable name was amended
with allowing the ~ character as part of the variable name, this was
never done to the regular expression that matches export
lines. Similarly, the regular expression that was used for matching
unset variables also used the one without support for the ~ character.
This unifies the regular expressions. For good measures it also
corrects the regular expression used to match a variable flag name for
the unset command to match the one used when setting a variable flag.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For some reason, BBProgress.setextra() always adds a space at the end.
This prevents the task progress bars from filling the entire width of
the terminal, making them stop one character short. This looks odd
when shown together with the main progress bar, which does fill the
entire terminal width.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For example:
$ bitbake core-image-minimal --runall unpack
...
NOTE: Tasks Summary: Attempted 326 tasks of which 88 didn't need to be rerun and all succeeded.
$ bitbake core-image-minimal --runall patch
NOTE: Tasks Summary: Attempted 453 tasks of which 332 didn't need to be rerun and all succeeded.
This can replace fetchall as well:
$ bitbake core-image-minimal --runall fetch
NOTE: Tasks Summary: Attempted 135 tasks of which 119 didn't need to be rerun and all succeeded.
Signed-off-by: Matthew McClintock <msm-oss@mcclintock.net>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some task events are missing the 'virtual:native[sdk]:' prefixes.
The Toaster has code to help match missing prefixes, but needs
additional help resolving between 'native:' and 'nativesdk:', by
way of the '_package' event field.
[YOCTO #10849]
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: brian avery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Treat tar files compressed with 7-Zip in the same way as tar files
compressed with other compression formats.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Class for fetching files from Amazon S3 using the AWS Command Line
Interface. The aws tool must be correctly installed and configured
prior to use.
The class supports both download() and checkstatus(), which therefore
allows S3 mirrors to be used for SSTATE_MIRRORS.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Forcing the use of "\n" in mirror variables is pointless, we can just require that
there are pairs of values.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes [YOCTO #10405]
The section on the SVN Fetcher was missing information on the
"path_spec" option. I added this option and also updated the
examples at the bottom of the section to include that parameter.
Also, made the other two examples consistent.
I also removed the "date" parameter.
Also, updated the "protocol" parameter as well as the "modify"
parameter. For "modify" I removed the reference to "rsh". I
applied a small wording change to the "protocol" parameter.
Finally, I added a new "ssh" parameter.
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
|
|
The package-depends.dot and pn-depends.dot files are inaccurate,
missing out key dependencies such those made via the [depends]
flags. As such they can be misleading to the user.
They mainly exist for historical reasons, coming from a time
before we had task based execution.
This commit removes the two dated file formats and replaces
them with a recipe-depends.dot which is a flattened version
of task-depends.dot.
The old format files are removed if present so that the user
can't get confused about why data might not match between files.
The code is also rewritten to use 'with f: f.write()' syntax as
is more commonly used now. Also update the docs to match the change.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Cleanup some more usage of bb.data APIs in the fetchers.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The bb.utils.filter() function can be used to filter a variable
containing whitespace separated words based on another set of words.
It has been modeled after the bb.utils.contains_any() function.
A typical example of how it can be used is to simplify constructs for
PACKAGECONFIG that depend on DISTRO_FEATURES:
-PACKAGECONFIG ?= "\
- ${@bb.utils.contains('DISTRO_FEATURES', 'acl', 'acl', '', d)} \
- ${@bb.utils.contains('DISTRO_FEATURES', 'selinux', 'selinux', '', d)} \
-"
+PACKAGECONFIG ?= "${@bb.utils.filter('DISTRO_FEATURES', 'acl selinux', d)}"
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Python3 regards b"" as False so it is not being converted to a string by
d0f904d407f57998419bd9c305ce53e5eaa36b24. This confusingly causes three
different potential types for each member of the returned tuple.
Let's just assume that everything that's not None is a bytes object and
convert it to a string.
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Most angular2 packages have names of the form @angular/xxx.
The / obviously can't be used in a file name, replace it with -.
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Paul Eggleton
Peter Kjellerstedt
Richard Purdie
Ross Burton
Elizabeth 'pidge' Flanagan
Diana Thayer
Patrick Ohly
Christopher Larson
Juro Bystricky
Joshua Lock
Andre McCurdy
Matthew McClintock
David Reyna
Scott Rifenbark
Alexander Kanavin
Mike Crowe
Anders Darander