| Age | Commit message (Collapse) | Author |
|---|
|
This imports the crate fetcher from OE-Core to resolve various module issues
and adds some very very basic tests of that new fetcher.
(Bitbake rev: 1f06f326fa8b47e2a4dce756d57a9369a2225201)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If an unexpected exception occurs in a setscene task, it is currently hidden
from the user and not recorded in any logs. This isn't helpful to debug
such failures.
Change the code so that even in the "silent" or "quiet" task case (setscene
tasks), a warning is shown with the traceback unless it was an "handled"
exception. This means the failing function can show it's own warning/error
instead if it wants to and then raise a handled event.
(Bitbake rev: 41dcdc61eb40def8c14a42e8d7bb9ce5a34afa57)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These variables were removed from the codebase, update the docs to match.
(Bitbake rev: 32180d5057c818a69987aada482e82acf3c72ef2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 143070a647b7db902a68d81f0b488e45e4d3f6bb)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
git is different from `git clone`'s
The URL expected by git fetcher when the SSH protocol is used differs
from the one given by Git servers to be used with the `git clone`
command.
Add a note making this specificity known to users.
Cc: Quentin Schulz <foss+yocto@0leil.net>
(Bitbake rev: 81c09d434aed13fcfc4ba426ae6a0b849447eb5f)
Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The different stamp policies were poor versions of the siggen code and task
hashes, predating it and being used by packaged staging. They had many
limitations, hence their replacement. I'm not aware of any users of that
code any more so I believe it and the assoicated stamp whitelist variable
can simply be removed.
(Bitbake rev: 98407efc8c670abd71d3fa88ec3776ee9b5c38f3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This patch changes behaviour. After this change any task which does not
have the network flag set will have networking disabled on systems that
support that (kernel version dependent).
Add a "network" task specific flag which then triggers networking to
be enabled for this task, it is otherwise disabled.
This needs to happen before we enter the fakeroot environment of the task
due to the need for the real uid/gid which we save in the parent process.
(Bitbake rev: 0746b6a2a32fec4c18bf1a52b1454ca4c04bf543)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a function which uses the unshare glibc call to disable networking
in the current process. This doesn't work on older distros/kernels
but will on more recent ones so for now we simply ignore the cases we
can't execute on. uid/gid can be passed in externally so this can
work with pseudo/fakeroot contexts.
(Bitbake rev: 9d6341df611a1725090444f6f8eb0244aed08213)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is deprecated in python 3.12 and Fedora 35 is throwing warnings so
move to the new functions.
(Bitbake rev: 68a18fbcb5959e334cf307d7fa8dc63832edb942)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These urls are no longer adding much to the test coverage but the intermittent
network issues connecting to them are painful. Drop the urls.
(Bitbake rev: bdf5739c5d831dc97a7d81568f94a0953c71017f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
BB_HASHSERVE_UPSTREAM
(Bitbake rev: 23cb09108b3064e46e79f6644b802f3539069088)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Always use a temporary file for the user config 'NPM_CONFIG_USERCONFIG'
because npm otherwise failed if configs and npmrc aren't set:
double-loading config "/dev/null" as "global", previously loaded as "user"
(Bitbake rev: 9f272ad7f76c1559e745e9af686d0a529f917659)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Quote destdir in run chmod command to support special characters in
package name and to avoid syntax error for packages like
'@(._.)/execute'.
(Bitbake rev: a701dfce3f0e74b4d7c687eeda83fe9c8e7240b1)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Internal bitbake shutdown timings must have changed recently as we're
seeing race issues on the autobuilder around the removal of the bitbake.lock
file. Improve the lockfile race code to cover bitbake's lockfile too
and use it in all the tests.
[YOCTO #14658]
[YOCTO #14652]
(Bitbake rev: bd1912bed64424f9fb28396b71bb49b6090ed087)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We should not redirect stderr to stdout if we need to get separated
stdout and stderr contents from Popen.communicate() later.
(Bitbake rev: 1ecc1d9424877df89fcda2f23c306998998a65ff)
Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: b6f0c29346ad6463c0e521248633e71886bfb5dc)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the console output for signature dumps e.g. with bitbake-dumpsig
isn't deterministic. Add some sorting to improve that.
(Bitbake rev: a663440b1623f97c9c169df5566e429fbc932a53)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: ba3aa8591327d43935f000c6884637997438ecb2)
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a parameter that limits sending Basic authentication in the
Authorization header to only the first host and not any that we're
redirected to. Ignoring potential security concerns, temporary AWS URLs
will reject any request that includes authentication details in both the
query parameters (from the redirect) and in the Authorization header.
Temporary AWS URLs are now being used for release assets from private
Github repositories. According to the previous discussion linked below,
they're also in use by bitbucket.
See also:
https://lore.kernel.org/bitbake-devel/CAC9ffDEuZL-k8199bUyN+8frjw6bg-g=vrumxxtvt+RVParQ8Q@mail.gmail.com/
(Bitbake rev: a6ab32013a4381a1b694ed46caf2c9da932644d0)
Signed-off-by: Justin Bronder <jsbronder@cold-front.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 7e1ca7ab50e3c6b642c3c11504c7c8f52cfa4528)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: f0442a30d3a8459195dbf51a778ffb6150688a0a)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a parameter `striplevel` to the SRC_URI to strip NUMBER leading
components (levels) from file names on extraction.
For example, if the archive `archive.tar.gz` contains `some/file`,
the SRC_URI `https://.../archive.tar.gz;subdir=other;striplevel=1`
will extract `some/file` to `other/file`.
This is useful to extract archives to a specified directory instead of
the original root component of the archive. The feature is required for
the npm support. The npm package contents should reside in a subfolder
inside a npm archive (usually it is called package/). npm strips one
directory layer when installing the package.
(Bitbake rev: aa4926e5d9c92f33b4434e2da709ff0bf3049f5b)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The tar command and its arguments are repeated for many archive types in
the unpack function. Unify the common parts in a variable to prepare
further extension.
(Bitbake rev: a08e57c9eaec1d9740a96149bf4843e576da4e5c)
Signed-off-by: Stefan Herbrechtsmeier <stefan.herbrechtsmeier@weidmueller.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
functions
Fixed when oe-core's path contians append/prepend/remove, e.g.:
/path/to/append_test/oe-core/
Initial a build in any build dirs:
$ bitbake -p
ERROR: Variable __anon_32__buildarea2_xhou_builds_append_test_layers_oe_core_meta_classes_patch_bbclass
contains an operation using the old override syntax. Please convert this
layer/metadata before attempting to use with a newer bitbake.
The anonymous fuctions has no names, so skip checking for it to fix the issue.
(Bitbake rev: ebd00330c41c75797529ff38d6a0955b93f05d1b)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 0d0e552d87c52c1f90b601698c3d54eec427ee21)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: ffd87a89393f25924f53dbc86dcf5a98c3a8d0ff)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This syntax is obsolete, update to the correct modern version.
(Bitbake rev: 744e9a4c2b6f44116435feb62ac64ff256c752e9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: d05529cc322ba9198edc12954b982729fc628fa0)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 3ebaba2b2eada35e26c1105291de0d85bd6d2bb8)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 0ac887253e6b2df187bcbac0b060dd39920538af)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Using "" as a target for .replace() is a really bad idea as it duplicates the replacement
for every character in the string. Add a testcase which triggered this and correct the
code to return the correct result.
(Bitbake rev: 3af1ecf049d2eed56f6d319dc7df6eb4a3d4eebc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The fetcher mirror code can go crazy creating lock filenames which exceed the
filesystem limits. When this happens, the code will loop/hang.
Handle the filename too long exception correctly but also truncate lockfile
lengths to under 256 since the worst case situation is lockfile overlap
and lack of parallelism.
(Bitbake rev: 63baf3440b16e41ac6601de21ced94a94bdf1509)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This code has been unchanged since 2006 apart from attempts to optimise
performance by minimising chdir() calls.
There is no reason the modern bitbake parser should be changing directory
all the time. We did have some path assumptions in the mists of time but
those were resovled and the code is deterministic and doesn't depend on
cwd now for parsing. We can therefore drop the changes in directory.
Also, TOPDIR is now being set by cookerdata in all cases so we don't
need the fallbacks in this code (which was used to effectively initialise
a value). We don't need to change TOPDIR when parsing a recipe, that makes
no sense. If we stop all the other messing around, we don't need to expand
TMPDIR either.
These changes have the potential to break some obscure use cases such
as an anonymous function assuming the current working directory, or some
case which depends on TOPDIR changing but I believe any such uses should
be fixed at this point.
(Bitbake rev: add5d488e1d6607a98441836075d01cb1dc9c0fa)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
TOPDIR is set internally deep within the parser to os.getcwd(). Rather
than do that, set it explicitly if not set. Note that modern code will
almost always have a bblayers.conf file which would have already set
TOPDIR before this new code.
Also fix findTopdir since the conf/bitbake.conf codepath is just
plain incorrect, it would find build metadata, not the current
build directory that bitbake would use. Again, the use of bblayers.conf
means hitting the fallback code was unlikely.
This change makes everything clear and explicit.
(Bitbake rev: c03df5283408dfd089b6317677d2b7af6fa73936)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It was clear with testing that we're asuming bitbake sets TOPDIR correctly
when running these tests. Remove that implict assumption and make it
explicit.
(Bitbake rev: c8f1eb377ceb1fc78cbfaed976107720ad78c075)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
>From contents from the Yocto Project manual
Took the opportunity to reorder SRC_URI fetchers and options
alphabetically.
(Bitbake rev: ee6a951de31471c610030d0cf745039a71706b50)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The right hand side of dependencies in the task dependency file generated
by bitbake -g was missing multiconfig prefixes, corrupting the data. Fix
this.
[YOCTO #14621]
(Bitbake rev: 1d5ca721040c5e39aefa11219f62710de6587701)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There have been requests to better document the contraints of fetcher
design and operation. This README attempts to start that.
(Bitbake rev: d9cda7835816ecd5a60f0575f6ce832ec9c6aced)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When build with nativesdk-python3 (3.10) from buildtools:
/path/to/bitbake/lib/pyinotify.py:55: DeprecationWarning: The asyncore module is deprecated. The recommended replacement is asyncio
The pyinotify.py's upstream didn't have any update in recent 7 years:
https://github.com/seb-m/pyinotify
And bitbake doesn't use the asyncore module, so remove the related code.
(Bitbake rev: 58fbb01c3e2111bef4f79f88e1aac1827350c82a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The previous fix introduced a race where the queue might not be empty
but all the parser processes have exited. Handle this correctly to avoid
occasional errors.
(Bitbake rev: 8e7f2b6500e26610f52d128b48ca0a09bf6fb2cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We had an issue where a webserver serving sstate had filesystem issues so
would accept connections but effectively not do anything with them. This
causes bitbake to hang whilst processing things like sstate objects inside
the checkstatus() calls. It can be replicated by setting up a server like:
socat -u TCP4-LISTEN:NNN,fork OPEN:/dev/null
and pointing SSTATE_MIRRORS in OE at that address.
Adding a timeout to the checkstatus calls of 15s means that whilst the
system will pause, it will then continue and not hang entirely. Since there
isn't a large transfer here, 30s should be a reasonable response time after
which we should fall back to building things ourselves.
[YOCTO #13716]
(Bitbake rev: edc3b0c3953cab675e29fe295b58cfa84ba811c3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
combined with +=/.=
Operations like XXX:append += "YYY" are almost always wrong and this
is a common mistake made in the metadata. Show warnings for these usages
with a view to making it a fatal error eventually.
(Bitbake rev: 8c31e75557dc6a8d8f407b5d24d6327889a3e3b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 19291665fa8b6cc331290f2542af3e8e653203f1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The previous fix for runall option handling had a small bug in it, it
didn't clear the originally processed task list which meant it was running
too many tasks. Fix this so the list is reset and rebuild correctly.
(Bitbake rev: 87c9e120897ed04dfc64d4752fc602f9bfcb8645)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently if you have something like:
SRC_URI = "file://foobar;subdir=${S}"
and a file like:
foobar/1/somefile
and then move it to:
foobar/2/somefile
the task checksums don't reflect/notice this. The file-checksum fields
encode two pieces of data, the file path and whether or not the file
exists. Changing the code which uses these fields is problematic.
We can however add a "/./" path element which means "include the bit
after the marker in the checksum" which the path walking code can use
to mark which bits of the path are visible to the fetcher.
I'm not convinced this is great design but it does appear to work.
(Bitbake rev: b4975d2ecf615ac4c240808fbc5a3f879a93846b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If one of the parse threads disappears during parsing for some reason, bitbake
currently hangs. Avoid this (and zombie threads hanging around) by joining()
threads which have exited.
(Bitbake rev: dc86a533d951d13643ce446533370da804782afc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
vcs.pcre.org was a redirect to github which we use for subversion testing.
With the protocol changes at github and the removal of the redirect, use a
direct address for github.
(Bitbake rev: 6230ca71eb7eb2a6db162e28a01727d00af5299b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When the user specify an invalid upstream hash equivalence server in
BB_HASHSERVE_UPSTREAM notify the user that we can't connect the server.
(Bitbake rev: be45aeb9a84f30c28711e87e2d2a4a86320a8d94)
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The runbuild option handling in runqueue was flawed as items deleted from the
main task list may be dependencies and hence cause index errors.
Rather than modify runtaskentries straight away, compute a new shorted list
and use that as an input to the second phase. This avoids the need to add tasks
back to the list meaning delcount can be simplifed to a simple counter.
The second use case in runonly doen't re-add items so doesn't have this
issue.
(Bitbake rev: 3428e3c54eb5cc03ff96f9cee6dc839afee7a419)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a test to ensure that a parameter like protocol=git can be rewritten
to a different url and protocol.
(Bitbake rev: 69b4f9a09ff74378788cc2ec1ad58cd66b27ca59)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Michael Opdenacker
Quentin Schulz
Stefan Herbrechtsmeier
Anton Mikanovich
Alexander Kanavin
Justin Bronder
Robert Yang
Jose Quaresma