| Age | Commit message (Collapse) | Author |
|---|
|
github is dropping support for git protocol in Git urls. Add code to remap
this to https in a way that could be used in older bitbake versions.
(Bitbake rev: 964958b8b11dc69fb289fc6c97c1dbc8d76ad0f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
os.errno used to happen to work but is invalid. Correct to use errno.
[YOCTO #13068]
(Bitbake rev: b3fc65289d33274cd5dace4d4ffe55be11c991f4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently these tests rely upon multiple uptream webservers which may change
or be unavailable. Add local copies of the test data, copy the httpserver
from OE-Core (used for testing there) and run these tests against a local
server instead.
(Bitbake rev: a21671e8a483ba8a6986d961987eda2d36ec61ca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It looks like we're about to see a lot of changes in branch names in repos. If
we have the prune option here, those old names are lost, the changes propagate
to our source mirrors and our old releases break.
We have the force option so any replaced references should be replaced, its only
orphaned branches which will now be preserved.
I believe this behaviour will cause us fewer problems given the changes that
look likely to happen.
(Bitbake rev: 12d8cc3fecd550c4aadf0519e80711d755ee75ba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Upstream is unavailable, breaking tests. Switch to a YP mirror since
if we can't reach that there are bigger problems. This should remove
a source of intermittent failures on the autobuilder.
(Bitbake rev: f4e60b29df88393302957c5bbdbe24ca38c4633c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
multiconfig dependencies should be excluded from BB_TASKDEPDATA.
However in thud, multiconfig filtering on task dependencies doesn't
happen until after deps has already been added to taskdepdata.
One manifestation of this results in multiconfig dependencies leaking
into staging processing.
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:extend_recipe_sysroot(d)
0003:
File: '/home/user/thud/meta/classes/staging.bbclass', lineno: 344, function: extend_recipe_sysroot
0340: #bb.note(" start is %s" % str(start))
0341:
0342: # Direct dependencies should be present and can be depended upon
0343: for dep in set(start):
*** 0344: if setscenedeps[dep][1] == "do_populate_sysroot":
0345: if dep not in configuredeps:
0346: configuredeps.append(dep)
0347: bb.note("Direct dependencies are %s" % str(configuredeps))
0348: #bb.note(" or %s" % str(start))
Exception: KeyError: 'multiconfig:musl:/home/user/thud/meta/recipes-kernel/linux/linux-yocto_4.18.bb:do_deploy'
This can be reproduced on thud by backporting the multiconfig.MultiConfig.test_multiconfig
test and mcextend bbclass from warrior.
d22b6e03a5504145abed7c2ca44cf12854df85da mcextend: Add helper class useful for multiconfig
d9018a3d9c828551c465b68b27920ec4681524ae selftest: Add multiconfig test
Flipping the ordering to match warrior's behavior fixes the test case.
(Bitbake rev: b690030efc87850951e8e3ecf4ae3c1dd1dc9b63)
Signed-off-by: Kyle Russell <bkylerussell@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There was a weird error in OE-Core where "devtool modify virtual/kernel"
was showing basehash mismatch errors. This was due to SRCPV sometimes being:
AUTOINC+b867b78b50_47b80ef7bd and sometimes AUTOINC+b867b78b50_255a750d28.
The latter hash comes from KBRANCH and meant sometimes the correct branch
was seen, sometimes it was not. The issue was complicated by the execution
using a remote datastore over tinfoil.
The problem turns out to be a fetcher caching error. If the datastore
changes, the cached url data may not be valid.
We therefore ensure we match cached url data against the datastore that
generated it, which appears to fix this issue.
(Bitbake rev: 97067634b1f149b56844b10e3a5e8d0d980b6e34)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
SIGTERM handler
The bitbake-worker child on the SIGTERM signal handling send the SIGTERM to all
processes in it's process group. In cases when the bitbake-worker child got
SIGTERM after registering own SIGTERM handler and before the os.setsid() call
it can send SIGTERM to unwanted processes.
In the worst case during SIGTERM processing the bitbake-worker child can be in
the group of the process that started BitBake itself. As a result it can kill
processes that not related to BitBake at all.
(Bitbake rev: b51877cbb8a7c713aa2bcec8354ec66e2f3dad51)
Signed-off-by: Ivan Efimov <i.efimov@inango-systems.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The variable values that result from parsing multiconfig should be
included in the cooker data hash, otherwise changes to these files won't
be detected, which will allow the parsing cache to be loaded with the
old values for the multiconfigs. This can either manifest as the
variable values simply not updating, or getting basehash changed errors
when building.
This bug was previously undetected because all of the multiconfig base
files were a direct file dependency in all parsed recipes. This was
fixed in 34137a00f60 ("bitbake: bitbake: cooker: Rename __depends in all
multiconfigs"), exposing this bug.
[YOCTO #13541]
(Bitbake rev: 6b045e074c6fea97d4e305a5a3c8bf82135d95eb)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring
self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0])
AssertionError: '' is not true : Could not find upstream version for db
[YOCTO #13496]
The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget.
Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the
recipe. Also change the version being checked.
(Bitbake rev: 8a58c3c64240c6ab14858d18e6b89febdb315311)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* noticed on read-only sshfs premirror
* it was showing the warning about renaming the file:
WARNING: laser-geometry-1.6.4-r0 do_fetch: Renaming /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz to /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441
and then failed because of movefile() issue with python3 (fixed in previous commit):
ERROR: laser-geometry-1.6.4-r0 do_fetch: Error executing a python function in exec_python_func() autogenerated:
with movefile() fixed, it let do_fetch continue and re-fetch locally with the right
checksum, but still the renamed file didn't exist, because of movefile failure - add
another warning when the movefile fails - for whatever reason - unfortunately movefile
prints error messages with just print() so the real error is hidden only in log.do_fetch
in this case:
movefile: Failed to move /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz to /jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441 [Errno 30] Read-only file system: '/jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz' -> '/jenkins/mjansa/sshfs/webos-ose-thud/downloads/laser_geometry-1.6.4.tar.gz_bad-checksum_1ee7479b8c5914b4ffae996945121441'
(Bitbake rev: d36438759344caa447d9a0bf30749a0aa31d1fba)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* with python3 this fails with:
File: 'bitbake/lib/bb/utils.py', lineno: 799, function: movefile
0795: try:
0796: os.rename(src, destpath)
0797: renamefailed = 0
0798: except Exception as e:
*** 0799: if e[0] != errno.EXDEV:
0800: # Some random error.
0801: print("movefile: Failed to move", src, "to", dest, e)
0802: return None
0803: # Invalid cross-device-link 'bind' mounted or actually Cross-Device
Exception: TypeError: 'OSError' object is not subscriptable
(Bitbake rev: 9f92322fa8d6f1a68c0c3f4984afdf65126b51dc)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
new SRCREV
If the system had previously fetched a source repository for use by gitsm,
and then the SRCREV was updated and the new commit already existed, the system
would not re-evaluate the submodules and update them accordingly.
The cause of this issue was that need_update was being used, unmodified, from
the base git fetcher. It did not have any knowledge, nor did it care if we
were moving commits and needed to re-evaluate what was happening due to this
switch.
To fix the issue, during the download process we add all processed (by
gitsm) srcrevs to the git config file, as bitbake.srcrev. This allows us to
use a new need_update function that not only checks if the git commit is
present, but if we have previously processed this commit to ensure all of the
submodule components are also present.
This approach is used, instead of iterating over the submodules in need_update
to avoid a potential race condition that has affected us in the past. The
need_update is called only with the parent locking. Any time we need to dive
into the submodules, we need to lock, and unlock them, at each stage. This
opens the possibility of errors in either the code, or unintended race
conditions with rm_work.
This issue was discovered by William A. Kennington III <wak@google.com>. The
included test case was also written by him, and included unmodified.
(Bitbake rev: 4ce92f43eeac6a4bfd06e8567fa6891614b5b3b0)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Because we are trying to avoid network activity and use our own fetcher,
the system emulates the behavior of 'git submodule init'.
git submodule init uses the .gitmodules file, where typically the module
name and path are the same. However, in this case the module name and
path (in the tree) were different. i.e.:
[submodule "edgelet/hsm-sys/azure-iot-hsm-c/deps/azure-c-shared-utility"]
path = edgelet/hsm-sys/azure-iot-hsm-c/deps/c-shared
url = https://github.com/Azure/azure-c-shared-utility.git
Previously the code assumed the 'path' was both the checkout location
under .git/modules, as well as the path to extract the components. This
proved to be incorrect as the .git/modules path needs to match the submodule
'name'. This causes the components that were fetched to be initialized in
the wrong location, which later caused the 'git submodule update' process to
skip not properly initialized modules.
A test case was added for this specific case to ensure a regression does
not appear in the future.
(Bitbake rev: ffd7ed530a17d22df576d986ac78428a6979e79c)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously, this would happen:
======================================================================
ERROR: test_vercmpstring (bb.tests.utils.VerCmpString)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/alexander/development/poky/bitbake/lib/bb/tests/utils.py", line 45, in test_vercmpstring
result = bb.utils.vercmp_string('1.', '1.1')
File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 143, in vercmp_string
return vercmp(ta, tb)
File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 135, in vercmp
r = vercmp_part(va, vb)
File "/home/alexander/development/poky/bitbake/lib/bb/utils.py", line 124, in vercmp_part
elif ca < cb:
TypeError: '<' not supported between instances of 'NoneType' and 'int'
----------------------------------------------------------------------
(Bitbake rev: bd953d56d007a8bfa5ecb6e753da4abfb035f9f2)
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The renaming of the __depends variable to __base_depends and file
watches needs to occurs for all multiconfigs, not just the base config.
Failing to do this for all multiconfigs will result in a huge increase
in the size of the parsing cache (about 5x for a single mulitconfig)
because all multiconfig caches will still depend on the base config
files. This will also seen a similar jump in the amount of time required
to load the parsing cache from memory, both because the cache is larger
and because of explosion of additional existence checks that must be
done for the base files.
[YOCTO #13359]
(Bitbake rev: da5d1560d4ad1c735f6166a5d9ce94f36c94186b)
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix the Bitbake version ID for Thud (1.40).
[YOCTO #13356]
(Bitbake rev: f8a23ad4f8f0336470a5d5b074b7a2d02ec94631)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move the submodule processing outside of the if statement to avoid any
optimizations that may happen.
Update the test cases to include the additional case, and split the other
test cases into individual tests to make it easier to figure out what
the failure may be.
(Bitbake rev: 7c1eb51d1e8a4c5f39bf9dddf05fb0b3598da72b)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 0ec98c01ae50f95c9c74acf53013ac59e0e72b08)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the submodule is in a subdirectory, it needs to have that structure
preserved. This means the unpack path needs to be in the 'dirname' of the
final path -- since the unpack directory name is specified in the URI.
Additional specific test cases were added to ensure this is working properly
based on two recent error reports.
(Bitbake rev: acca06d060e49b2441562b4dc94416af9ab8187e)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 8c8ecec2a722bc2885e2648d41ac8df07bdf660d)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Prior code happened to work with relative URLs, when the code was recently
restructured it caused all relative urls to no longer work. Restore the prior
code flow for relative support and better comment why that code is there.
(Bitbake rev: 39afa6ead1d72813b2a294f065d759bad08fb53d)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 14bb383af0ca98e0e04ec217e537b6a899f3658e)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The update_submodules and unpack_submodules functions were nearly indentical,
so we made a common function where the different behavior could be passed
in by the download and unpack users. The new function is process_submodules.
Moved the parse_gitmodules function under the new process_submodules, since
there are no external callers.
Refactor the file relative path processing to the URL translation code.
We also add a warning to the translation if a relative ssh URL has been
detected. Since this can cause a problem.
In the case of a relative URL that does not work after being translated,
it should be possible to use the MIRROR functions to manual translate the
generated relative URL into one that works properly.
Remove 'git config' processing on download contents. It turns out this is not
necessary since all of the later components work using the git fetcher.
Limit the 'git submodule update' call to only when unpacking a non-bare
repository. Submodules are always loaded as bare, so this prevents
intermediate unpacks from being attempted.
Finally, the test cases were updated and the new commit ids in the test
repository were updates as well.
(Bitbake rev: 8b02eb8e5d8e3a09e19ed96a4ccdf5f755e97a74)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 610dbee5634677f5055e2b36a3043cd197fb8c51)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A custom shallow submodule is no longer necessary, as the regular git
fetcher is used and shallow handling works with the same code.
The only general difference between the regular change is simply declaring a
clone as shallow, when appropriate.
This also removes a potential race condition in copying repositories
vs cloning them.
The gitsm shallow fetcher test was revised to verify that the submodule
is shallow cloned along with the primary repository.
The first step of this change was to be sure to clean the gitsubmodule download
directory, as was previously done with the may gitsource directory.
Additional test components were added to verify commit counts, and an
obsolete (and likely incorrect) test for the .git/modules directory to be
empty was also removed.
(Bitbake rev: 85dc1c65b661f9712ae98587d4d0d868146c8cff)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit f9cc4684dcf4281acc557cda8cb35602354ac3d6)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Greatly simply the unpack rule by copying the general functionality of
update_submodules as unpack_submodules. This will recursively construct
a set of urls and unpack them using the standard system behaviors.
The overall code may be slightly bigger, but this ensures that all of the
standard locks are inplace, ensuring the code doesn't change out from
under the unpack function. (This could have happened before due to using
'cp' instead of further unpacks on submodules. This may still happen in
shallow clones.)
(Bitbake rev: 02ce6783bd3cfc117c77ca0380a87b0e1c60d8db)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7d7ee630f1c65e7dd234f945edf5e3b3bcb0fc30)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It was reported that a race condition on a shared download directory could
occur with the gitsm fetcher, the result happened with a call to
git config
that occured within the update_submodules. Since the fetch is locked by the
upper level, it was probably the prior need_update(...) function causing this
because of some old code.
The gitsm class inherits the git class. The need_update was overridding the
version in gitsm, so that it forceably checked the submodules.
It's clear we can optimize the code by only updating if the primary repository
needs updating. Since we don't care if the submodule repository has changed
because if the primary hasn't, references to the submodule won't change.
(Bitbake rev: 4660933f83e528766d71eab662cc79dcf17b4be7)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 346338667edca1f58ace769ad417548da2b8d981)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In order to test the ssh processing in gitsm, we add an alternative
testcase that can be downloaded from git.yoctoproject.org. However,
this test case requries (read) access, via ssh, to git.yoctoproject.org.
(Bitbake rev: 54802438f2f2d73aba3e7fb8d6cce45aa7cffe77)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit c8554cdc1287620fe8e8960561e614567879a010)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The following appear to be the git supported formats:
proto://user:pass@host/path (URI format)
user@host:path (SSH format)
/path or ./path or ../path (local file format)
We adjust the parsing to find out if we have a URI format or not.
When we are NOT in URI format, we do our best to determine SSH or
file format by looking for a ':' in the overall string. If we find
a ':' we assume SSH format and adjust accordingly.
Note, in SSH format we simply replace the ':' with a '/' when constructing
the URL. However, if the original path was ":/...", we don't want '//' so
we deal with this corner case as well.
(Bitbake rev: a21b2598531d52123933a0ac1c4deeecfdd1697d)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dcac05e7dc6b0c5f8e63d36ad105b7eab41f0016)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It is possible for a submodule to be defined in the .gitmodules file, but
never initialized in the repository itself. This shows itself when searching
for the defined module hash you will get back a empty value.
Similarly we need to identify and skip defined but not initialized submodules
during the unpack stages as well.
Thanks to raphael.lisicki@siemens.com for their help is figuring out how
to resolve this issue.
Additionally a problem was found where, while unlikely, it may be possible
for the wrong revision to have been searched using ls-tree. This has been
resolved in the update_submodules function by keeping the correct revision
along with the submodule path.
(Bitbake rev: e9c965bd816c8e4e5cc8e45db8e0dab597ce5a6f)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 49e1ff920143dcd4e7f73933d7ab8a84b8ffa1a3)
Signed-off-by: Scott Murray <scott.murray@konsulko.com>
Signed-off-by: Armin Kuster <akuster@mvista.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There were a couple of problems with the multiconfig dependency resolution:
- the "if mc" condition triggering this code wasn't correct, it needs
to be "if more than one multiconfig" configured
- after adding providers we need to call add_unresolved again
and rebuild mcdeps within the "while new" loop
By fixing these issues we allow various other combinations of multiconfig
builds to work which previously didn't.
[YOCTO #13090]
[YOCTO #13130]
(Bitbake rev: b59cb2bc63940b9ebd8288de7ca4b1d9e96e026c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There was already a fix to ignore some multiconfig dependencies but its
'opposite' case wasn't covered. Cover that combination to so as to avoid
tracebacks in multiconfig builds.
[YOCTO #13090]
[YOCTO #13130]
(Bitbake rev: 3d4e557a539a8ef4a3b7a1e36fc99b118c01cbf4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The consumers of BB_TASKDEPDATA in OE metadata can't cope with multiconfig
dependencies. The choice is either to start adding code to each of them to
filter out multiconfig dependencies, or do this at source.
After consideration we've decided to do this at source as doing otherwise
is code duplication and error prone and in any case we've looked at, they
don't make sense.
[YOCTO #13090]
[YOCTO #13130]
(Bitbake rev: 7f157ea8ecf9ba259bb7e226cfd5f2870b7853a3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the mechanism for breaking out of the dependnecy loop analysis
code is broken and doesn't work leading to bitbake appearing to hang.
Add in a custom exception for this purpose and fix the code to exit
as intended, fixing the hang and making the dependency loop code
usable again.
(Bitbake rev: e39dbd72ef44eebae32f9fe3b75a1bf789605558)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Unseting -> Unsetting
(Bitbake rev: 238a936f60cbffb73f7b370b867e45f91925951c)
Signed-off-by: Scott Rifenbark <srifenbark@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The intent of the code was to catch one kind of error, it was actually swallowing
all exceptions and looping indefinitely. Fix it to work as intended.
This explains some mystery hangs we've been seeing.
(Bitbake rev: d89358c7b8aa69f12b8c384c4fdb493782633494)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are typos in the fixture files that need to be fixed to correctly
check out the 'thud' branch.
[YOCTO #13064]
(Bitbake rev: 03a2bc00a070794452aa1df0fd75a338bdccc47d)
Signed-off-by: David Reyna <David.Reyna@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the bitbake server recieved multiple connections, it currently closes
ones it can't handle (while its dealing with another). This is rather
antisocial behaviour which causes clients to quickly run through their
retries and abort.
Instead, queue any other connections until the current one is closed. This
way the client can decide when it wants to stop waiting for the server. If the
client is gone by the time we handle it, we handle that gracefully.
This also fixes a number of bugs in the connection handling where connections
which did drop early were badly handled causing tracebacks in the logs.
Also, handle queue incomming connections in a loop to ensure that the main
client handling doesn't starve that piece of the system.
This code was stress tested by running 50 connection attempts in parallel at
once, ensuring the code correctly handled them.
(Bitbake rev: 220193dc38c4e78cb7cf36132a3a5b499a35bc8a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Now that we set a timeout for the socket, it can return EWOULDBLOCK
if a signal or other event happens to wake up even if we don't timeout.
If this happens, retry the connection, else we simply see it quickly
loop through the retries and abort the connection in a very short
interval.
(Bitbake rev: c2000651a200530ba08161207ade5eea8bbeec43)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current value of 2 seconds has shown to be short in
wider testing.
(Bitbake rev: 469cc520593ba52775a373faad03072b7af05dba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're seeing hangs in oe-selftest where server startup and shutdown are
racing. The assumption was a connect would timeout however no timeout is
set which can leave processes hanging. Set a short timeout for
the connection to avoid this.
(Bitbake rev: 833d95f538c007c27c6eb8d8f2f97094dc2b1a41)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/main.py", line 464, in setup_bitbake
server_connection = bb.server.process.connectProcessServer(sockname, featureset)
File "/home/pokybuild/yocto-worker/nightly-oe-selftest/build/bitbake/lib/bb/server/process.py", line 490, in connectProcessServer
if command_chan_recv:
UnboundLocalError: local variable 'command_chan_recv' referenced before assignment
(Bitbake rev: 257ed88590883f46beec0164749e45733e67954e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its possible the read may return a smaller number of characters. Remove
the possibility by using a single character to signal the server is ready.
(Bitbake rev: 7fb2d6bed06439e59a81dd91798d886ee0a72e99)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're seeing slow startup in bitbake, add some timeing debug messages so
the logs are more useful for debugging when its slow.
(Bitbake rev: 92f61dfe409da2fdd7c609cf1125878aad273d04)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
On loaded production systems we've seen bitbake server take over
40s to start up. Increase the timeout to 90s which tries to avoid
failures in selftests.
The delays come from setting up the inotify watches (31s) so can't
really be avoided.
After 5s delay we now warn the user we're waiting for 90s so the
interactive exeperience shouldn't be much changed and its very
unlikely the user would see that anyway.
(Bitbake rev: 492a5c1b32bee1f5d7978954ee0ebdf4fccdf56f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When an indirect multiconfig dependency exists, such as:
A depends on B, B has a multiconfig dependency to C,and our build
target is A, the multiconfig dependency to C is not processed on
time, hence no providers are added for it, causing an exception in
the runqueue because the dependency does exist in it.
Call add_unresolved() for all available multiconfigs before processing
providers for multiconfig dependencies, detecting mcdepends on time so
providers for them can be added correctly.
(Bitbake rev: 25b585b981cc7e4ed48b0f7c89a075486fa1eb2b)
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Follow dependent hash changes recursively also when specifying two
signature files explicitly. Previously this was only done when using the
--task option.
(Bitbake rev: a5f5ec9e09e2b2891cade97d0568284fc064cb26)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The functionalities of bitbake-diffsigs and bitbake-dumpsig are so
similar that they can be merged into one. Add an option --dump to make
bitbake-diffsigs dump the last signature data instead of comparing it.
Keep bitbake-dumpsig as a symbolic link to bitbake-diffsigs. When it is
called as bitbake-dumpsig, it behaves as if --dump was specified.
Also make -D the short option for --debug again (the way it used to be,
and still was for bitbake-dumpsig), so that -d can be used as the short
option for --dump.
(Bitbake rev: e7130f8bd86843c0b780b2ecabd297cd35ddcbe3)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
start
We're seeing issues where the server doesn't start with no logs as to why. Allow
the server to print the last 60 log lines just in case this shows us something useful
about what is failing.
(Bitbake rev: 1351978585b76262cb104f3d609d79c184ee5d2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
10 log lines may not capture any full traceback, increase the number of
lines to 60 which covers most tracebacks.
(Bitbake rev: 2626ff964c0a5726037e539cfd07027aded0b7a9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
[YOCTO #12898]
There might be no bitbake-cookerdaemon.log, print a message for debugging.
(Bitbake rev: 9c3c965b9d90617e8aa2cf439019534a35a7bcb1)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This simplifies the code and makes it easier to read but has the
same functionality.
(Bitbake rev: b829d1b5eb486786cae088f6927530433a7e08e8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: ff2e5d435046886791551587a0988f21ddfe6e2a)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Kyle Russell
Ivan Efimov
Joshua Watt
Armin Kuster
Martin Jansa
Mark Hatle
Alexander Kanavin
David Reyna
Scott Rifenbark
Alejandro Enedino Hernandez Samaniego
Peter Kjellerstedt
Robert Yang
Ross Burton