Age | Commit message (Collapse) | Author |
|
(Bitbake rev: 3603da4b5c4b7cbb29f93a1b8374198ee82e18eb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These urls are no longer adding much to the test coverage but the intermittent
network issues connecting to them are painful. Drop the urls.
(Bitbake rev: 598ea8584f792b30799231c4d3151d40852fc43c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit bdf5739c5d831dc97a7d81568f94a0953c71017f)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
vcs.pcre.org was a redirect to github which we use for subversion testing.
With the protocol changes at github and the removal of the redirect, use a
direct address for github.
(Bitbake rev: 3ce3efde167e8cd109659b81275a6935875877eb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 6230ca71eb7eb2a6db162e28a01727d00af5299b)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Username or password replacements in URIs were being appended rather than
replaced in mirror url remapping. Fix this and add a test case.
[YOCTO #13823]
(Bitbake rev: 5a13bca0e9088be4191f6120336c79dadfb86f76)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 66ad58bb87e5158aced572be4f1d5726bc97fcce)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: a95f05359e65c3336eeb8366c0c35a23d3821dda)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 07fca7e3ab696ba985b3ef86ab9031d688bf2df2)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: bcd4285116ea4990f10d53698e0a81ae1e7ce24c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 19c5605b72e25e06f5ea57bcc1672650e7978a7f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For some reason several git-annex files in Debian 10 buster
are read-only and removing them with "rm -rf" fails.
Fixes test failures like:
$ bitbake-selftest
...
rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied
rm: cannot remove '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource/annex/objects/f87/4d5/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855/SHA256E-s0--e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855': Permission denied
EE..................................ssss.sssssssssssssss.sssss.......................................................................................................
======================================================================
ERROR: test_shallow_annex (bb.tests.fetch.GitShallowTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1773, in test_shallow_annex
fetcher, ud = self.fetch_shallow(uri)
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1541, in fetch_shallow
bb.utils.remove(ud.clonedir, recurse=True)
File "/home/builder/src/base/poky/bitbake/lib/bb/utils.py", line 700, in remove
subprocess.check_call(cmd + ['rm', '-rf'] + glob.glob(path))
File "/usr/lib/python3.7/subprocess.py", line 347, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['rm', '-rf', '/tmp/tmpwmfn4w64/download/git2/tmp.tmpwmfn4w64.gitsource']' returned non-zero exit status 1.
Also, one "chmod" call was failing since the .git/annex subdirectory doesn't exist so just chmod
the whole temporary directory which should cover any directory name differences between
different git-annex versions. Fixes tests failing after chmod call:
Running 'export PSEUDO_DISABLED=1; unset _PYTHON_SYSCONFIGDATA_NAME; chmod u+w -R /tmp/tmpwmfn4w64/git//.git/annex' in /tmp/tmpwmfn4w64/git/
(Bitbake rev: 14c5f0735947307b9d69c57f7334fefaea7311b3)
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 7729ef2983c72867e99fad82d671069ba5cb32b2)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A CI user validating changes does not have any git push rights or
even a .gitconfig file so fix tests so that they run
by setting the user.name and user.email for the repo before
committing changes.
Fixes errors like:
ERROR: test_that_unpack_throws_an_error_when_the_git_clone_nor_shallow_tarball_exist (bb.tests.fetch.GitShallowTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 2055, in test_that_unpack_throws_an_error_when_the_git_clone_nor_shallow_tarball_exist
self.add_empty_file('a')
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1562, in add_empty_file
self.git(['commit', '-m', msg, path], cwd)
File "/home/builder/src/base/poky/bitbake/lib/bb/tests/fetch.py", line 1553, in git
return bb.process.run(cmd, cwd=cwd)[0]
File "/home/builder/src/base/poky/bitbake/lib/bb/process.py", line 184, in run
raise ExecutionError(cmd, pipe.returncode, stdout, stderr)
bb.process.ExecutionError: Execution of 'git commit -m a a' failed with exit code 128:
*** Please tell me who you are.
Run
git config --global user.email "you@example.com"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
(Bitbake rev: 1e1d1187e602aa1ef50c23551eec07f1a0cd81ef)
Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 57c0811f1ee19b6619f4840a39e01e3cb98c34c4)
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: ed8e1fd4cf9d5ac8a8203638add99d686b4b3521)
Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Insert an explicit pass to fetch all blobs needed by Git LFS, during the
fetch() function. This avoids the default behavior of Git LFS to wait
until 'git checkout' to begin downloading the blobs pointed to by LFS records.
Network access is not allowed at that point in the recipe's lifecycle.
[YOCTO #14191]
(Bitbake rev: 0efac66043662e7a2027192f50e92e982db2ba1c)
Signed-off-by: Matt Hoosier <matt.hoosier@garmin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To exercise the previous patch, add a test case.
(Bitbake rev: efdf6d5c4cd8155bc54dc667346f58dccb138e07)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The usehead url parameter for git repositories causes bitbake to use
whatever commit the repository HEAD is pointing to if the repository
happens to have the name 'default'. This is the default name so in many
cases it works just fine, but if a different name is specified with the
url parameter 'name=newName' then it will fail to parse the recipe with
an error along the lines of:
ERROR: ExpansionError during parsing /path/to/my/recipe.bb
Traceback (most recent call last):
File "/path/to/poky/bitbake/lib/bb/fetch2/git.py", line 235, in Git.urldata_init:
> ud.setup_revisions(d)
File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1302, in FetchData.setup_revisions:
for name in self.names:
> self.revisions[name] = srcrev_internal_helper(self, d, name)
File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1167, in srcrev_internal_helper(name='newName'):
if srcrev == "AUTOINC":
> srcrev = ud.method.latest_revision(ud, d, name)
File "/path/to/poky/bitbake/lib/bb/fetch2/__init__.py", line 1562, in Git.latest_revision(name='newName'):
except KeyError:
> revs[key] = rev = self._latest_revision(ud, d, name)
return rev
File "/path/to/poky/bitbake/lib/bb/fetch2/git.py", line 650, in Git._latest_revision(name='newName'):
raise bb.fetch2.FetchError("Unable to resolve '%s' in upstream git repository in git ls-remote output for %s" % \
> (ud.unresolvedrev[name], ud.host+ud.path))
bb.data_smart.ExpansionError: Failure expanding variable SRCPV, expression was ${@bb.fetch2.get_srcrev(d)} which triggered exception FetchError: Fetcher failure: Unable to resolve 'master' in upstream git repository in git ls-remote output for /path/to/local/git/repo
Let's fix this by setting the unresolved rev of _all_ repository names
to 'HEAD' when the usehead url parameter is specified. Update the
currently failing test, test_local_gitfetch_usehead_withname, to now
expect success.
This change preserves existing behavior that allows usehead to be
overridden by a valid looking revision if one happens to be specified
instead of AUTOREV.
(Bitbake rev: 01e901c44ab0f496606b1d45c8953dc54970204c)
Signed-off-by: Joey Degges <jdegges@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add tests for fetching a URL with the usehead parameter set and a
non-default name set. We currently expect the local version of this test
to fail since there is a bug in the usehead implementation that breaks
for non-default names.
(Bitbake rev: a2345110f217fac429f6ec15f699c87c39531e7c)
Signed-off-by: Joey Degges <jdegges@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The test `test_gitfetch_usehead' exercises a way to override the usehead
feature by setting SRCREV. It may not be obvious that this is what is
being exercised here so let's add some comments to document the expected
behavior.
(Bitbake rev: 1cd998c19101e3b093e81c126b3048c5d56058b0)
Signed-off-by: Joey Degges <jdegges@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move the local only usehead test to the FetcherLocalTest class so it
will be run when BB_SKIP_NETTESTS=yes since it does not require
network access.
Rename the usehead tests to better match the new organization.
(Bitbake rev: 137cfa13d5319bc91c3e5fe6c7062cb8c8484d64)
Signed-off-by: Joey Degges <jdegges@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 8b9471e02528320f6ef8d35840b5618883e85447)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Implements backslashes in local filenames.
A typical usecase for such a filename is a systemd unit.
Example: `dev-disk-by\x2dlabel-FOO.device`
(Bitbake rev: 14a35f273b579d5cd5fd92765b89c28f870dd577)
Signed-off-by: Leif Middelschulte <leif.middelschulte@klsmartin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: e7dab75c8d1923abcbbc7c9ac7de215d720ccf26)
Signed-off-by: Charlie Davies <charles.davies@whitetree.xyz>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're either hitting rate limiting with freedesktop.org or the
servers have intermittent network connections. Use our own mirror
of these repositories instead.
(Bitbake rev: a1b7ab5c9d5e64969f5ca0e41c0ac13c723e3761)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Globbing in file:// urls is terminally broken. Currently when its used, the
file checksum code is basically bypassed. This means changes to the source
files don't change the task checksum, the task doesn't rebuild when the
inputs change and things generally break.
To make globbing work generically, we'd have to scan the file system for
all possible matches to the glob and log whether they exist or not. We can't
simply log the files which exist, we have to also know which files could
later exist and influence the choice of file so we know when to reparse.
For a simple file://xxx/*, this could be done but for bigger patterns,
it becomes much more problemtic. We already support file://xxx/ in urls.
So, lets decide we'll not support globs in file://urls. Worse case users
can put files in a directory and reference that, moving files into place
if needed.
Remove all the glob special cases (see the comments if anyone doesn't
believe this is terminally broken) and error to the user if they have
such urls.
(Bitbake rev: 0c9302d950c6f37bfcc4256b41001d63f668bdf7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Its becomming clear the upstream server doesn't like this, drop these
two urls from the test, not sure we need them here anyway.
(Bitbake rev: ab2ef942dc21f9639793c972f2e546edf9444783)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
fixes:
svn: warning: W175002: Unexpected HTTP status 504 'Gateway Timeout' on '/openembedded/bitbake/!svn/vcc/default'
svn: E205011: Failure occurred processing one or more externals definitions
picked pcre2
[Yocto #13948]
(Bitbake rev: 1483d17108da02f5d615e83403d5fd6288ca957c)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Upstream is unavailable, breaking tests. Switch to a YP mirror since
if we can't reach that there are bigger problems. This should remove
a source of intermittent failures on the autobuilder.
(Bitbake rev: 232471083d6c574c7ada6320f9379ad7d7862a9c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 2e26e97129d4c54bf86cdea8f9791696a06a36b4)
Signed-off-by: Paul Barker <pbarker@konsulko.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This commit adds some tests to validate the npmsw fetcher:
- bb.tests.fetch.NPMTest.test_npmsw
- bb.tests.fetch.NPMTest.test_npmsw_bad_checksum
- bb.tests.fetch.NPMTest.test_npmsw_destsuffix
- bb.tests.fetch.NPMTest.test_npmsw_dev
- bb.tests.fetch.NPMTest.test_npmsw_mirrors
- bb.tests.fetch.NPMTest.test_npmsw_no_network_no_tarball
- bb.tests.fetch.NPMTest.test_npmsw_no_network_with_tarball
- bb.tests.fetch.NPMTest.test_npmsw_npm_reusability
- bb.tests.fetch.NPMTest.test_npmsw_premirrors
(Bitbake rev: ba205df20b6a07a4b1125332601c6c54c7b019b5)
Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This commit adds some tests to validate the npm fetcher:
- bb.tests.fetch.NPMTest.test_npm
- bb.tests.fetch.NPMTest.test_npm_bad_checksum
- bb.tests.fetch.NPMTest.test_npm_destsuffix_downloadfilename
- bb.tests.fetch.NPMTest.test_npm_mirrors
- bb.tests.fetch.NPMTest.test_npm_no_network_no_tarball
- bb.tests.fetch.NPMTest.test_npm_no_network_with_tarball
- bb.tests.fetch.NPMTest.test_npm_package_invalid
- bb.tests.fetch.NPMTest.test_npm_package_none
- bb.tests.fetch.NPMTest.test_npm_premirrors
- bb.tests.fetch.NPMTest.test_npm_registry_alternate
- bb.tests.fetch.NPMTest.test_npm_registry_invalid
- bb.tests.fetch.NPMTest.test_npm_registry_none
- bb.tests.fetch.NPMTest.test_npm_version_invalid
- bb.tests.fetch.NPMTest.test_npm_version_latest
- bb.tests.fetch.NPMTest.test_npm_version_none
(Bitbake rev: b166bd3cc6cc1ca63e885319091f17daaaaa2537)
Signed-off-by: Jean-Marie LEMETAYER <jean-marie.lemetayer@savoirfairelinux.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently these tests rely upon multiple uptream webservers which may change
or be unavailable. Add local copies of the test data, copy the httpserver
from OE-Core (used for testing there) and run these tests against a local
server instead.
(Bitbake rev: d5a4a352723258b4d499d3a51f340109c4f36f60)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
removed unused imports which made the code harder to read, and slightly
but less efficient
(Bitbake rev: 4367692a932ac135c5aa4f9f2a4e4f0150f76697)
Signed-off-by: Frazer Clews <frazer.clews@codethink.co.uk>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
[YOCTO #13586]
(Bitbake rev: f7a973604fbfd1059875ff1fabb3f885df9c5b95)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a test case to exercise the detection of git-lfs repositories and the
behaviour of the lfs parameter.
(Bitbake rev: a7cf4fc72cce357c425084dc2c5f35b5ed1a4b7b)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
FAIL: test_wget_latest_versionstring (bb.tests.fetch.FetchLatestVersionTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/pokybuild/yocto-worker/oe-selftest/build/bitbake/lib/bb/tests/fetch.py", line 1229, in test_wget_latest_versionstring
self.assertTrue(verstring, msg="Could not find upstream version for %s" % k[0])
AssertionError: '' is not true : Could not find upstream version for db
[YOCTO #13496]
The Oracle UPSTREAM_CHECK_URI used changed and does not work with logic in wget.
Update UPSTREAM_CHECK_URI and UPSTREAM_CHECK_REGEX to match the ones used in the
recipe. Also change the version being checked.
(Bitbake rev: 4cf5bb761c561ddea86f2875be35d05abc8486e1)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
to git clone
test_git_submodule_dbus_broker, test_git_submodule_CLI11, test_git_submodule_update_CLI11,
test_git_submodule_aktualizr and test_git_submodule_iotedge try to access the network via
git clone, which fails when there is no network available. Add the relevant skip tag.
(Bitbake rev: 9b0538753da0514e6518723dac537007abf7a649)
Signed-off-by: Luca Boccassi <luca.boccassi@microsoft.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
fetcher
Add a new option to the svn fetcher url "externals=allowed". This will allow
a user to enable svn co w/ externals. However, this does avoid the fetcher,
network access and mirror systems.
By default we no longer allow externals in the checkout. This ensures a
deterministic download. The system does attempt to identify SVN repos that
have externals enabled, and will warn the user. It is up to the user to
determine if these are necessary for the recipe. They may disable the warning
by adding "externals=nowarn" to the url.
In the future we would like to parse this list and see if the items are already
in the SRC_URI for that recipe, but with SVN being in limited use these days
that extra work is likely not worth the trouble.
Add test cases that generated a local SVN tree, with an external source
set to github bitbake in svn format. One test case checks that externals are
ignored, and one checks that they in downloaded.
(Bitbake rev: bf53f07c3647e57d8452a7743a2b04bcb72c80d6)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Because we are trying to avoid network activity and use our own fetcher,
the system emulates the behavior of 'git submodule init'.
git submodule init uses the .gitmodules file, where typically the module
name and path are the same. However, in this case the module name and
path (in the tree) were different. i.e.:
[submodule "edgelet/hsm-sys/azure-iot-hsm-c/deps/azure-c-shared-utility"]
path = edgelet/hsm-sys/azure-iot-hsm-c/deps/c-shared
url = https://github.com/Azure/azure-c-shared-utility.git
Previously the code assumed the 'path' was both the checkout location
under .git/modules, as well as the path to extract the components. This
proved to be incorrect as the .git/modules path needs to match the submodule
'name'. This causes the components that were fetched to be initialized in
the wrong location, which later caused the 'git submodule update' process to
skip not properly initialized modules.
A test case was added for this specific case to ensure a regression does
not appear in the future.
(Bitbake rev: fd27ab60d33553dba13de39394edaaac04e446b3)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There are much better ways to handle this and most editors shouldn't need this
in modern times, drop the noise from the files. Its not consitently applied
anyway.
(Bitbake rev: 5e43070e3087d09aea2f459b033d035c5ef747d0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the introduction of SPDX-License-Identifier headers, we don't need a ton
of header boilerplate in every file. Simplify the files and rely on the top
level for the full licence text.
(Bitbake rev: 695d84397b68cc003186e22f395caa378b06bc75)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This adds the SPDX-License-Identifier license headers to the majority of
our source files to make it clearer exactly which license files are under.
The bulk of the files are under GPL v2.0 with one found to be under V2.0
or later, some under MIT and some have dual license. There are some files
which are potentially harder to classify where we've imported upstream code
and those can be handled specifically in later commits.
The COPYING file is replaced with LICENSE.X files which contain the full
license texts.
(Bitbake rev: ff237c33337f4da2ca06c3a2c49699bc26608a6b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
new SRCREV
If the system had previously fetched a source repository for use by gitsm,
and then the SRCREV was updated and the new commit already existed, the system
would not re-evaluate the submodules and update them accordingly.
The cause of this issue was that need_update was being used, unmodified, from
the base git fetcher. It did not have any knowledge, nor did it care if we
were moving commits and needed to re-evaluate what was happening due to this
switch.
To fix the issue, during the download process we add all processed (by
gitsm) srcrevs to the git config file, as bitbake.srcrev. This allows us to
use a new need_update function that not only checks if the git commit is
present, but if we have previously processed this commit to ensure all of the
submodule components are also present.
This approach is used, instead of iterating over the submodules in need_update
to avoid a potential race condition that has affected us in the past. The
need_update is called only with the parent locking. Any time we need to dive
into the submodules, we need to lock, and unlock them, at each stage. This
opens the possibility of errors in either the code, or unintended race
conditions with rm_work.
This issue was discovered by William A. Kennington III <wak@google.com>. The
included test case was also written by him, and included unmodified.
(Bitbake rev: 30fe86d22c239afa75168cc5eb262b880886ef8a)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
getVar() has been defaulting to expanding by default for
a long time (2016), thus remove the True option from
getVar() calls with a regex search and replace.
Search & replace made using the following command:
sed -e 's|\(d\.getVar \?\)( \?\([^,()]*\), \?True)|\1(\2)|g' \
-i $(git grep -E 'getVar ?\( ?([^,()]*), ?True\)' \
| cut -d':' -f1 \
| sort -u)
(Bitbake rev: 3bba0dbd524cf72176a765957adff544ae5c255a)
Signed-off-by: André Draszik <andre.draszik@jci.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Move the submodule processing outside of the if statement to avoid any
optimizations that may happen.
Update the test cases to include the additional case, and split the other
test cases into individual tests to make it easier to figure out what
the failure may be.
(Bitbake rev: 0ec98c01ae50f95c9c74acf53013ac59e0e72b08)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If the submodule is in a subdirectory, it needs to have that structure
preserved. This means the unpack path needs to be in the 'dirname' of the
final path -- since the unpack directory name is specified in the URI.
Additional specific test cases were added to ensure this is working properly
based on two recent error reports.
(Bitbake rev: 8c8ecec2a722bc2885e2648d41ac8df07bdf660d)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Prior code happened to work with relative URLs, when the code was recently
restructured it caused all relative urls to no longer work. Restore the prior
code flow for relative support and better comment why that code is there.
(Bitbake rev: 14bb383af0ca98e0e04ec217e537b6a899f3658e)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The update_submodules and unpack_submodules functions were nearly indentical,
so we made a common function where the different behavior could be passed
in by the download and unpack users. The new function is process_submodules.
Moved the parse_gitmodules function under the new process_submodules, since
there are no external callers.
Refactor the file relative path processing to the URL translation code.
We also add a warning to the translation if a relative ssh URL has been
detected. Since this can cause a problem.
In the case of a relative URL that does not work after being translated,
it should be possible to use the MIRROR functions to manual translate the
generated relative URL into one that works properly.
Remove 'git config' processing on download contents. It turns out this is not
necessary since all of the later components work using the git fetcher.
Limit the 'git submodule update' call to only when unpacking a non-bare
repository. Submodules are always loaded as bare, so this prevents
intermediate unpacks from being attempted.
Finally, the test cases were updated and the new commit ids in the test
repository were updates as well.
(Bitbake rev: 610dbee5634677f5055e2b36a3043cd197fb8c51)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A custom shallow submodule is no longer necessary, as the regular git
fetcher is used and shallow handling works with the same code.
The only general difference between the regular change is simply declaring a
clone as shallow, when appropriate.
This also removes a potential race condition in copying repositories
vs cloning them.
The gitsm shallow fetcher test was revised to verify that the submodule
is shallow cloned along with the primary repository.
The first step of this change was to be sure to clean the gitsubmodule download
directory, as was previously done with the may gitsource directory.
Additional test components were added to verify commit counts, and an
obsolete (and likely incorrect) test for the .git/modules directory to be
empty was also removed.
(Bitbake rev: f9cc4684dcf4281acc557cda8cb35602354ac3d6)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In order to test the ssh processing in gitsm, we add an alternative
testcase that can be downloaded from git.yoctoproject.org. However,
this test case requries (read) access, via ssh, to git.yoctoproject.org.
(Bitbake rev: c8554cdc1287620fe8e8960561e614567879a010)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Also use assetIn instead of assertTrue which aids debugging failures.
(Bitbake rev: 19dee675bb9ad012d28e1e57a888931355a831cb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Provide more information in the case the sources are not found in the
unpack step.
(Bitbake rev: 27a2214bf6f2e7c61bfc422a20959a55f7e0d25d)
Signed-off-by: Urs Fässler <urs.fassler@bbv.ch>
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A user friendly error is throw when neither the clonedir nor
fullshallow exist. Without the check, a difficult to interpret error
is throw from within the fetch command.
(Bitbake rev: 30cf2506007d25162f0805051212f54c39034ff3)
Signed-off-by: Urs Fässler <urs.fassler@bbv.ch>
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 3434e64e7c077c1ecff7b36f02b6c6b59a7d1fe9)
Signed-off-by: Urs Fässler <urs.fassler@bbv.ch>
Signed-off-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|