Age | Commit message (Collapse) | Author |
|
If the subpath parameter to the git fetcher ends with a trailing '/',
bb.utils.prunedir() will be called on '/'...
Fixes [YOCTO #7620].
(Bitbake master rev: 380a3fb372c8b0a53dd7528562e6e7a222dc76ef)
(Bitbake rev: faffa1c4a4d8353b21a0d359076153da0dc31a05)
Signed-off-by: Anders Darander <anders@chargestorm.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, if you reference a file url, its checksum is included in the
task hash, however if you change to a different file at a different
location, perhaps taking advantage of the FILESPATH functionality, the
system will not reparse the file in question and change its checksum to
match the new file.
To correctly handle this, the system not only needs to know if the
existing file still exists or not, but also check the existance
of every file it would have looked at when computing the original file.
We already do this in the bitbake parsing code for class inclusion. This
change uses the same technique to log the file list we looked at and
if files in these locations exist when they previously did not, to
invalidate and reparse the file.
Since data stored in the cache is flattened text, we have to use a string
form of the data and split on the ":" character which is ugly, but is
an internal detail we can improve later if a better method is found.
The cache version changes to trigger a reparse since the previous
cache data is now incompatible.
[YOCTO #7019]
(Bitbake rev: 67ebf368aab8fbe372374190f013bdf2c83c59de)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In some cases for cache purpoes we not only need to know which file
is going to be used but also which paths were considered. Add a
localpaths method which includes the history.
The core which() funciton already supports this, this just extends
the function to preserve the extra data we need. localpath becomes
just a special case of the case with history.
(Bitbake rev: d71407dbbf82659f245e002ecaad02b26838f455)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This makes it possble to fetch Gerrit review references which are
normally stored under refs/changes.
Please disregard previous patch with the same topic.
(Bitbake rev: ab8cbf2a71750f5ea36e218036b050857301607b)
Signed-off-by: Fredrik Svensson <fredrik.svensson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 2ac33aac3446cb12227f1b8daa5f27f417c9bb9e)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Forcing strict to be a string, to avoid problems when performing comparisons
[YOCTO #6762]
(Bitbake rev: b8ed2098bdea2afd93ab4e3e1b834f3a31cb60de)
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This fixes urls of the form file://some/path/file;subdir=b. It also
adds in a couple of tests so we now tests these corner cases.
(Bitbake rev: 46306912a96444790efa9418d934dfdd36773ba1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Check if the 'subdir' parameter exists and assign it to 'destdir' so that
files are copied in ${WORKDIR}/destdir. This fixes urls that are of the form
file://a;subdir=b.
(Bitbake rev: 836a986b365eb9798563ec08d90b346596de7791)
Signed-off-by: Roxana Ciobanu <roxana.ciobanu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 5bf5a937b26896bedbfea78dd1d62bce5a26ac2a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I could apply the label "release_1.0" to a super project that contains
many sub projects. If my recipes have SRC_URI's that use that label but
grab different sub-folders, than there's a bug where the cached localfile
(tar.gz) will not be unique and reused at the wrong times.
SRC_URI = "p4://perforce::1234@//depot/SuperProject/MiniProjectAAA/...;label=release_1.0 \
p4://perforce::1234@//depot/SuperProject/MiniProjectBBB/...;label=release_1.0"
(Bitbake rev: 3b5b1703b77490116dda895b29737cea51a3d6a0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We should only add user/password options if they're specified as in the
fetch case. Patch from Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
(Bitbake rev: 303e6256947f4df4f283b75b7ccfdffa72864d67)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Make OE do_unpack logs slightly prettier.
(Bitbake rev: cec5859dec56529d76a0a685cc67b8da29453c69)
Signed-off-by: Andre McCurdy <armccurdy@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The clearcase fetcher is used to retrieve files from a ClearCase
(http://en.wikipedia.org/wiki/Rational_ClearCase) repository.
Usage in the recipe:
SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module"
SRCREV = "EXAMPLE_CLEARCASE_TAG"
PV = "${@d.getVar("SRCREV").replace("/", "+")}"
The fetcher uses the rcleartool or cleartool remote client, depending on which one is available.
Supported SRC_URI options are:
- vob
(required) The name of the clearcase VOB (with prepending "/")
- module
The module in the selected VOB (with prepending "/")
The module and vob parameters are combined to create
the following load rule in the view config spec:
load <vob><module>
- proto
http or https
Related variables:
CCASE_CUSTOM_CONFIG_SPEC
Write a config spec to this variable in your recipe to use it instead
of the default config spec generated by this fetcher.
Please note that the SRCREV loses its functionality if you specify
this variable. SRCREV is still used to label the archive after a fetch,
but it doesn't define what's fetched.
User credentials:
When using cleartool:
The login of cleartool is handled by the system. No special steps needed.
When using rcleartool:
In order to use rcleartool with authenticated users an `rcleartool login` is
necessary before using the fetcher.
(Bitbake rev: 76cff0aaad4cf10b325bceadd8ea90b3d75edcc2)
Signed-off-by: Dennis Meier <meier.dennis@siemens.com>
Reviewed-by: Roger Meier <r.meier@siemens.com>
Reviewed-by: Christian Liechti <christian.liechti@siemens.com>
Reviewed-by: Henrique Mendonca <henrique.mendonca@siemens.com>
Reviewed-by: Pascal Bach <pascal.bach@siemens.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There may be a need to set the user for the transport rather than the subversion
command itself. Add a parameter to the url to allow this.
[YOCTO #6475]
(Bitbake rev: 56c294dc30b6c2575b1cf904e26b8b8bef7677c2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Handle the following situations in a URL (e.g. in SRC_URI):
* Trailing semicolon in a URL - this is now ignored.
* Parameter specified with no value (no equals sign). This still
produces an error, but at least it is MalformedUrl with a proper
message rather than "ValueError: need more than 1 value to unpack".
(Bitbake rev: bfd13dfbc4c9f1dd8315002271791b1d9e274989)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A fix for the former patch when checking out a repository with
username and password using HG
(Bitbake rev: 0e7b594ccbceb3149f38776cea204807031ef69f)
Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: ef2bf63e56b87e19d1a9059dd2d81a9a1a537254)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Ensure there is default command available if FETCHCMD_p4 isn't set.
Also clean up the getVar references whilst here.
(Bitbake rev: d150226d11d5f041f78c8c3ce4abc5465dbc81d8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
specified in url
(Bitbake rev: 9a328262e047562a6edf7b2ec7b8a1949e287cbe)
Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 42b7be1a6c5ff559f4a213eed04b370ca955460d)
Signed-off-by: Nick Lewis <nick.lewis@usa.g4s.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The initial version of this patch was reverted in
0a94e568152de550dedc8135a766beb18bf064ab
However, it was later agreed upon that it was reverted due to
a misunderstanding during code review; the patch is ok.
This version of the patch also removes an unneeded initialization of
the 'checksum' variable outside the scopes it is being used in.
(Bitbake rev: 9ee19a3ca2f8e11a91f0289ea3486310c61d40f2)
Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Following up a previous patch for mercurial fetcher, I just fixed a
problem when calling update on a repository with subrepositories enabled.
Attached a patch that fixes this problem
>From caae519a2bd64bf7c729bb26aff344827def47fb Mon Sep 17 00:00:00 2001
From: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Date: Tue, 29 Apr 2014 15:29:47 +0200
Subject: [PATCH] bitbake: - fixed authentication issues in case of using sub
repositories
(Bitbake rev: dfa041c940caad12da50126a559afc8de089eeda)
Signed-off-by: Volker Vogelhuber <v.vogelhuber@digitalendoscopy.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 6dfb33fd7204e09815bde7ba5bf21941a73fe965.
It causes an unindeded change in behaviour.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Cleanup the fix done in f9416e76e272ba3249abff099f6f3a47fe82e03e.
Instead of adding continue statements we can just move the last
statements into the final else-clause.
(Bitbake rev: 6dfb33fd7204e09815bde7ba5bf21941a73fe965)
Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We only ever clone other repositories, if there were a problem such as power
failure, we'd blow away data and rebuild. As such we don't need fsync(). With
filesystems like ext*, the fsync pushes nearly all the data out to disk
which impacts all running processes.
We therefore set a configuration parameter to disable the fsync() calls.
Also fixup a case where basecmd wasn't being used for no good reason.
(Bitbake rev: 0a26abaf3a1e34d556c9375068dd17c879568d0f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If there is a directory matching a wildcard in SRC_URI when getting file
checksums, we should recurse into that instead of producing an error.
(Bitbake rev: ae87b7eb414e3d5eefd2effec7b30c22d2186b02)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The Local fetcher's localpath is returning the parent directory for a
wildcard match; we need to handle this and add the wildcard
specification so that we checksum the correct files.
Fixes [YOCTO #6127].
(Bitbake rev: 4a90edd62c16cbf41b5b93280e155077564c774a)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Previously, if a file listed in SRC_URI was missing at the time of
parsing, and then was added and bitbake run again, it would not be
picked up because the recipe was cached without it. If we allow the file
to be added to the list of files to checksum, then it will be checked
for and found on the second run.
Fixes [YOCTO #4790].
(Bitbake rev: 71da822762cb298261cccdfa54b9c0fea02c3c5d)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
For a while its been puzzling me why connman-gnome rebuilds as often as it
does. It turns out you can trigger this with a new checkout of the metadata.
The SRC_URI that is causing the problems is:
SRC_URI = "file://images/*"
and rather oddly the results in checksums for a file "." being added to
the tree, e.g.:
('.', 'ab48a68186f0e0f277c21ef4cb390b4b')
The problem is that when iterating files lists, the checksum variable can
become set yet we don't break the out from the for loop, which leads to
odd (and non-deterministic) entries being added into the file checksum list.
The exact item added probably depends on the order of items on the disk.
Before this change, bitbake-diffsigs on connman-gnome:do_fetch would report:
This task depends on the checksums of files: [
('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'),
('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'),
('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'),
('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'),
('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'),
('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'),
('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'),
('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'),
('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'),
('.', 'ab48a68186f0e0f277c21ef4cb390b4b')]
Afterwards:
This task depends on the checksums of files: [
('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'),
('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'),
('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'),
('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'),
('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'),
('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'),
('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'),
('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'),
('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85')]
which is correct and deterministic without the "." entry.
(Bitbake rev: f9416e76e272ba3249abff099f6f3a47fe82e03e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When setting the username after already having set the password, the password
was unexpectedly reset. This change fixes this issue and introduces unit tests
to make sure it doesn't happen again.
(Bitbake rev: 25faef3a047f9c7564089463d7c96f6910b640cb)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is a case in meta-intel where a SRC_URI contains both a query string and
URI parameter:
https://edc.intel.com/Download.aspx?id=6190;downloadfilename=LIN_IEMGD_1_14_GOLD_2443.tgz
Python's urlparse thought the URI parameters were part of the query parameter
value, but in the bitbake context this is obviously not the case. As bitbake's
usage isn't really RFC compliant, we have to extract and remove the URI parameters
*before* urlparse sees the URI.
(Bitbake rev: c2f27ae4271985b48f957747b6ea372c8699cb49)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This change introduces the .query property of the URI class. It is a
read/write dict of the parameters supplied in the query string of the
URI. E.g.:
http://example.com/?foo=bar => .query = {'foo': 'bar'}
(Bitbake rev: 1cb2b3c458c8c5521591d2c8f2e0058143fc77bb)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(Bitbake rev: 4d502578f022bcf772780550c047b8c09ba01443)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some of the previous cleanups broke the downloadfilename functionality.
This change fixes the code to ensure the commandline is correctly built.
Thanks Kristof Robot <krirobo@gmail.com> for reporting the issue.
(Bitbake rev: e008d9bb07e5d1a3584cc04ca2cd3dd906fd5759)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The svk fetcher never appears to be used by anyone and the development
on svk appears to have stopped in 2010. We might as well drop support
for it.
(Bitbake rev: 8239264753977bd06ad5b1b574245d3842af489b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These two functions have little in common, separate them out.
(Bitbake rev: 7413e0fa4dca9571ea98f32dab87d4fd60bc8de9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Start to clean up wget fetcher command construction to allow clearer
and more extensible code structure. Drops support for ${URI} and
${FILE} directly in the commands.
(Bitbake rev: 4e59fe45be2088996abc21e9a631a32b9a9642c9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
These variables were dropped from OE-Core some time ago, drop their
usage from the fetcher as well.
(Bitbake rev: bd33e709ab65d6966b234010641861834d170e2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Clean up some horrible old code and drop usage of the old style *COMMAND
variable and MKTEMPDIRCMD whilst in here. This means we don't need to touch
OVERRIDES either.
(Bitbake rev: c127bb3a9b7b1d2ab2c833ff73186b6ead0dc29c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
MKTEMPDIRCMD
Clean up some horrible old code and drop usage of the old style *COMMAND
variable and MKTEMPDIRCMD whilst in here. This means we don't need to touch
OVERRIDES either.
(Bitbake rev: 6b79789769da160d0e7fca0f9c6044dc1e11a107)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is other code which can want to run ls-remote style commands with
different parameters so split out the function.
(Bitbake rev: 13f1138f5504feee0ee8e8f3a0675d0bea490351)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Git versions older than 1.7.10 put absolute paths in configuration files
for the submodule repositories, leading to errors when the repository
checkout is moved. We move the repository as a matter of course in the
gitsm fetcher; the failure occurs in do_unpack). Change the absolute
paths to be relative during processing to fix this.
(At the time of writing, Ubuntu 12.04.4 LTS ships Git version 1.7.9.5,
hence the desire to fix this rather than just mandating a newer Git
version.)
Fixes [YOCTO #5525].
(Bitbake rev: e700d5a41deed4ee837465af526ed30c8a579933)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
A typo was meaning that the mirror creation method wasn't being called
when it should have been. Fix the type to fix mirror tarball creation.
[YOCTO #5284]
(Bitbake rev: 348971d410bfd5d8b1757468d73e1d24ae78a594)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
I can't think of a reason we'd download zero sized files however there are
reasons zero length files can accidently make it onto source mirrors.
This check allows us to ignore the broken files and switch to another
mirror rather than fail with odd checksum failures.
(Bitbake rev: 300cba2e1a720dba4b83b0c76208ea93c608c1de)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We need to deference tags when trying to map them to commit IDs with
ls-remote. If we don't do this, a given commit might not show up
later in a specific branch. There appears to be no good reason not
to do this.
(Bitbake rev: 8ef24f4c834298348172b96ec0b855bf09552b09)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When specifying tags, they're searched for unanchored so foo/bar could
match:
refs/heads/abc/foo/bar
refs/heads/xyz/foo/bar
refs/heads/foo/bar
This change anchors the expressions so they are based against heads
or tags (or any other base level tree that has been created).
(Bitbake rev: df2e0972cd1db7abd5ec8b7cb295fb0c42e284a4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The current message can be ambiguous, improve it (and also rename a
variable to clean up the rest of the function).
(Bitbake rev: 0c1bb7c0fce7b0f334311a2893ccb00385fa8d55)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a sanity check so that if some SRCREV is set and a rev parameter is given
to the url, the revision given should match.
Any tag parameter behaves the same as rev. If both are specified, error to
tell the user we're confused rather than do something which may or may not
be what they intended.
Also add some unittests for this.
(Bitbake rev: e82a4ab48991035866da9914c8b75a9bfbc9a7fc)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently INVALID and None are checked as incorrect values under different
circumstances. This code standardises those checks to be consistent. We
should phase out the use of "INVALID".
(Bitbake rev: 86ef4e65ce18b71dc69643586bd2aa8f48703171)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The fetcher will try:
1) PREMIRROR
2) Upstream
3) MIRROR
If it fails to download from the Upstream, but succeeds from the MIRROR,
and ud.localpath != origud.localpath (for example, the git tarball),
then we will get the error (e.g.: xf86-video-omapfb):
ERROR: Function failed: Fetcher failure for URL: 'xxx'. Unable to fetch URL from any source.
ERROR: Logfile of failure stored in: /path/to/log.do_fetch.28024
It should not show the error and let the build go on since it succeeds.
(e.g.: xf86-video-omapfb)
[YOCTO #5686]
(Bitbake rev: c08ca1e4eeb04f78e1354780cf5a4c3855e49572)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|