summaryrefslogtreecommitdiffstats
path: root/bitbake/lib/bb/fetch2/__init__.py
AgeCommit message (Collapse)Author
2015-07-09bitbake: fetch2: Add fetch parameter to checkstatusyocto-1.9_M1Aníbal Limón
In order to pass connection cache object to checkstatus function add fetch parameter. (Bitbake rev: fbb9c6f5538084e125b58118a86968908e6f895b) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-07-09bitbake: fetch2/__init__.py: Add FetchConnectionCache classAníbal Limón
FetchConnectionCache class acts as a container for socket connections useful when implement connection cache into fetcher modules. (Bitbake rev: 454da2cd17539ceb9caad6d76f034757e44ee12f) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-06-23bitbake: bitbake: Add explict getVar param for (non) expansionRichard Purdie
Rather than just use d.getVar(X), use the more explict d.getVar(X, False) since at some point in the future, having the default of expansion would be nice. This is the first step towards that. This patch was mostly made using the command: sed -e 's:\(getVar([^,()]*\)\s*):\1, False):g' -i `grep -ril getVar *` (Bitbake rev: 659ef95c9b8aced3c4ded81c48bcc0fbde4d429f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-24bitbake: fetch2/__init__.py: Make get_srcrev output configurableMike Looijmans
The idea here is to support package version numbering similar to gitpkgv in meta-openembedded. This commit is the first step towards such functionality. The original plan was to add a "get_pretty_srcrev" method to the fetcher, as per Richard's suggestion [1]. While writing this, I noticed that it would become a copy of get_srcrev with only two lines changed. So to create something more Pythonic than a boolean argument and conditionals around the calls to the fetcher's sortable_revision, I just made the method to be called on the fetcher an argument to the method. Defaulting to 'sortable_revision' prevents affecting existing code. Now if the git fetcher were to implement, say 'gitpkgv_revision' one could set the following in a recipe: PKGV="1.2+${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}" and this would yield the same result as gitpkgv's GITPKGV variable. See for the discussion leading to this change: [1] http://lists.openembedded.org/pipermail/openembedded-core/2015-January/100345.html (Bitbake rev: 2f1f4483493cc290f5d2c07f9906e90eaea2f4c1) Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16bitbake: fetch2: Improve mirror building function error handlingRichard Purdie
First, when building mirror urls we don't do any fetching so we should never be calling clean functions. Currently, if a mirror url fails, we don't process it further to see if there are any mirrors of the mirror. We should do this even when the mirror url fails, else we may miss out on valid/useful mappings, particularly in the case of file:// urls. (Bitbake rev: b7fd3ec9994f664b17fc86423e6e7afac07e897b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-16bitbake: fetch2: Allow GIT_SMART_HTTP to be passed throughLiam R. Howlett
Allow GIT_SMART_HTTP to be passed through to the fetch command so that servers that cannot use GIT_SMART_HTTP can be used by the fetcher. (Bitbake rev: e5c97a85bed0436d48eeaac2e32962cfb5371d2f) Signed-off-by: Liam R. Howlett <Liam.Howlett@WindRiver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12bitbake: fetch2: Add BB_ALLOWED_NETWORKS supportLiam R. Howlett
BB_ALLOWED_NETWORKS is a list of hosts that the fetcher will be allowed to use when BB_NO_NETWORK is not set. If BB_NO_NETWORK is set, then networking is still disabled. If BB_ALLOWED_NETWORKS is not set, the behaviour remains the same as today. If BB_NO_NETWORK is NOT set, and BB_ALLOWED_NETWORKS is configured, then only the hosts in the list are usable by the fetcher. eg: BB_ALLOWED_NETWORKS="yoctoproject.org git.gnu.org" The fetcher will be able to download from yoctoproject.org, git.gnu.org, but not ftp.gnu.org or any other hostname that is not in the list. There is also limited support for wildcards on the beginning of the hosts, so BB_ALLOWED_NETWORKS="*.gnu.org" with match git.gnu.org and ftp.gnu.org as well as foo.git.gnu.org (Bitbake rev: c7263096ba31ba45daeeb9de90c1cb9ebef24a28) Signed-off-by: Liam R. Howlett <Liam.Howlett@WindRiver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-05-12bitbake: fetch2: Allow git to use a different CA Bundle with private keysJason Wessel
Setting BB_ENV_EXTRAWHITE to include GIT_SSL_CAINFO is enough to get "git ls-remote" to work for recipe parsing but it is not enough to get the fetcher to clone properly. This is because the fetcher has its own idea about what variables should be exported in the environment for some operations. It is desirable to use alternate CA Bundles for internal testing prior to using public keys for https, so we should allow the GIT_SSL_CAINFO to pass through. (Bitbake rev: 40ff92282bbf32cf644b021bf7cbb1f393dbb856) Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-03-09bitbake: fetch2: Revalidate checksums, YOCTO #5571Clemens Lang
[YOCTO #5571] -- https://bugzilla.yoctoproject.org/show_bug.cgi?id=5571 The following workflow (whether accidentally or deliberately) would previously not result in a checksum error, but would be helpful to do so: - Write a recipe with correct checksums - Fetch the sources for this recipe using bitbake - Change the checksums Since the bitbake fetcher writes a done stamp after the initial download and does not verify the checksums again (even if they are changed in the recipe afterwards), the change of checksums is silently ignored. Fix this without the overhead of computing the checksums from scratch on every do_fetch by storing them in pickled format in the done stamp and verifying that they still match those in the recipe. (Bitbake rev: fbd4a0d422cf7f43db2f9eab2e47c41246a9031e) Signed-off-by: Clemens Lang <clemens.lang@bmw-carit.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-21bitbake: fetcher: use os.path.splitext() instead of string.split()Ross Burton
(Bitbake rev: 1689edd35477a126bfbf4bd3257c4f1f9e16f20d) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-21bitbake: lib/fetch2: Tweak get_srcrev() function descriptionRichard Purdie
Tweak the comments on get_srcrev() to better describe its function. (Bitbake rev: b4d40f1ac7b32990c456cce261f99a5a157b5ae5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-01-08bitbake: cache/fetch2/siggen: Ensure we track include history for file checksumsRichard Purdie
Currently, if you reference a file url, its checksum is included in the task hash, however if you change to a different file at a different location, perhaps taking advantage of the FILESPATH functionality, the system will not reparse the file in question and change its checksum to match the new file. To correctly handle this, the system not only needs to know if the existing file still exists or not, but also check the existance of every file it would have looked at when computing the original file. We already do this in the bitbake parsing code for class inclusion. This change uses the same technique to log the file list we looked at and if files in these locations exist when they previously did not, to invalidate and reparse the file. Since data stored in the cache is flattened text, we have to use a string form of the data and split on the ":" character which is ugly, but is an internal detail we can improve later if a better method is found. The cache version changes to trigger a reparse since the previous cache data is now incompatible. [YOCTO #7019] (Bitbake rev: 6c0706a28d72c591f1b75b6e3f3b645859387c7e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-12-09bitbake: fetch2: add means of disabling SRC_URI checksumsPaul Eggleton
If we're fetching outside of the context of a recipe, it's handy to be able to disable checksum functionality so you don't get a meaningless warning about the signatures being missing. (Bitbake rev: 49dbcfbc56a206964acc5de761bba31be0283ba1) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-28bitbake: fetch2: add .lz compression supportKoen Kooi
See http://www.nongnu.org/lzip/lzip.html for details on the compression format. (Bitbake rev: 9027b1273b5405c7269b013604ab417771b5eafe) Signed-off-by: Koen Kooi <koen@dominion.thruhere.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-11bitbake: fetcher: fix getVar call due to incorrect argument datatypeAlejandro Hernandez
(Bitbake rev: 2ac33aac3446cb12227f1b8daa5f27f417c9bb9e) Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-10-06bitbake: fetcher: fix BB_STRICT_CHECKSUM datatype checkAlejandro Hernandez
Forcing strict to be a string, to avoid problems when performing comparisons [YOCTO #6762] (Bitbake rev: b8ed2098bdea2afd93ab4e3e1b834f3a31cb60de) Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-29bitbake: fetch: Extend testing of subdir unpack parameter and fixRichard Purdie
This fixes urls of the form file://some/path/file;subdir=b. It also adds in a couple of tests so we now tests these corner cases. (Bitbake rev: 46306912a96444790efa9418d934dfdd36773ba1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-09-29bitbake: fetch: SRC_URI parameter "subdir" does not work for local filesRoxana
Check if the 'subdir' parameter exists and assign it to 'destdir' so that files are copied in ${WORKDIR}/destdir. This fixes urls that are of the form file://a;subdir=b. (Bitbake rev: 836a986b365eb9798563ec08d90b346596de7791) Signed-off-by: Roxana Ciobanu <roxana.ciobanu@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-07-21bitbake: fetch2: Add module for ClearCase (ccrc://)Dennis Meier
The clearcase fetcher is used to retrieve files from a ClearCase (http://en.wikipedia.org/wiki/Rational_ClearCase) repository. Usage in the recipe: SRC_URI = "ccrc://cc.example.org/ccrc;vob=/example_vob;module=/example_module" SRCREV = "EXAMPLE_CLEARCASE_TAG" PV = "${@d.getVar("SRCREV").replace("/", "+")}" The fetcher uses the rcleartool or cleartool remote client, depending on which one is available. Supported SRC_URI options are: - vob (required) The name of the clearcase VOB (with prepending "/") - module The module in the selected VOB (with prepending "/") The module and vob parameters are combined to create the following load rule in the view config spec: load <vob><module> - proto http or https Related variables: CCASE_CUSTOM_CONFIG_SPEC Write a config spec to this variable in your recipe to use it instead of the default config spec generated by this fetcher. Please note that the SRCREV loses its functionality if you specify this variable. SRCREV is still used to label the archive after a fetch, but it doesn't define what's fetched. User credentials: When using cleartool: The login of cleartool is handled by the system. No special steps needed. When using rcleartool: In order to use rcleartool with authenticated users an `rcleartool login` is necessary before using the fetcher. (Bitbake rev: 76cff0aaad4cf10b325bceadd8ea90b3d75edcc2) Signed-off-by: Dennis Meier <meier.dennis@siemens.com> Reviewed-by: Roger Meier <r.meier@siemens.com> Reviewed-by: Christian Liechti <christian.liechti@siemens.com> Reviewed-by: Henrique Mendonca <henrique.mendonca@siemens.com> Reviewed-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-30bitbake: fetch2: improve handling of two classes of URL parameter mistakesPaul Eggleton
Handle the following situations in a URL (e.g. in SRC_URI): * Trailing semicolon in a URL - this is now ignored. * Parameter specified with no value (no equals sign). This still produces an error, but at least it is MalformedUrl with a proper message rather than "ValueError: need more than 1 value to unpack". (Bitbake rev: bfd13dfbc4c9f1dd8315002271791b1d9e274989) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-05-08bitbake: Fix refetch of local files when they are read onlyNick Lewis
(Bitbake rev: 42b7be1a6c5ff559f4a213eed04b370ca955460d) Signed-off-by: Nick Lewis <nick.lewis@usa.g4s.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-29bitbake: fetch2: Cleanup file checksum generation, v2Jacob Kroon
The initial version of this patch was reverted in 0a94e568152de550dedc8135a766beb18bf064ab However, it was later agreed upon that it was reverted due to a misunderstanding during code review; the patch is ok. This version of the patch also removes an unneeded initialization of the 'checksum' variable outside the scopes it is being used in. (Bitbake rev: 9ee19a3ca2f8e11a91f0289ea3486310c61d40f2) Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-28bitbake: Revert "fetch2: Cleanup file checksum generation"Richard Purdie
This reverts commit 6dfb33fd7204e09815bde7ba5bf21941a73fe965. It causes an unindeded change in behaviour. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-27bitbake: fetch2: Cleanup file checksum generationJacob Kroon
Cleanup the fix done in f9416e76e272ba3249abff099f6f3a47fe82e03e. Instead of adding continue statements we can just move the last statements into the final else-clause. (Bitbake rev: 6dfb33fd7204e09815bde7ba5bf21941a73fe965) Signed-off-by: Jacob Kroon <jacob.kroon@mikrodidakt.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08bitbake: fetch2: fix traceback when a wildcard matches a directoryPaul Eggleton
If there is a directory matching a wildcard in SRC_URI when getting file checksums, we should recurse into that instead of producing an error. (Bitbake rev: ae87b7eb414e3d5eefd2effec7b30c22d2186b02) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08bitbake: fetch2: handle wildcards correctly when recording file checksumsPaul Eggleton
The Local fetcher's localpath is returning the parent directory for a wildcard match; we need to handle this and add the wildcard specification so that we checksum the correct files. Fixes [YOCTO #6127]. (Bitbake rev: 4a90edd62c16cbf41b5b93280e155077564c774a) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-08bitbake: fetch2: avoid cache ignoring missing filesPaul Eggleton
Previously, if a file listed in SRC_URI was missing at the time of parsing, and then was added and bitbake run again, it would not be picked up because the recipe was cached without it. If we allow the file to be added to the list of files to checksum, then it will be checked for and found on the second run. Fixes [YOCTO #4790]. (Bitbake rev: 71da822762cb298261cccdfa54b9c0fea02c3c5d) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-04-06bitbake: fetch2: Fix bug in file checksum generationRichard Purdie
For a while its been puzzling me why connman-gnome rebuilds as often as it does. It turns out you can trigger this with a new checkout of the metadata. The SRC_URI that is causing the problems is: SRC_URI = "file://images/*" and rather oddly the results in checksums for a file "." being added to the tree, e.g.: ('.', 'ab48a68186f0e0f277c21ef4cb390b4b') The problem is that when iterating files lists, the checksum variable can become set yet we don't break the out from the for loop, which leads to odd (and non-deterministic) entries being added into the file checksum list. The exact item added probably depends on the order of items on the disk. Before this change, bitbake-diffsigs on connman-gnome:do_fetch would report: This task depends on the checksums of files: [ ('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'), ('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'), ('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'), ('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('.', 'ab48a68186f0e0f277c21ef4cb390b4b')] Afterwards: This task depends on the checksums of files: [ ('connman-signal-03.png', 'f6c16aee57b37b73793a2f1dea433ffa'), ('connman-signal-02.png', 'ad0cd22710c097d8174121fc1023c3be'), ('connman-signal-01.png', '8842bd83d2fa9ba56480df34c727c629'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('connman-signal-05.png', '808589e7e8d502b44c7b007e9e68d48c'), ('connman-signal-04.png', 'ab48a68186f0e0f277c21ef4cb390b4b'), ('null_check_for_ipv4_config.patch', 'a23271e41c9fe81551244d875106af10'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85'), ('0001-Removed-icon-from-connman-gnome-about-applet.patch', 'e2d8269357c1e8c84291606da24eea85')] which is correct and deterministic without the "." entry. (Bitbake rev: f9416e76e272ba3249abff099f6f3a47fe82e03e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-24bitbake: fetch2.URI: Set username/password should not change the otherOlof Johansson
When setting the username after already having set the password, the password was unexpectedly reset. This change fixes this issue and introduces unit tests to make sure it doesn't happen again. (Bitbake rev: 25faef3a047f9c7564089463d7c96f6910b640cb) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-24bitbake: fetch2.URI: Support URIs with both query strings and paramsOlof Johansson
There is a case in meta-intel where a SRC_URI contains both a query string and URI parameter: https://edc.intel.com/Download.aspx?id=6190;downloadfilename=LIN_IEMGD_1_14_GOLD_2443.tgz Python's urlparse thought the URI parameters were part of the query parameter value, but in the bitbake context this is obviously not the case. As bitbake's usage isn't really RFC compliant, we have to extract and remove the URI parameters *before* urlparse sees the URI. (Bitbake rev: c2f27ae4271985b48f957747b6ea372c8699cb49) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-24bitbake: fetch2.URI: add support for query parametersOlof Johansson
This change introduces the .query property of the URI class. It is a read/write dict of the parameters supplied in the query string of the URI. E.g.: http://example.com/?foo=bar => .query = {'foo': 'bar'} (Bitbake rev: 1cb2b3c458c8c5521591d2c8f2e0058143fc77bb) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-24bitbake: fetch2.URI: Coerce urlparse to use netloc for all schemesOlof Johansson
(Bitbake rev: 4d502578f022bcf772780550c047b8c09ba01443) Signed-off-by: Olof Johansson <olof.johansson@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-03-02bitbake: fetch2: Drop svk fetcherRichard Purdie
The svk fetcher never appears to be used by anyone and the development on svk appears to have stopped in 2010. We might as well drop support for it. (Bitbake rev: 8239264753977bd06ad5b1b574245d3842af489b) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-02-24bitbake: fetch2: Fix mirror repo tarball creationRichard Purdie
A typo was meaning that the mirror creation method wasn't being called when it should have been. Fix the type to fix mirror tarball creation. [YOCTO #5284] (Bitbake rev: 348971d410bfd5d8b1757468d73e1d24ae78a594) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21bitbake: fetch2: Improve invalid SRCREV error messageRichard Purdie
The current message can be ambiguous, improve it (and also rename a variable to clean up the rest of the function). (Bitbake rev: 0c1bb7c0fce7b0f334311a2893ccb00385fa8d55) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21bitbake: fetch2: Sanity check SRCREV matches rev/tag parameterRichard Purdie
Add a sanity check so that if some SRCREV is set and a rev parameter is given to the url, the revision given should match. Any tag parameter behaves the same as rev. If both are specified, error to tell the user we're confused rather than do something which may or may not be what they intended. Also add some unittests for this. (Bitbake rev: e82a4ab48991035866da9914c8b75a9bfbc9a7fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-21bitbake: fetch2: Clean up srcrev_internal_helperRichard Purdie
Currently INVALID and None are checked as incorrect values under different circumstances. This code standardises those checks to be consistent. We should phase out the use of "INVALID". (Bitbake rev: 86ef4e65ce18b71dc69643586bd2aa8f48703171) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-19bitbake: fetch2/__init__.py: let try_mirror_url return correct valueRobert Yang
The fetcher will try: 1) PREMIRROR 2) Upstream 3) MIRROR If it fails to download from the Upstream, but succeeds from the MIRROR, and ud.localpath != origud.localpath (for example, the git tarball), then we will get the error (e.g.: xf86-video-omapfb): ERROR: Function failed: Fetcher failure for URL: 'xxx'. Unable to fetch URL from any source. ERROR: Logfile of failure stored in: /path/to/log.do_fetch.28024 It should not show the error and let the build go on since it succeeds. (e.g.: xf86-video-omapfb) [YOCTO #5686] (Bitbake rev: c08ca1e4eeb04f78e1354780cf5a4c3855e49572) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-18bitbake: fetch2: Don't allow '/' in user:pass, fix branch containing '@'Martin Jansa
* currently decode_url regexp parses branch=@foo as username so it ends like this: - ('git', '', 'foo', 'git.openembedded.org/bitbake;branch=', '', {}) + ('git', 'git.openembedded.org', '/bitbake', '', '', {'branch': '@foo'}) * http://hg.python.org/cpython/file/2.7/Lib/urlparse.py also assumes that there is at least one '/' as separator between netloc and path, params, so it looks reasonable to prevent including '/' in username (Bitbake rev: 2c82742114091cb55055328b54223686816582f2) Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-08bitbake: bitbake: gitannex.py: Add Git Annex supportOtavio Salvador
This add a Git Annex backend which reuses the Git fetcher code; it allows managing files with git, without checking the file contents into git, being useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, time, or disk space. (Bitbake rev: a61fc4db598e9d13c966712a6a0e4783e19448be) Signed-off-by: Otavio Salvador <otavio@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02bitbake: bitbake: fetcher2: clean(): remove the .patch.doneRobert Yang
There was a problem: $ bitbake xf86-video-omapfb -cfetch && bitbake xf86-video-omapfb -ccleanall Everything should be removed, but the 0006-omapfb-port-to-new-xserver-video-API.patch.done still exists in the DL_DIR, this is because the clean() in the fetch2/__init__.py skips removing the local file, so that it will skip removing the .done. The local file (file://) isn't needed to be removed since it is not downloaded into DL_DIR, but the .done should be removed, this patch will remove the .done, and it doesn't remove anything else since the clean() in local.py does nothing. [YOCTO #5687] (Bitbake rev: 2bc99b9dfa532430a13c39fca4e5ef3a2206b3b8) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2014-01-02bitbake: bitbake: replace 3 spaces with 4Robert Yang
A part of fetch2/__init__.py uses 3 spaces as the indent, I think that they should be typos. (Bitbake rev: abafd85e2fcf23cee872e0e9e468898101430f1f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-22bitbake: fetch2: avoid printing "no checksum" error message twicePaul Eggleton
Because of the way we were handling this error, it was printed twice - once via logger.error() (to avoid the log being printed) and a second time when the exception gets wrapped in a FuncFailed at a higher level. Call logger.error() earlier and change the text we send in the exception to be more brief, so it more closely resembles the behaviour when there is an invalid checksum. (Bitbake rev: 46765369d7f76ec7f67b90430131a79eb6a66235) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-22bitbake: fetch2: fail checksum validation if SRC_URI checksums set to ""Paul Eggleton
We were checking SRC_URI md5sum/sha256sum values against None here, so if they were set to "" then no error was produced. Since the value is still effectively unset in this case, this is not the right behaviour; just check if the value doesn't evaluate to False instead. (Bitbake rev: 040943a718795c64dc4e604abfcf08b26b7d00e6) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-22bitbake: fetch2: Stop passing around the pointless url parameterRichard Purdie
There is no good reason to keep passing around the url parameter when its contained within urldata (ud). This is left around due to legacy reasons, some functions take it, some don't and its time to cleanup. This is fetcher internal API, there are a tiny number of external users of the internal API (buildhistory and distrodata) which can be fixed up after this change. (Bitbake rev: 6a48474de9505a3700863f31839a7c53c5e18a8d) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-22bitbake: fetch2: Clean up some url parameters to functionsRichard Purdie
urldata contains the url so we might as well stop passing around pointless function parameters. This was done for legacy reasons but its time to clean this mess up. This is a first step in cleanup and is a standalone patch but there is more to be done in a second patch. (Bitbake rev: 06590cfebbcf6565a17b80cc298e3ecdfaba4656) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-19bitbake: fetch2: Fix handling of SCM mirrors in MIRRORSRichard Purdie
If an SCM mirror is in PREMIRRORS, the tarball is downloaded and then found by the "upstream" check and handled correctly. If an SCM mirror is in MIRRORS, the tarball is downloaded but not used since there is no "upstream" run after MIRRORS completes. It therefore sits there useless and unused. This code change forces the upstream to run after a mirror tarball is found and fixes the usage of SCM mirrors in MIRRORS. (Bitbake rev: a66ee0994645aa5658b2f5ea134ed17d89f8751a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-07-31bitbake: fetch2: fix unpack of .xz filesAndré Draszik
If a file ends with .xz, it currently gets overwritten during unpack: The decompress command for .xz files is: 'xz -dc %s > %s' % (file, efile) and as efile == file, we end up overwriting file (the source). Fix this by adding .xz to the list of suffixes that that need to be removed from a file name for an extract command, leaving the bare file name. Now, for a given file foo.xz, file == foo.xz and efile == foo, similar to how .gz .bz2 and .Z files are treated. (Bitbake rev: 2cd2d0a48e12ab4358fb967eaf7a56c17993f48d) Signed-off-by: André Draszik <andre.draszik@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20bitbake: Correct a typoPeter Kjellerstedt
(Bitbake rev: 087ec3a1b2dde04a8e5ce2943802fccd424ab3f3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-06-20bitbake: Allow checksums to be used for files retrieved using SFTPPeter Kjellerstedt
(Bitbake rev: fd438a334d1f90ff07dded61c9648987da42c34f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>