summaryrefslogtreecommitdiffstats
path: root/bitbake
AgeCommit message (Collapse)Author
2021-08-18bitbake: contrib: vim: Add "remove" override highlightingJoshua Watt
"remove" was accidentally omitted when defining which override operators should be highlighted (Bitbake rev: a0248338452f9ec26b588ef83679aca6263e7e76) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-18bitbake: bitbake: Make 3.6.0 the minimum python versionRichard Purdie
OE-Core did this a while ago, it is simpler if bitbake matches. (Bitbake rev: a3050aee21b6a23b55232d52f89980a3bbd3a290) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12bitbake: fetch2/wget: fetch securely by defaultRoss Burton
The days of broken certificates are behind us now, so instead of always passing --no-check-certificate to wget, don't pass it by default and instead only pass it BB_CHECK_SSL_CERTS = "0". [ YOCTO #14108 ] (Bitbake rev: 4104850dd36096a9ff01836c5fca9ac0e452bcf8) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12bitbake: fetch2/wget: ensure all variables are set when calling urllibRoss Burton
Instead of just exporting the proxy variables when calling into urllib, use bb.utils.environment() to export all of the known variables that are needed for proper connectivity. Specifically, this ensures that SSL_CERT_FILE is set, so that libssl can find the certificates in buildtools environments (Bitbake rev: 116637b0e9aabae7f680b102dbf3577b8a58f049) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12bitbake: fetch2: expose environment variable names that need to be exportedRoss Burton
There is a list of environment variable names that need to be exported when executing external commands, such as 'http_proxy'. To avoid duplication, make this a top-level variable. Also add SSL_CERT_FILE, which is used by OpenSSL to locate the certificate bundle. This is needed in buildtools environments where the default path isn't valid. (Bitbake rev: 5c2cf57fd11d91f749a3b562f6f0a863f15013ed) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12bitbake: utils: add environment updating context managerRoss Burton
bb.utils.environment() is a context manager to alter os.environ inside a specific block, restoring it after the block is closed. (Bitbake rev: 9974848f67581ff7d76cef52a94f505af99b4932) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06bitbake: ui/taskexp: Fix to work with empty build directoriesRichard Purdie
If run on an empty build directory, taskexp wasn't working as it didn't send the current environment to the server. This means HOSTTOOLS in oe-core couldn't be built and gave an error. Add the missing updateToServer call in. [YOCTO #14408] (Bitbake rev: 06a0bbe746f879ae539223e7fdb6f07d55d13719) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06bitbake: ui/taskexp: Improve startup exception handlingRichard Purdie
When an exception occurs at startup, show it to the user. [YOCTO #14408] (Bitbake rev: cc1df1af67cfd3e223b39e2b7ea5f86b8cf78aee) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06bitbake: command: Ensure we catch/handle exceptionsRichard Purdie
If an exception occurs in early setup, bitbake could just hang. Return the exception rather than doing that. [YOCTO #14408] (Bitbake rev: c8a4107132ce51f84ae84bf1ceb1c3fd90f156d3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06bitbake: process: Improve traceback error reporting from main loopRichard Purdie
Currently the code can just show nothing as the exception if there was a double fault, which in this code path is quite likely. This leads to an error log which effectively says "it failed" with no information about how. Improve things so we get a nice verbose traceback left in the logs/output which is preferable to no logs. (Bitbake rev: e5782b71647d1eb6de53bde7bc4f6019a5589f21) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06bitbake: fetch/tests/toaster: Override conversion fixupsRichard Purdie
Fix some references that missed during the overrides syntax migration or were incorrect. Thanks to Quentin Schulz <foss@0leil.net> for the patch. (Bitbake rev: 6184cb07dfa44f5f76f1c423533b4547d80b20ab) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-06bitbake: doc: Fix append/prepend/remove referencesRichard Purdie
Fix some references missed during the overrides syntax migration. Thanks to Quentin Schulz <foss@0leil.net> for the patch. (Bitbake rev: 2fd03ec7b136c694f2ced43b3abb69f719c99ec2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-05bitbake: runqueue: Improve multiconfig deferred task issuesRichard Purdie
The previous patches have exposed new issues with this code path, the issues being around what should happen when the hash of a task changes and the task is or is not on the deferred task list. Rather than rebuilding the deferred task list during each rehash event, build it once at the start of a build. This avoids the problem of tasks being added back after they have run and also avoids problems of always ensuring the same task is deferred. It also allows the 'outrightfail' codepath to be handled separately as the conditions are subtly differnt. One significant win for the new approch is the build is not continually printing out lists of deferred tasks, that list remains fairly static from the start of the build. Logic is added in to ensure a rehashed task with a hash matching other deferred tasks is deferred along with them as a small optimization. An interesting test case for this code was reported by Mark Hatle with four multiconfigs, each the same apart from TMPDIR and running a build of: bitbake buildtools-tarball mc:{one,two,three,four}:core-image-minimal which is interesting in that the build of buildtools partially overlaps core-image-minimal and the build has a rehash event for qemuwrapper-cross even without any external hash equivalence server or preexisting data. (Bitbake rev: bb424e0a6d274d398f434f7df63951da9ce305b3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04bitbake: contrib: vim: Update for new override syntaxJoshua Watt
Updates the Vim syntax highlighting to account for the new override syntax and also highlight "append" and "prepend" overrides (Bitbake rev: 01a6322315a6ff6ab55a349f9fcd1e2d93448bfd) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-04bitbake: data_smart: Fix inactive overide accidental variable value corruptionRichard Purdie
Setting something like: BAR:append:unusedoverride should cause BAR to be None, not "" which was what the datastore was returning. This caused problems when mixing variables like: RDEPENDS:${PN}:inactiveoverride RDEPENDS:${BPN} since key expansion would report key overlap when there was none. This is a bug in the datastore. Fix it and add a test too. [YOCTO #14088] (Bitbake rev: 699e36c270d863258502d315ed00a1b940bfbf96) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02bitbake: doc: bitbake-user-manual: grammar fix for the number of "metadata"Michael Opdenacker
"metadata" is used both as singular and as plural. This fixes a case in which the verb has a singular conjugation, which conflicts with the absence of article indicating a plural case. (Bitbake rev: fff7ade48d6cb9381284b93742bb2255976d6b41) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Reviewed-by: Quentin Schulz <foss@0leil.net> Reviewed-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02bitbake: doc: bitbake-user-manual: update bitbake option helpMichael Opdenacker
(Bitbake rev: 214d11867ea5dd9f1d7e50e128d45d7cc4eaf7ea) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02bitbake: doc: bitbake-user-manual: fix syntax in example and improve descriptionMichael Opdenacker
The previous syntax of the example was clearly incorrect The example and description were also ambiguous, one could think that it was "bitbake recipe" instead of "bitbake <recipename>" (Bitbake rev: 9d6664bbb68fac9bb4fbcbe0b35beb0a022a5a1f) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02bitbake: bitbake: Update to version 1.51.1Richard Purdie
(Bitbake rev: ca88466f6d244042b12b66ccd69e27ca2f057d17) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02bitbake: doc/lib: Add fixes for issues missed by the automated conversionRichard Purdie
The examples and tests use non-standard override names, convert these to the new syntax by hand. (Bitbake rev: a6c40eca1146c0160da7e4e0bd7ac52fef2029e0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02bitbake: doc/lib: Update to use new override syntax containing colonsRichard Purdie
This runs the overrides conversion script in OE-Core over the bitbake code base including the docs. A handful of things were excluded in toaster and for the Changelog file. (Bitbake rev: 47f8d3b24fd52381bf3b41e2f55a53e57841344c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-02bitbake: bitbake: Switch to using new override syntaxRichard Purdie
This change updates the datastore to use the new override syntax using colons instead of underscores exclusively. It is expected layers would have to be converted to work with bitbake after this change. Supporting mixed syntax isn't possible, it is only feasible to have one internal representation of overrides. Whilst we can't warn for every possible override that may be set in the old format, show errors for _append/_prepend/_remove since those should never be present. (Bitbake rev: 7dcf317cc141dc980634f8c18bfa84f83e57206a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-29bitbake: data_smart: Allow colon in variable expansion regexRichard Purdie
Now that ":" is a valid character in variable key names, it needs to be allowed by the variable expansion code too, to match. (Bitbake rev: 019251649a38754d5877759d13b664e28dea77de) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-29bitbake: parse/ast: Substitute '~' when naming anonymous functionsPaul Barker
When parsing an anonymous python function, bitbake generates a name for the function based on the full path to the file in which it was found. As not all characters which are valid in file paths are valid in Python function names we have a translation table. However, this translation table was missing an entry for '~'. (Bitbake rev: b201c0b284e25c20685d9d206797c4cc650eeca0) Signed-off-by: Paul Barker <paul@pbarker.dev> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-29bitbake: bitbake: asyncrpc: Catch early SIGTERMJoshua Watt
If the SIGTERM signal is sent to an asyncrpc server before it has installed the SIGTERM handler in the main loop, it may miss the signal which will can cause the calling process to wait forever on the join(). To resolve this, the calling process should mask of SIGTERM before forking the server process and the server should unmask the signal only after the handler is installed. To simplify the usage of the server, an new helper function called serve_as_process() is added to do this automatically and correctly. Thanks: Scott Murray <scott.murray@konsulko.com> for helping debug (Bitbake rev: ef2865efa98ad20823267364f2159d8d8c931400) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-29bitbake: asyncrpc: Set timeout when waiting for reply from serverPaul Barker
(Bitbake rev: b31f1853d7fcb8b8f8885ca513a0021a5d0301e6) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-29bitbake: asyncrpc: Fix bad message error in clientPaul Barker
If there is an issue with the format of the reply given by the server then we should print this reply line in the error message. Printing the message which the client sent doesn't illuminate anything here. (Bitbake rev: bd8f8d7b055da15cd7bdd0b383061852a0f54cb7) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-26bitbake: doc: bitbake-user-manual: replace ``FOO`` by :term:`FOO` where possibleQuentin Schulz
If a variable has a glossary entry and some rST files write about those variables, it's better to point to the glossary entry instead of just highlighting it by surrounding it with two tick quotes. The script that is used to do the replacement of ``FOO`` by :term:`FOO` is the following Python code: import re from pathlib import Path from runpy import run_module import contextlib import io import sys re_term = re.compile(r'variables.html#term-([a-zA-Z_0-9]*)') terms = [] new_terms = set() with contextlib.redirect_stdout(io.StringIO()) as f: run_module('sphinx.ext.intersphinx', run_name='__main__') objects = f.getvalue() match = re_term.search(objects) while match: if match.group(1): terms.append(match.group(1)) match = re_term.search(objects, match.end()) for rst in Path('.').rglob('*.rst'): with open(rst, 'r') as f: content = "".join(f.readlines()) for term in terms: content = re.sub(r'``({})``(?!.*\s+[~=-]{{{:d},}})'.format(term, len(term)), r':term:`\1`', content) with open(rst, 'w') as f: f.write(content) This script takes one argument as input: an objects.inv which can be gotten from doc/_build/html/objetcs.inv after running `make html`. Note that this excludes from replacement terms that appear in section titles as it requires refs to be changed too. This can be automated too if need be but right now it looks a bit confusing to have an anchor link (for sections) also have a term/reference link in it. I am not sure this is desired today. (Bitbake rev: aba88f40c47133ed9bc999e0298aca3bc8490912) Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com> Signed-off-by: Quentin Schulz <foss@0leil.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-26bitbake: doc: bitbake-user-manual: ref-variables: force glossary output to ↵Quentin Schulz
be alphabetically sorted Even though, care should be taken to have the terms in the glossary sources alphabetically ordered, it is possible some terms might be in the wrong place. This makes sure that whatever the order of terms in the glossary sources, the generated medium is correctly sorted. (Bitbake rev: 98809ebc6ad51f0a94cedccfaff9c11d3744dc0d) Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com> Signed-off-by: Quentin Schulz <foss@0leil.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-26bitbake: doc: bitbake-user-manual: ref-variables: order alphabetically the ↵Quentin Schulz
glossary sources This reorders a few entry so that they are alphabetically sorted. (Bitbake rev: 311350ed5cb164d975c2119e60255a409e27dffb) Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com> Signed-off-by: Quentin Schulz <foss@0leil.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-26bitbake: doc: Makefile: turn warnings into errors by defaultQuentin Schulz
-W is for turning warnings into errors, --keep-going to make sure the whole documentation is built and all warnings will be printed before failing. Since there's currently no warning, it's time to introduce it! (Bitbake rev: 6b0b3d707f662ca7b1d0de99fe032e4f35867d10) Signed-off-by: Quentin Schulz <quentin.schulz@theobroma-systems.com> Signed-off-by: Quentin Schulz <foss@0leil.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-20bitbake: server: Fix early parsing errors preventing zombie bitbakeJoshua Watt
If the client process never sends cooker data, the server timeout will be 0.0, not None. This will prevent the server from exiting, as it is waiting for a new client. In particular, the client will disconnect with a bad "INHERIT" line, such as: INHERIT += "this-class-does-not-exist" Instead of checking explicitly for None, check for a false value, which means either 0.0 or None. (Bitbake rev: 13e2855bff6a6ead6dbd33c5be4b988aafcd4afa) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-20bitbake: bitbake: Add piping compression libraryJoshua Watt
Adds a library that implements file-like objects (similar to gzip.GzipFile) that can stream to arbitrary compression programs. This is utilized to implement a LZ4 and zstd compression API. (Bitbake rev: 61c3acd058ea018696bd284b3922d0b458838d05) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-20bitbake: data_smart/parse: Allow ':' characters in variable/function namesRichard Purdie
It is becomming increasingly clear we need to find a way to show what is/is not an override in our syntax. We need to do this in a way which is clear to users, readable and in a way we can transition to. The most effective way I've found to this is to use the ":" charater to directly replace "_" where an override is being specified. This includes "append", "prepend" and "remove" which are effectively special override directives. This patch simply adds the character to the parser so bitbake accepts the value but maps it back to "_" internally so there is no behaviour change. This change is simple enough it could potentially be backported to older version of bitbake meaning layers using the new syntax/markup could work with older releases. Even if other no other changes are accepted at this time and we don't backport, it does set us on a path where at some point in future we could require a more explict syntax. I've tested this patch by converting oe-core/meta-yocto to the new syntax for overrides (9000+ changes) and then seeing that builds continue to work with this patch. (Bitbake rev: 0dbbb4547cb2570d2ce607e9a53459df3c0ac284) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-11bitbake: fetch2/s3: allow to use credentials from environment variablesAdam Romanek
Previously access to AWS S3 was expected to be preconfigured and credentials to be stored in ~/.aws/credentials. With this change one can use Bitbake s3 fetcher without AWS credentials stored permanently as above, just with them exported as the following environment variables: - AWS_ACCESS_KEY_ID, - AWS_SECRET_ACCESS_KEY. - AWS_DEFAULT_REGION. (Bitbake rev: 01825699044c42e87e485e6c64cc1dd9b6f87f48) Signed-off-by: Adam Romanek <romanek.adam@gmail.com> Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-10bitbake: providers: replace newly added logger.warn() with logger.warning()Denys Dmytriyenko
Commits https://git.openembedded.org/bitbake/commit/?id=78cd63285713fde59506eb2e71a7b7ee59a594ff and https://git.openembedded.org/bitbake/commit/?id=5cbf6d95fc1009e78e7d0745a49e0bf418b37abb added few calls to logger.warn(), which is deprecated and instead should use logger.warning(): https://lists.openembedded.org/g/bitbake-devel/topic/82742194#12377 (Bitbake rev: a28ba2d31cd3aa557d4977e9376c5d01cd863e9a) Signed-off-by: Denys Dmytriyenko <denis@denix.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-07-10bitbake: fetch/git: run gc in foreground to avoid race with tarAdam Romanek
It looks like git gc can interrupts the package creation when BB_GENERATE_MIRRORS_TARBALL is in use. Log excerpts: tar -czf TOPDIR/../downloads/git2_bitbucket.name-hidden.git.tar.gz . failed with exit code 1, output: tar: ./objects/pack/pack-89a1d76f6c08f53172ef1d02ff851d90564362c4.pack: file changed as we read it tar: ./objects/pack/pack-b4a48ada355d333630fdf6b4f67205b7c264dc2c.idx: File removed before we read it Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping. (Bitbake rev: a8d8cb847063862d1a7998963dd8b767ff73d877) Signed-off-by: Adam Romanek <romanek.adam@gmail.com> Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-06bitbake: doc: bitbake-user-manual: fix erroneous statement in glossary introMichael Opdenacker
This is the BitBake glossary. Remove an erroneous statement probably coming from the introduction to the YP/OE glossary. (Bitbake rev: 398a1686176c695d103591089a36e25173f9fd6e) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-06bitbake: docs: Add BB_HASHSERVE definition to glossaryMichael Opdenacker
(Bitbake rev: 0ab3255f8e9d9c8c8aa4788504ab06a50d1bb1f2) Signed-off-by: Michael Opdenacker <michael.opdenacker@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-06bitbake: build: warn on setting noexec/nostamp/fakeroot flag to any value ↵Chris Laplante
besides '1' Currently setting those flags to even the empty string "" causes it to be set, which is contrary to the documentation. In a future version of BitBake, we'd like to change the behavior so that setting the flag to "" does not set it. This will allow conditionally setting noexec, using variable expansion or inline Python. I found no places in poky or meta-openembedded where this warning would trigger. [YOCTO #13808] (Bitbake rev: 1e7655c4f765ba7b4791c4cca048a69bf8d9c93d) Signed-off-by: Chris Laplante <chris.laplante@agilent.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-06bitbake: doc: user-manual: ref-manual: remove mentions to ↵Quentin Schulz
BB_SETSCENE_VERIFY_FUNCTION2 This variable was removed in July 2019 with commit 5deaa5df730a "runqueue: Drop unused BB_SETSCENE_VERIFY_FUNCTION2". There's no replacement, so let's update the documentation to not mention this variable anymore. This was found by running: git grep -hoP '^ :term:`\K\w+(?=`)' doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst | \ xargs -I{} sh -c \ "grep -Rq --include='*.py' --include='*.conf' --include='*.bbclass' --include='*.bb' --include='*.bbappend' --include='*.inc' \ -w -E {}'_*(_[a-z]+[0-9a-z]*[a-z]+)*' || \ echo {}" (Bitbake rev: c2c0f5126c7c784bfd7a08f127e161a58c6b5d12) Signed-off-by: Quentin Schulz <foss@0leil.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-06bitbake: doc: user-manual: remove mentions to BBVERSIONSQuentin Schulz
BBVERSIONS support was dropped in November 2016 in commit 0bb188f01e39 "ast: remove BBVERSIONS support". Let's update the documentation to reflect this. This was found by running: git grep -hoP '^ :term:`\K\w+(?=`)' doc/bitbake-user-manual/bitbake-user-manual-ref-variables.rst | \ xargs -I{} sh -c \ "grep -Rq --include='*.py' --include='*.conf' --include='*.bbclass' --include='*.bb' --include='*.bbappend' --include='*.inc' \ -w -E {}'_*(_[a-z]+[0-9a-z]*[a-z]+)*' || \ echo {}" (Bitbake rev: 376e20c1aaff197020cdb68d309b3b22f19dafe8) Signed-off-by: Quentin Schulz <foss@0leil.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-06bitbake: asyncrpc: Reduce verbosityPaul Barker
(Bitbake rev: cc803609167b4c399ab768d9e131d618c086a4f2) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-06bitbake: asyncrpc: Add ping methodPaul Barker
This method is needed to support startup of the prservice. As it is so generic we can add it to the common asyncrpc module. (Bitbake rev: 25ccd697ea76f66b813be2296866b2d3405b079c) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-06bitbake: fetch2: add check for empty SRC_URI hash stringScott Weaver
No error was being reported when the hash string was set to empty. For example: SRC_URI[md5sum] = "" On a related note (not a bug): Because whitespace in the string will result in a checksum mismatch, the error message was updated to make it a little clearer why the error was thrown. For example: SRC_URI[md5sum] = " " or SRC_URI[md5sum] = " 209f8326f5137d8817a6276d9577a2f1" Now creates a message like this: File: '/home/scott/yocto-cache/downloads/rsync-3.2.3.tar.gz' has md5 checksum '209f8326f5137d8817a6276d9577a2f1' when ' 209f8326f5137d8817a6276d9577a2f1' was expected [YOCTO #14232] (Bitbake rev: a13510d0028e234ea2f4744b0d0c38558395c70f) Signed-off-by: Scott Weaver <weaverjs@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-18bitbake: server/process: Handle error in heartbeat funciton in OOM caseRichard Purdie
We've seen cases where an OOM error causes bitbake server to hang: 9171 02:21:09.127810 Command Completed Traceback (most recent call last): File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/bin/bitbake-server", line 51, in <module> bb.server.process.execServer(lockfd, readypipeinfd, lockname, sockname, timeout, xmlrpcinterface) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 550, in execServer server.run() File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 108, in run ret = self.main() File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 242, in main ready = self.idle_commands(.1, fds) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/server/process.py", line 370, in idle_commands bb.event.fire(heartbeat, self.cooker.data) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 216, in fire fire_class_handlers(event, d) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 123, in fire_class_handlers execute_handler(name, handler, event, d) File "/home/pokybuild/yocto-worker/qemux86/build/bitbake/lib/bb/event.py", line 93, in execute_handler ret = handler(event) File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 182, in defaultrun_buildstats write_host_data(os.path.join(bsdir, "host_stats"), e, d, "interval") File "/home/pokybuild/yocto-worker/qemux86/build/meta/classes/buildstats.bbclass", line 160, in write_host_data output = subprocess.check_output(c.split(), stderr=subprocess.STDOUT, timeout=limit).decode('utf-8') File "/usr/lib/python3.6/subprocess.py", line 356, in check_output **kwargs).stdout File "/usr/lib/python3.6/subprocess.py", line 423, in run with Popen(*popenargs, **kwargs) as process: File "/usr/lib/python3.6/subprocess.py", line 729, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1295, in _execute_child restore_signals, start_new_session, preexec_fn) OSError: [Errno 12] Cannot allocate memory We need to wrap the calls in the same high level wrapper as idle function calls and trigger an exit upon an unhandled exception. (Bitbake rev: 74042b5b89d5a170013fc1a327ce3a6530fbf7d5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-14bitbake: hashserver/client.py: drop unused importsArmin Kuster
(Bitbake rev: 9dc77266085c605b108641a9d76ac4dbdc064c34) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-14bitbake: hashserv/server.py: drop unused importsArmin Kuster
remove unused vars. (Bitbake rev: 3287d28a506f67abd192799e61ef28e74ce7002d) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-14bitbake: fetch2/wget: when checking latest versions, consider all numerical ↵Alexander Kanavin
directories Previously the regex was maching x.y, but wasn't matching x, which is a problem e.g. here: https://download.gnome.org/sources/epiphany/ (the new gnome version scheme adds 40-series at the end). (Bitbake rev: c03101576f447263ea38e8464210d3a3a2c27226) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06bitbake: bitbake: Switch to post release version number 1.51.0Richard Purdie
(Bitbake rev: 97a64d12f70eb02f1d35b4ffefb291b80ca8c425) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>