summaryrefslogtreecommitdiffstats
path: root/scripts
AgeCommit message (Collapse)Author
2019-05-22resulttool: Enable report for single result fileYeoh Ee Peng
Current validation check function inside resulttool disallow the report for single result file although the underlying library was able to handle both directory and file as source input to report. Removed the validation check as it was no longer needed and to enable report for single result file. (From OE-Core rev: dc15a21cbb3090dfb56e78f622486f843be4f54a) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-22devtool: prevent starting git gc in backgroundTaras Kondratiuk
Devtool creates a git repository for extracted sources in a temporary directory and then moves it to a final destination after patching is done. Unfortunately devtool is not aware that some of its git operations may have caused git garbage collector to start in background. If timing is just right a repository move fails because GC is removing git objects while they are being moved. The issue was hit on Krogoth release, but the code that moves created git repository is still the same. Disable gc.autodetach to make GC run synchronously and block git until it is done. Traceback (most recent call last): File "<snip>/openembedded-core/scripts/devtool", line 342, in <module> ret = main() File "<snip>/openembedded-core/scripts/devtool", line 329, in main ret = args.func(args, config, basepath, workspace) File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 352, in extract initial_rev = _extract_source(srctree, args.keep_temp, args.branch, False, rd) File "<snip>/openembedded-core/scripts/lib/devtool/standard.py", line 644, in _extract_source shutil.move(srcsubdir, srctree) File "/usr/lib/python2.7/shutil.py", line 299, in move copytree(src, real_dst, symlinks=True) File "/usr/lib/python2.7/shutil.py", line 208, in copytree raise Error, errors Error: [('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e', '<snip>/build/grpc/grpc_src/.git/objects/5e', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/5e'"), ('/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57', '<snip>/build/grpc/grpc_src/.git/objects/57', "[Errno 2] No such file or directory: '/tmp/devtool5RXkuX/workdir/grpc-1.2.5/.git/objects/57'"), many git objects ... ] (From OE-Core rev: eae5caaf050f54a065e2654a60f55fda2c3afd95) Signed-off-by: Taras Kondratiuk <takondra@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-22correct do_patch for kernel bbappend in sdkYann CARDAILLAC
do_patch rule of SDK's workspace/appends/linux-*.bbhappend may fail if script are not written in Python that was the case with Phytec's BSP, the fix was to replace the do_patch rule with : do_patch[noexec]="1" when the file was generated in scripts/lib/devtool/standard.py (From OE-Core rev: e041812a84025a9ff9121efabc3805ff2dfe6375) Signed-off-by: Yann CARDAILLAC <yann.cardaillac@smile.fr> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09resulttool/manualexecution: Refactor and simplify codebaseYeoh Ee Peng
Simplify and removed unnecessary codes. Refactor to allow pythonic loop. (From OE-Core rev: 31449f3a7649be781b7b61f915d5e879728e87af) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09resulttool/manualexecution: Fixed step sorted by integerYeoh Ee Peng
Currently the manual execution display step by sorting the step as string, where steps were not being sorted correctly when there are more than 9 steps. Fixed the step sorting by sorting step as integer. (From OE-Core rev: 2d96a9228fb451752c061c25582a5cc1735099bf) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09resulttool/manualexecution: Enable display full steps without press enterYeoh Ee Peng
Current manualexecution required pressing enter button to show each step information, where this was wasting execution time. Enable display full steps without needing to any press enter button. (From OE-Core rev: 6bb181b1e5f14d166cab8023b14316e54012c583) Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09resulttool/manualexecution: Standardize input checkYeoh Ee Peng
Current input checking does not match the standard input practiced by QA team. Change the input checking to match the standard input practiced by the QA team. (From OE-Core rev: f28cfbdad40a69f0cc1a7e71d20260c9621e6194) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09resulttool: Allow extraction of ptest dataRichard Purdie
Rather than simply discarding the ptest data, change the code to discard it when writing out the new testresult files, or optionally either preserve it, or write it as seperate discrete logs. This means the autobuilder should start writing out individual ptest log files as well as allowing ueers to extract these manually. (From OE-Core rev: 3c5e5730031ccfb8e851734db23814a4d4fcae4a) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-09resulttool: Allow store to work on single filesRichard Purdie
Store operations using a single file as a source weren't working as the os.walk command didn't like being given a single file. Fix the store operation to work for single files. (From OE-Core rev: 5173954c1ec75629bedbe06d6979dae36eb71b6f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26scripts/wic: Be consistent about how we call bitbakeRichard Purdie
Whilst its tempting to call bitbake_main(), the script also directly executes bitbake. By doing both this creates all kinds of races. Therefore change to use subprocess execution in all cases. (From OE-Core rev: 0507ec4c6069490c9a503e9e2b65f0e7b1962d34) (From OE-Core rev: c48c37a436b3ebfe4d8ca5d12e97a0cdff0e20a1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/manualexecution: To output right test case idMazliana
We found that manualexecution does not capture test suite values correctly if there are more than one test suite in test cases. After verification has made we found out we should retrieved full test cases value <test_module.test_suite.test_case> from oeqa/manual/ json file rather than split it them into new variables test_suite and test_cases. (From OE-Core rev: c4c63cdc1eca2acebc4ec9486b0d0592e47a8ae2) Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/report: Enable roll-up report for a commitYeoh Ee Peng
Enable roll-up all test results belong to a commit and to provide a roll-up report. (From OE-Core rev: f2d6019f39a4037cf0914b1cfb3ccb3a03dacfc6) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26scripts/resulttool: Enable manual result store and regressionYeoh Ee Peng
To enable store for testresults.json file from manualexecution, add layers metadata to configuration and add "manual" map to resultutils.store_map. To enable regression for manual, add "manual" map to resultutils.regression_map. Also added compulsory configurations ('MACHINE', 'IMAGE_BASENAME') to manualexecution. (From OE-Core rev: 3e711cd0482e9df644b011a7d6dadffdfe139f8e) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/regression: Ensure regressoin results are sortedYeoh Ee Peng
Sorted regression results to provide friendly viewing of report. (From OE-Core rev: cbe96a6dcb95032c61393bac348141e15a6b6590) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/store: Fix missing variable causing testresult corruptionRichard Purdie
(From OE-Core rev: 0d98b9957c611af354efb33f4e915c5166c87ad3) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/report: Ensure ptest results are sortedRichard Purdie
(From OE-Core rev: 2e3ffad9e97c5f757e1457721308eb8da63bcabf) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/report: Ensure test suites with no results show up on the reportRichard Purdie
ptest suites with no results don't show up on the reports even though we have a duration for them. Fix this so the fact they report no tests is visible. (From OE-Core rev: eb67a3cc4f8f57e85f1f95ced8f2c32ab5090866) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/report: Handle missing metadata sections more cleanlyRichard Purdie
Currently some older results files cause the code to give tracebacks. Handle these missing sections more cleanly. (From OE-Core rev: 93b87a9a7056fb507a585c454cef8db7194d7393) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/store: Handle results files for multiple revisionsRichard Purdie
Currently we cant store results if the results files span multiple different build revisons. Remove this limitation by iterating. (From OE-Core rev: 63d2c53c01d2aec90b3f0ab298a61a551b23d8fc) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool/resultutils: Avoids tracebacks for missing logsRichard Purdie
(From OE-Core rev: faacf35d748067fd17d169c09bc2759606e6e819) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool: Improvements to allow integration to the autobuilderRichard Purdie
This is a combined patch of the various tweaks and improvements I made to resulttool: * Avoid subprocess.run() as its a python 3.6 feature and we have autobuilder workers with 3.5. * Avoid python keywords as variable names * Simplify dict accesses using .get() * Rename resultsutils -> resultutils to match the resultstool -> resulttool rename * Formalised the handling of "file_name" to "TESTSERIES" which the code will now add into the json configuration data if its not present, based on the directory name. * When we don't have failed test cases, print something saying so instead of an empty table * Tweak the table headers in the report to be more readable (reference "Test Series" instead if file_id and ID instead of results_id) * Improve/simplify the max string length handling * Merge the counts and percentage data into one table in the report since printing two reports of the same data confuses the user * Removed the confusing header in the regression report * Show matches, then regressions, then unmatched runs in the regression report, also remove chatting unneeded output * Try harder to "pair" up matching configurations to reduce noise in the regressions report * Abstracted the "mapping" table concept used to pairing in the regression code to general code in resultutils * Created multiple mappings for results analysis, results storage and 'flattening' results data in a merge * Simplify the merge command to take a source and a destination, letting the destination be a directory or a file, removing the need for an output directory parameter * Add the 'IMAGE_PKGTYPE' and 'DISTRO' config options to the regression mappings * Have the store command place the testresults files in a layout from the mapping, making commits into the git repo for results storage more useful for simple comparison purposes * Set the oe-git-archive tag format appropriately for oeqa results storage (and simplify the commit messages closer to their defaults) * Fix oe-git-archive to use the commit/branch data from the results file * Cleaned up the command option help to match other changes * Follow the model of git branch/tag processing used by oe-build-perf-report and use that to read the data using git show to avoid branch change * Add ptest summary to the report command * Update the tests to match the above changes (From OE-Core rev: b4513e75f746a0989b09ee53cb85e489d41e5783) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26scripts/resulttool: enable manual execution and result creationMazliana
Integrated “manualexecution” operation to resulttool scripts. Manual execution script is a helper script to execute all manual test cases in baseline command, which consists of user guideline steps and the expected results. The last step will ask user to provide their input to execute result. The input options are passed/failed/blocked/skipped status. The result given will be written in testresults.json including log error from the user input and configuration if there is any.The output test result for json file is created by using OEQA library. The configuration part is manually key-in by the user. The system allow user to specify how many configuration they want to add and they need to define the required configuration name and value pair. In QA perspective, "configuration" means the test environments and parameters used during QA setup before testing can be carry out. Example of configurations: image used for boot up, host machine distro used, poky configurations, etc. The purpose of adding the configuration is to standardize the output test result format between automation and manual execution. To use these scripts, first source oe environment, then run the entry point script to look for help. $ resulttool To execute manual test cases, execute the below $ resulttool manualexecution <manualjsonfile> By default testresults.json store in <build_dir>/tmp/log/manual/ [YOCTO #12651] (From OE-Core rev: 07054cc2691fd2822028a3fd55185af457f79ebf) Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-26resulttool: enable merge, store, report and regression analysisYeoh Ee Peng
OEQA outputs test results into json files and these files were archived by Autobuilder during QA releases. Example: each oe-selftest run by Autobuilder for different host distro generate a testresults.json file. These scripts were developed as a test result tools to manage these testresults.json file. Using the "store" operation, user can store multiple testresults.json files as well as the pre-configured directories used to hold those files. Using the "merge" operation, user can merge multiple testresults.json files to a target file. Using the "report" operation, user can view the test result summary for all available testresults.json files inside a ordinary directory or a git repository. Using the "regression-file" operation, user can perform regression analysis on testresults.json files specified. Using the "regression-dir" and "regression-git" operations, user can perform regression analysis on directory and git accordingly. These resulttool operations expect the testresults.json file to use the json format below. { "<testresult_1>": { "configuration": { "<config_name_1>": "<config_value_1>", "<config_name_2>": "<config_value_2>", ... "<config_name_n>": "<config_value_n>", }, "result": { "<testcase_namespace_1>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, "<testcase_namespace_2>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, ... "<testcase_namespace_n>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, } }, ... "<testresult_n>": { "configuration": { "<config_name_1>": "<config_value_1>", "<config_name_2>": "<config_value_2>", ... "<config_name_n>": "<config_value_n>", }, "result": { "<testcase_namespace_1>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, "<testcase_namespace_2>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, ... "<testcase_namespace_n>": { "status": "<PASSED or FAILED or ERROR or SKIPPED>", "log": "<failure or error logging>" }, } }, } To use these scripts, first source oe environment, then run the entry point script to look for help. $ resulttool To store test result from oeqa automated tests, execute the below $ resulttool store <source_dir> <git_branch> To merge multiple testresults.json files, execute the below $ resulttool merge <base_result_file> <target_result_file> To report test report, execute the below $ resulttool report <source_dir> To perform regression file analysis, execute the below $ resulttool regression-file <base_result_file> <target_result_file> To perform regression dir analysis, execute the below $ resulttool regression-dir <base_result_dir> <target_result_dir> To perform regression git analysis, execute the below $ resulttool regression-git <source_dir> <base_branch> <target_branch> [YOCTO# 13012] [YOCTO# 12654] (From OE-Core rev: 39cda3498ec68b91a671ead256231324d74f7d4c) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-03-24send-error-report: Add --no-ssl to use http protocolRobert Yang
The script use https protocol by default, but the error-report-web server's https connection may not work (e.g., doesn't work with python 2.7.6), so add an option --no-ssl to make it use http. (From OE-Core rev: a4c89902a24c0961657c8281425d480097aadfa6) (From OE-Core rev: f9dac6e62623b0ac1a2b2f40b54337721afab7a5) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25oe-build-perf-report/gitarchive: Move common useful functions to libraryRichard Purdie
These functions can be reused by the resulttool code so move to the common function library for this purpose. (From OE-Core rev: c66f848938c04e133259c5b6903dc592866ab385) (From OE-Core rev: 4b1bd35030c5502873106782a35c4f5a9446e20c) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25scripts/oe-git-archive: Separate out functionality to library functionRichard Purdie
This turns the core of the script into a library function. Ultimately this will let us call that code with custom 'keywords' rather than relying on the data parsed from bitbake metadata which can't be used when archiving historical results. (From OE-Core rev: 4820ca2b0850e29b04a4fd5659a6e9837d6714d0) (From OE-Core rev: e47c48768a01191ec36b9732288e05c9f5e80d52) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25lib/buildstats: Improve error messageRichard Purdie
Just stating the recipe counts are different isn't helpful, showing the differences makes it much easier to understand what changed. (From OE-Core rev: 1d84b782e3af6f0e6922d72895c905877cc33739) (From OE-Core rev: 047fd7cf6fc2a3d5d170241f5cc5b61b8d9d8522) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25runqemu: Make QB_MEM easier to setRobert Yang
It only could be set as the following in the past: QB_MEM = "-m 256" Now it also can be set as: QB_MEM = "-m 256M (or m)" QB_MEM = "256M (or m)" [YOCTO #11522] (From OE-Core rev: ad246f5ce0652bd917d85884176baa746e1379ff) (From OE-Core rev: ed10ef4f8deaec739ce3713b6ce3b63e134a8c4a) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25runqemu: Let qemuparams override default settingsRobert Yang
Fixed: In meta/conf/machine/include/qemuboot-x86.inc: QB_CPU_x86-64 = "-cpu core2duo" $ runqemu qemux86-64 qemuparams="-cpu coreduo" Check /proc/cpuinfo, it should use coreduo rather than core2duo since user specifies it, but it doesn't, append qemuparams to the last can fix the problem. [YOCTO #11773] (From OE-Core rev: a847dd7202a2c493788c45d11eb86866264af7a4) (From OE-Core rev: 81ab6332bee0426201626cc8a0339ce3e6b81d6f) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25wic/engine: Fix missing parted autobuilder failuresRichard Purdie
OE-Core rev: a88bcbae850a2e6d182291d3f8e167aabdbe4842 broke the ability to find parted as it may be in sbin which is not in PATH for some users on some distros. Iterate on the original patch to fix this and also fix the original problem. (From OE-Core rev: af3803e5189d7814f9dbd238fb6dab200f351e1a) (From OE-Core rev: 6e2bf8691ffd6004cb40e71a2d1d002b5ed5808f) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25wic/engine.py: Load paths from PATH environment variableWilliam Bourque
Load self.paths from environment variable and if it fails, fall back to hardcoded list. This is required for users that would need to load different e2fsprogs binaries if their system's ones are not recent enought (From OE-Core rev: a88bcbae850a2e6d182291d3f8e167aabdbe4842) (From OE-Core rev: 319ee1bb146b91f78d7a141def2b1cf92cf88887) Signed-off-by: William Bourque <wbourque@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25scripts/lib/wic/engine: Fix cp's target path for ext* filesystemsLeonardo Augusto
Python subprocess' shell=True defaults to /bin/sh[1][2], which often refers to a POSIX-compliant shell. As the -e flag is not defined in the POSIX standard[3], some shells may interpret "-e" as the first argument to the echo command, causing the rest of the command line to fail. In this particular case, "echo -e 'cd {}'" is interpreted as "-e cd {}", which causes the first line of the command to fail, and causing cp to always place the source file in the filesystem's root. Replacing "echo -e" for a printf command makes this more portable. This issue only affects "wic cp" for ext* filesystems. [1] https://docs.python.org/2/library/subprocess.html [2] https://docs.python.org/3/library/subprocess.html [3] http://pubs.opengroup.org/onlinepubs/9699919799 (From OE-Core rev: 9a5b703d4e60c3ccdf80b5289b8e6fc91133fcde) (From OE-Core rev: bf51d6da725a3d5850c3cbd70fb91b91c08e507f) Signed-off-by: Leonardo Augusto <leobsd@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25checklayer: Avoid adding the layer if it is already presentRobert Yang
* Rename add_layer() to add_layers() so that add_layer_dependencies() can re-use it. * Avoid adding the layer if it is already present [YOCTO #13148] (From OE-Core rev: b9cc18d83f55ff48c3d6e60c56359f6736d5a06a) (From OE-Core rev: 64310d9d18bb1751595c197e6955774ad5514844) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25checklayer: generate locked-sigs.inc under builddirChangqing Li
yocto-check-layer will find locked-sigs.inc under builddir, but locked-sigs.inc is generated under current bitbake working dir. if run yocto-check-layer outside builddir, we will met error like "No such file or directory: *locked-sigs.inc". change to run bitbake -S under builddir to fix this problem. [YOCTO #12973] (From OE-Core rev: 2ec0bc0b038bc6413978c8f34ef5c0d22b4bc3e7) (From OE-Core rev: f236d8782958df953ca1e2a6f09779cbc8f2d600) Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25yocto-check-layer-wrapper: Fix path for oe-init-build-envRobert Yang
We only could run it in top of oe-core dir since it assumed oe-init-build-env was in cwd, this patch fixes the problem. [YOCTO #13148] (From OE-Core rev: 47d7b5fcad127171243cc95c586c2c7f35cfac3b) (From OE-Core rev: 09e4b9bead32f143f4a685794c4709fe41bf014d) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25oe-build-perf-report: Fix missing buildstats comparisionsRichard Purdie
Integers were being compared to strings leading to missing buildstats comparision data. Fix this. (From OE-Core rev: 2dbbf598192ae2b3aa488df042f56aa6c6634a00) (From OE-Core rev: 9ee8845a583a3fa9e9b07d1b0301201ae56c6c41) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25oe-build-perf-report: Improve branch comparision handlingRichard Purdie
When comparing branches, correctly filter the revisions corresponding to the specific branch specified. Also use the commit numbers as a way to gauge spatially related commits for comparision meaning comparisions for out of order build revisions becomes meaninful. This should improve the reporting for autobuilder generated builds. Also improve the branch option help text. (From OE-Core rev: 9f6f4ab6eec9dca07af7f53da5f737a6167bfb38) (From OE-Core rev: 111ee3499308bcde54268d58bf6163a98a9733db) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25oe-build-perf-report: Allow commits from different branchesRichard Purdie
The code won't currently allow comparisions of two commits on different branches even if the commits are specified by their hashes. This updates the code to search two branches for any relavent commits, hence allowing comparisions to be made. A particularly useful case is master vs. master-next for example. (From OE-Core rev: 2d835bb1570c515ae501442f3ce19fae8e249b27) (From OE-Core rev: c4263bf021daafd4e89acd4cba48c9a7fbd0d8a1) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25oe-build-perf-report: Allow branch without hostnameRichard Purdie
Allow the branch to be set without the hostname option. Previously if hostname wasn't set, branch would be overwritten regardless of whether it was set or not. (From OE-Core rev: 3ec43bf4c6c8f7730a67f63ad4e14903f289014e) (From OE-Core rev: b162c70dd6e926e1700553e27b41eb1c12ee2dce) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25multilib_header_wrapper.h: Use #pragma onceKhem Raj
Avoid infinite include loops, especially with bits/wordsize.h which is now possible with the synthesized headers since we now also synthesize bits/wordsize.h itelf for some arches e.g. arm/aarch64 In cases where extra preprocessing tools are used such as clang-tidy e.g. and these tools are not passed the knowledge about architecture then case comes where we enter into header include loop for bits/wordsize.h, since this template does explicitly include bits/wordsize.h To fix this emits the pragma once at beginning of file, this is better solution than include guards, and pragma once is practically supported on all compilers except few e.g. cray c/c++ compiler (From OE-Core rev: 6ab11ac25bb987642d5ca1a07f4bd5f30c66c9c5) (From OE-Core rev: 8f00480fe826ebaff2b78796d757b9e2734f2deb) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25send-error-report: Use https instead of http protocolKhem Raj
errors.yp.org can understand https (From OE-Core rev: f540fed0aec24e28eff8522fddbe3cf7ee45e30b) (From OE-Core rev: 857ebd7fac590f569bff7e8ca29b6ae22eaaa65a) Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25oe-git-proxy: Avoid resolving NO_PROXY against local filesJan Kiszka
NO_PROXY may contain * elements, and if we are unlucky (or want to match all hosts with *), we will pick up local files rather than doing the match in match_host. Quoting helps here. (From OE-Core rev: 7e216e806ca765152fd874e24785f783a3201284) (From OE-Core rev: 5b5c8ccaba188530ae4b05fd49804aaa74cbb949) Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25devtool: improve git repo checks before check_commits logicDan Dedrick
The check_commits logic assumes that both devtool-base and args.branch exist in the git repo that it is operating on. In order to prevent errors at that point it's best to first ensure that both of these refs actually exist. If they don't both exist then the check_commits logic should just be skipped, as it would be if the repo wasn't originally checked out by devtool. Previously if a user removed the args.branch branch from their devtool cloned repo this code would crash on adding the repo with -n. The crash would look like this: Traceback (most recent call last): File "/home/ddedrick/src/poky/scripts/devtool", line 344, in <module> ret = main() File "/home/ddedrick/src/poky/scripts/devtool", line 331, in main ret = args.func(args, config, basepath, workspace) File "/home/ddedrick/src/poky/scripts/lib/devtool/standard.py", line 812, in modify (stdout, _) = bb.process.run('git log devtool-base..%s' % branch, cwd=srctree) File "/home/ddedrick/src/poky/bitbake/lib/bb/process.py", line 178, in run raise ExecutionError(cmd, pipe.returncode, stdout, stderr) bb.process.ExecutionError: Execution of 'git log devtool-base..devtool' failed with exit code 128: fatal: ambiguous argument 'devtool-base..devtool': unknown revision or path not in the working tree. Use '--' to separate paths from revisions, like this: 'git <command> [<revision>...] -- [<file>...]' (From OE-Core rev: f13a3490fdb404bbd4c77e45b83540d6deec1358) (From OE-Core rev: ebd3c5e1534ba4ac2a6e97725f8d8650d4aa52a0) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25devtool: remove duplicate overridesDan Dedrick
DEVTOOL_EXTRA_OVERRIDES only needs one entry for each instance of overrides. Previous to these changes it would find every override to SRC_URI and add it to the list. This would duplicate instances where SRC_URI is modified multiple times with the same override like: SRC_URI_append_foo += "file://0001-foo.patch" SRC_URI_append_foo += "file://0002-bar.patch" A bbappend might also overwrite a SRC_URI override, which would also cause multiple instances to occur. When there are multiple instances of the same override in DEVTOOL_EXTRA_OVERRIDES it causes devtool modify to fail when creating override branches. The failure occurs when attempting to create the same override branch a second time and looks like this: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:devtool_post_patch(d) 0003: File: '/build/poky/meta/classes/devtool-source.bbclass', lineno: 202, function: devtool_post_patch 0198: 0199: for override in extra_override_list: 0200: localdata = bb.data.createCopy(d) 0201: if override in default_overrides: *** 0202: bb.process.run('git branch devtool-override-%s %s' % (override, devbranch), cwd=srcsubdir) 0203: else: 0204: # Reset back to the initial commit on a new branch 0205: bb.process.run('git checkout %s -b devtool-override-%s' % (initial_rev, override), cwd=srcsubdir) 0206: # Run do_patch function with the override applied File: '/build/poky/bitbake/lib/bb/process.py', lineno: 178, function: run 0174: if not stderr is None: 0175: stderr = stderr.decode("utf-8") 0176: 0177: if pipe.returncode != 0: *** 0178: raise ExecutionError(cmd, pipe.returncode, stdout, stderr) 0179: return stdout, stderr Exception: bb.process.ExecutionError: Execution of 'git branch devtool-override-foo devtool' failed with exit code 128: fatal: A branch named 'devtool-override-foo' already exists. (From OE-Core rev: 90f667db2219f04e6d61588cd61056d3d8da6d7d) (From OE-Core rev: 162be3d43f1d48a22b3aed32cc2f593355e5bf81) Signed-off-by: Dan Dedrick <ddedrick@lexmark.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25scripts/contrib/build-perf-test.sh: Remove itRichard Purdie
This is the older script which the code in lib/oeqa/buildperf replaces so delete the obsolete script to avoid confusion. (From OE-Core rev: 9908b7955e75e10d6a1384e4c08417ef534464af) (From OE-Core rev: 9dd395357d32286b5f81673920638316cf01f883) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-25scripts/contrib/build-perf-test-wrapper.sh: Improve interaction with ↵Richard Purdie
autobuilder automation This tweaks the script to: * Ensure directories exist and can be written to * Allow the downloads directory to be specified * Error early if the phantomjs or optipng dependencies are not installed * Allow the location of the globalres.log file to be specified This means that the main build directory can be destroyed and any state from the script is stored elsewhere allowing it to be triggered automatically from the autobuilder infrastructure. (From OE-Core rev: 1de5fbd484e6a747ae6419ccc89d8c2911b9706b) (From OE-Core rev: d6f00d9d89f62e60eaa87abf89477ed7865e2859) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06wic: sdimage-bootpart: Use mmcblk0 drive instead of bogus mmcblkAlexey Brodkin
Apparently either nobody ever used sdimage-bootpart.wks or fstab was kept untouched due to "--no-fstab-update" usage as some boards like IMX may have, see [1]. Otherwise addition of the following line to the target's fstab: ---------------------->8----------------- /dev/mmcblkp1 /boot vfat defaults 0 0 ---------------------->8----------------- gets us unusable target: ---------------------->8----------------- [* ] A start job is running for dev-mmcblkp1.device (23s / 1min 30s) [ TIME ] Timed out waiting for device dev-mmcblkp1.device. [DEPEND] Dependency failed for /boot. [DEPEND] Dependency failed for Local File Systems. You are in emergency mode. After logging in, type "journalctl -xb" to view system logs, "systemctl reboot" to reboot, "systemctl default" or "exit" to boot into default mode. Press Enter for maintenance ---------------------->8----------------- Fix it with use of "mmcblk0" device name. [1] https://lists.yoctoproject.org/pipermail/meta-freescale/2018-February/022156.html (From OE-Core rev: 57a925dbd949e488980b02896106efd82bf47ed2) (From OE-Core rev: a916a923dcd470a15d1136c818d77867e726600d) Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Cc: Maciej Borzecki <maciej.borzecki@open-rnd.pl> Cc: Maciek Borzecki <maciek.borzecki@gmail.com> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06runqemu-gen-tapdevs: Allow run --help without sudoRobert Yang
Then we can get user's UID and GID rather than hardcode to 1000, e.g.: - Without sudo $ runqemu-gen-tapdevs --help [snip] $ sudo ../poky/scripts/runqemu-gen-tapdevs 15220 100 4 tmp/sysroots-components/x86_64/qemu-helper-native/usr/bin [snip] - With sudo $ sudo ../poky/scripts/runqemu-gen-tapdevs --help [snip] $ sudo ../poky/scripts/runqemu-gen-tapdevs 15220 100 4 tmp/sysroots-components/x86_64/qemu-helper-native/usr/bin [snip] (From OE-Core rev: 446e7da7e56f9de3602498b5ef40e9e0f8f71837) (From OE-Core rev: 79471a27018d2e766b349540693e6c196fa75869) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06runqemu: clean up subprocess usageRoss Burton
Where possible pass lists instead of strings, don't use a subshell, and call check*() instead of using Popen directly. (From OE-Core rev: d2374623444752af1ad748ed36b68ea58f629bf6) (From OE-Core rev: 4b5a4b718c027f54ef0660a731f11a9d249b4af8) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-02-06scripts/oe-git-archive: fix non-existent key referencing errorYeoh Ee Peng
Without installing gitpython package, oe-git-archive will face error below, where it was referencing key that was non-existent inside metadata object. Traceback (most recent call last): File "<poky_dir>/scripts/oe-git-archive", line 271, in <module> sys.exit(main()) File "<poky_dir>/scripts/oe-git-archive", line 229, in main 'commit_count': metadata['layers']['meta']['commit_count'], KeyError: 'commit_count' Fix this error by adding exception catch when referencing non-existent key (based on inputs provided by Richard Purdie). [YOCTO# 13082] (From OE-Core rev: 9a3cc9b8523b78dda6c3f3f2e12798b2b907d7e5) (From OE-Core rev: f51e59bb6d571606cf887a3f623380cc3516c5a2) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-05-07 13:51:11 +0000