summaryrefslogtreecommitdiffstats
path: root/scripts/lib
AgeCommit message (Collapse)Author
2019-09-16oeqa/core/case.py: Add OEPTestResultTestCase for ptestresult helpersNathan Rossi
Add the OEPTestResultTestCase class as a mix-in class to provide helper functions for interacting with ptestresults within the extraresults object generated by the test case. This class also provides default compression of log text and log files. Also add support to resulttool for decoding/decompressing log files embedded in the test results. (From OE-Core rev: 06cba9883a5964320969301fd05eeb6bec3e786d) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-16resulttool: Handle multiple series containing ptestresultsNathan Rossi
Handle multiple results series having ptestresults content. The contents are merged on a per-result basis where duplicates are ignored (with a warning message printed). The 'ptestresults.sections' collection is also merged on a per-suite basis. (From OE-Core rev: 47edd51970ed0c33edbe04fd72abd1cfc6ecd3d1) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-06resulttool: Add reproducible log extractionJoshua Watt
Adds an argument to the log subcommand to extract the raw logs from the reproducible selftest. To prevent ambiguity, the "--raw" argument has been renamed "--raw-ptest", although the old "--raw" argument is kept around for compatibility. [YOCTO #13324] (From OE-Core rev: 7a4ebb361ff1efc22e7dafadfa60c98bc8a79ed4) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-06oe-selftest: Implement console 'keepalive' outputNathan Rossi
Similar to bitbake, implement a 'keepalive' output to the console to ensure CI systems do not kill the process. The default timeout for bitbake is 5000s. (From OE-Core rev: 77939cca96fa5467c88eafa3ac0db2db4aef09d6) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-03qemuriscv: Generate a wic rootFS with a larger filesystemAlistair Francis
This allows us to generate a rootFS with a large filesystem for use with QEMU. (From OE-Core rev: e06439200e44999c1e2f88d7d6c651da13698ca7) Signed-off-by: Alistair Francis <alistair.francis@wdc.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-09-01devtool: Avoid failure for recipes with S == WORKDIR and no local filesPeter Kjellerstedt
When extracting the sources for a recipe that has S == WORKDIR and no local files in the SRC_URI (which, e.g., can happen for a recipe with a URI that has the unpack=false attribute), the extraction fails with the following backtrace: Traceback (most recent call last): File ".../scripts/devtool", line 344, in <module> ret = main() File ".../scripts/devtool", line 331, in main ret = args.func(args, config, basepath, workspace) File ".../poky/scripts/lib/devtool/standard.py", line 762, in modify initial_rev, _ = _extract_source(srctree, args.keep_temp, args.branch, False, config, basepath, workspace, args.fixed_setup, rd, tinfoil, no_overrides=args.no_overrides) File ".../poky/scripts/lib/devtool/standard.py", line 647, in _extract_source bb.process.run('git %s commit -a -m "Committing local file symlinks\n\n%s"' % (' '.join(useroptions), oe.patch.GitApplyTree.ignore_commit_prefix), cwd=srctree) File ".../poky/bitbake/lib/bb/process.py", line 178, in run raise ExecutionError(cmd, pipe.returncode, stdout, stderr) bb.process.ExecutionError: Execution of 'git commit -a -m "Committing local file symlinks %% ignore"' failed with exit code 1: On branch devtool nothing to commit, working tree clean This is because no files were found in the oe-local-files directory and consequently no symbolic links were added using `git add`, but the `git commit` command was still executed. (From OE-Core rev: 3fdf304e72a1fb5de8bf9bc21e5b598fefb08648) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-30yocto-check-layer: Allow any case for README file detectionMark Hatle
It's become more commone for files to be named "readme" or "Readme" on github servers in recent time. So adjust the scanning to allow any mix of case. (From OE-Core rev: afe46eca15b6ddfa15c75cb7b707d6dd9aae3eae) Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-28scripts/lib/resulttool/report.py: Add more result typesNathan Rossi
Add additional result types into the dictionary to handle dejagnu style test results. These include PASS, FAIL, XPASS, XFAIL, UNSUPPORTED, UNTESTED, UNRESOLVED and ERROR. (From OE-Core rev: 684fc36402a23760b203f4761f284043031c799c) Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-21devtool: build: Also run deploy for devtool build if applicableJaewon Lee
Right now `devtool build` runs populate_sysroot and packagedata tasks. Adding deploy to this list, if the recipe has the deploy task, so that the newly built artifacts are available in the deploy directory. Applicable only for packages with deploy task, such as kernel. [YOCTO#13382] (From OE-Core rev: b38a1328f0c7bc4b4102a05daee4058fd3214489) Signed-off-by: Jaewon Lee <jaewon.lee@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-21resulttool: Prevent multiple results for the same testJon Mason
Currently, if a test occurs multiple times over different series, the code will sum these. This can lead to confusion over the desired results. Change the code to report the redundant tests and skip adding an additional values. (From OE-Core rev: caeaa0648dff64c56a33f52e45e82bfab6719e3e) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-21yocto-check-layer: Ensure we use OEBasicHash as the signature handlerRichard Purdie
The layer checks are designed to work with OEBasicHash so ensure that handler is in use rather than the new hash equivalency one as an example. (From OE-Core rev: a10bf92516a4771e2dc49ba9f74323d7a87a1619) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-12wic: Make disk partition size consistently computedJason Wessel
When using different root directories with a wks file wic is using the value from the original ROOTFS_SIZE which is not correct. Example: Number Start End Size File system Name Flags 1 20.5kB 318MB 318MB fat16 otaefi legacy_boot, msftdata 2 318MB 636MB 318MB ext4 otaboot 3 636MB 1709MB 1074MB ext4 otaroot 4 1709MB 2027MB 318MB ext4 otaboot_b 5 2027MB 3101MB 1074MB ext4 otaroot_b 6 3101MB 5249MB 2147MB ext4 fluxdata The partitions 1, 2, and 3 incorrectly inherit the size, instead of using a computed size. With the patch applied it is working properly: Number Start End Size File system Name Flags 1 20.5kB 14.5MB 14.5MB fat16 otaefi legacy_boot, msftdata 2 14.5MB 65.3MB 50.8MB ext4 otaboot 3 65.3MB 1139MB 1074MB ext4 otaroot 4 1139MB 1190MB 50.8MB ext4 otaboot_b 5 1190MB 2264MB 1074MB ext4 otaroot_b 6 2264MB 4411MB 2147MB ext4 fluxdata As for the removal of the bb.warn, if the size is not specified, it is not something to warn the end user about. Some of my default images make use of the head room + a computed directory size or while generating images. (From OE-Core rev: 8e48b4d6c4d0ed213089a7449fea63aa0656e786) Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-08-12wic: Add partition type for msdos partition tablesJason Wessel
In order to create a msdos partition table disk image that can auto expand after the image is copied to an SD card, wic needs the ability to have a primary partition as the last entry. The desired use case is to be able to create an A/B update partition image scheme with a /var volume that can be auto expanded to the remainder of the SD card at run time. The typical .wks file will look similar to the following: bootloader --ptable msdos part / --source rawcopy --sourceparams="file=u-boot.imx" \ --ondisk mmcblk --no-table --align 1 --size 1 part /boot --source bootimg-partition \ --ondisk mmcblk --fstype=vfat --label boot --active --align 4 --size 16 part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \ --ondisk mmcblk --fstype=ext4 --label otaboot --align 4 --type logical part / --source rawcopy --sourceparams="file=imx6.otaimg" \ --ondisk mmcblk --fstype=ext4 --label otaroot --align 4 --type logical part / --source rawcopy --sourceparams="file=imx6_boot.otaimg" \ --ondisk mmcblk --fstype=ext4 --label otaboot_b --align 4 --type logical part / --source rawcopy --sourceparams="file=imx6.otaimg" \ --ondisk mmcblk --fstype=ext4 --label otaroot_b --align 4 --type logical part /var --source rawcopy --sourceparams="file=imx6_var.otaimg" \ --ondisk mmcblk --fstype=ext4 --label fluxdata --align 4 Without the patch applied, wic will generate an SD card image that looks like: Disk boot.img: 890940s Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 2056s 48001s 45946s primary fat16 lba 2 48008s 132467s 84460s primary ext4 3 132472s 454467s 321996s primary ext4 4 454471s 890939s 436469s extended lba 5 454472s 538931s 84460s logical ext4 6 538936s 860931s 321996s logical ext4 7 860936s 890939s 30004s logical ext4 boot With the patch applied a primary partition can be created at the end of the image which can be expanded to fill the free space on the media where the image has been copied, which looks like: Disk boot.img: 890940s Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 2056s 48001s 45946s primary fat16 lba 2 48007s 860931s 812925s extended lba 5 48008s 132467s 84460s logical ext4 6 132472s 454467s 321996s logical ext4 7 454472s 538931s 84460s logical ext4 8 538936s 860931s 321996s logical ext4 3 860936s 890939s 30004s primary ext4 boot (From OE-Core rev: 56add7cc547e0113cdf980579d1421b14cc233e5) Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-23wic/plugins: Source that support both EFI and BIOSWilliam Bourque
Add a source plugin that support both EFI and legacy PC-Bios. While using this plugin, both bootloaders configurations reside in the same /boot partitions. This plugin has very little code : to avoid code duplication, we simply re-import bootimg-pcbios and bootmg-efi source and call both their SourcePlugin methods. (From OE-Core rev: c8545d54139c6c48bffd1dd1d39d79891626c6f7) Signed-off-by: William Bourque <wbourque@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-22devtool: remove temp dir in upgradeChen Qi
For now, the temp dir is left in system, although the temporary source directory has been cleaned up. So we clean it up too. (From OE-Core rev: 8a0602327d5afcf4f36850d3f05c9721305852af) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-19wic: Fix (again) partition files UIDs on multi rootfs imagesRicardo Ribalda Delgado
Commit 450335ba5e73a375eb9932b4c4cf37979640dbfc copies the pseudo database to the working directory in order to have ownership information when the filesystem is generated. Unfortunately this does not work anymore. The filenames on the database are absolute and there is no information about the new directory. Instead of fixing the database, we could redo a bit the way we patch the fstab file. Now I am saving the old contents of fstab, modifying the file and then reverting the changes on exit. This is faster than the previous approach, although it can cause indeterminism if the application is killed before finishing. (From OE-Core rev: dcbf7b864dd1713b54a172d8714ce1508482f086) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-17wic: add support for kernel with initramfs bundledChee Yang Lee
when INITRAMFS_IMAGE_BUNDLE and INITRAMFS_IMAGE are set, wic should look for kernel with initramfs image bundled. Include required variable MACHINE, INITRAMFS_IMAGE_BUNDLE, INITRAMFS_IMAGE, INITRAMFS_LINK_NAME and KERNEL_IMAGETYPE in WICVARS. No longer require default value for variable kernel as KERNEL_IMAGETYPE is not optional variable and included in WICVARS. image_types_wic to inherit kernel-artifact-names to obtain default INITRAMFS_LINK_NAME when INITRAMFS_IMAGE_BUNDLE are set. update wic.Wic2.test_image_env test case to filter optional variable INITRAMFS_LINK_NAME, INITRAMFS_IMAGE and INITRAMFS_IMAGE_BUNDLE. (From OE-Core rev: bac984fbb2d5ad5d13ba3275c8a3e878d8753c58) Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-17scripts/buildstats-diff: Add option to filter tasksJoshua Watt
Adds a command line option to filter out the buildstats-diff report by one more more tasks. e.g.: buildstats-diff --only-task do_compile A B will only show the differences for do_compile tasks. The --only-task option can be specified multiple times to filter out multiple tasks at once. (From OE-Core rev: a8c7960d24c48107fd3703e49c38f890e84e2226) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-17devtool: provide support for devtool menuconfig commandSai Hari Chandana Kalluri
All packages that support the menuconfig task will be able to run devtool menuconfig command. This would allow the user to modify the current configure options and create a config fragment which can be added to a recipe using devtool finish. 1. The patch checks if devtool menuconfig command is called for a valid package. 2. It checks for oe-local-files dir within source and creates one if needed, this directory is needed to store the final generated config fragment so that devtool finish can update the recipe. 3. Menuconfig command is called for users to make necessary changes. After saving the changes, diffconfig command is run to generate the fragment. Syntax: devtool menuconfig <package name> Ex: devtool menuconfig linux-yocto The config fragment is saved as devtool-fragment.cfg within oe-local-files dir. Ex: <workspace_path>/sources/linux-yocto/oe-local-files/devtool-fragment.cfg Run devtool finish to update the recipe by appending the config fragment to SRC_URI and place a copy of the fragment within the layer where the recipe resides. Ex: devtool finish linux-yocto meta [YOCTO #10416] (From OE-Core rev: 417feb559a74b367315e8658d6ba868a4f8d1340) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-17devtool/standard.py: Create a copy of kernel source within work-shared if ↵Sai Hari Chandana Kalluri
not present If kernel source is not already downloaded i.e staging kernel dir is empty, place a copy of the source when the user runs devtool modify linux-yocto. This way the kernel source is available for other packages that use it. [YOCTO #10416] (From OE-Core rev: bb42ab90835e8ec2f1dfbb35056c353784693266) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-17devtool/standard.py: Update devtool modify to copy source from work-shared ↵Sai Hari Chandana Kalluri
if its already downloaded In the regular devtool modify flow, the kernel source is fetched by running do_fetch task. This is an overhead in time and space. This patch updates modify command to check if the kernel source is already downloaded. If so, then instead of calling do_fetch, copy the source from work-shared to devtool workspace by creating hard links else run the usual devtool modify flow and call do_fetch task. [YOCTO #10416] (From OE-Core rev: 3c3a9bae296f849dbfe03942282f44036e6fa1fb) Signed-off-by: Sai Hari Chandana Kalluri <chandana.kalluri@xilinx.com> Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-11recipetool: add MD5 hash for the line-wrapped MPL-1.1 licenseRoss Burton
(From OE-Core rev: d2755523d339a4eba7a034bdef5563657609f563) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-03devtool: upgrade: fix handling of errors parsing upgraded recipePaul Eggleton
As part of upgrading a recipe we create the upgraded recipe file in the workspace and then try to parse it so we can then make further modifications. If for some reason that parsing fails then the failure was not being handled very well - the broken recipe was being left in place, breaking parsing until it was removed by hand. Fix that by adding a call to the cleanup function, and fix the following issues: * Fix the cleanup function which doesn't look like it has ever worked due to a typo in the function call * Fix double-printing the error message * Remove usage of DevtoolError in this case (DevtoolError is for simple usage errors, not this kind of issue which may be the result of a bug). We're still printing a traceback in this scenario but at least it doesn't break the build system requiring manual cleanup. I also introduced a command-line option to preserve the broken upgraded recipe file(s) for debugging purposes. (The reproducer for this is "devtool upgrade libnewt-python", however you need to check out revision b82ea144e144671d3f64c0785ba4beafe905cd4f or earlier since that recipe has now been absorbed into the libnewt recipe. The libnewt-python recipe was causing an issue with the upgrade because it actually included the libnewt recipe using ${PV} in the include statement, and of course PV was changing in the upgrade.) Fixes [YOCTO #13404]. (From OE-Core rev: c519ac360796675d7fc09a5250d21f0f5b6236fc) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-07-03recipetool: ignore zero-length setup.py filesPaul Eggleton
If a setup.py file exists it ought to have something in it before we consider the source tree to be a Python module and treating it as such. (A counter-example is https://www.bro.org/downloads/binpac-0.50.tar.gz - it's not clear why this has a zero-length setup.py in it but we should pay no attention to it.) Fixes [YOCTO #12923]. (From OE-Core rev: 548a5c8f42c6ac1b0f7962926d05276e71505678) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-30package: Build pkgdata specific to the current recipeRichard Purdie
This switches the code to build pkgdata specific to the current recipe which means that its filtered to the recipes dependencies and can perform better as we can drop the lockfile. It uses a similar method to the staging code to do this, using BB_TASKDEPDATA to construct a list of packagedata task output which this recipe should "see". The original pkgdata store is left unaltered so existing code works. The lock file was there to prevent files disappearing as they were read or as directories were listed. Since we have a copy of the data and only access output from completed tasks (as per their manifests), we can remove the lock. The lock was causing starvation issues on systems with parallelism. There was also a potential determinism problem as the current code could "see" data from recipes which it doesn't depend upon. [YOCTO #13412] (From OE-Core rev: 1951132576bfb95675b4879287f8b3b7c47524fa) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-30wic: python2 -> python3Robert Yang
(From OE-Core rev: 9303f92599343adf645fee5d2434fadd97e7febb) Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-28devtool: warn user about multiple layer having the same base nameChen Qi
Currently `devtool finish RECIPE meta' will silently succeed even if there are multiple layers having the same base name of 'meta'. e.g. meta layer from oe-core and meta layer from meta-secure-core. We should at least give user a warning in such case. With the patch, we will get warning like below. WARNING: Multiple layers have the same base name 'meta', use the first one '<PROJ_DIR>/oe-core/meta'. WARNING: Consider using path instead of base name to specify layer: <PROJ_DIR>/oe-core/meta <PROJ_DIR>/meta-secure-core/meta (From OE-Core rev: 2c8740f543c38dbaef3345e40827ef48b3f75405) Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-19wic/bootimg-efi: allow multiple initrdChee Yang Lee
Allow plugin bootimg-efi to configure with multiple initrd through source parameter. Uses ; to separate each initrd. e.g: --sourceparams="loader=${EFI_PROVIDER},initrd=initrd1;initrd2" (From OE-Core rev: c7b0823f9ab6a9959aaa809b8c3f70d199feb64d) Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-12resulttool/merge: Enable control TESTSERIES and extra configurationsYeoh Ee Peng
Current QA team need to merge test result files from multiple sources. Adding TESTSERIES configuration too early will have negative implication to report and regression. Enable control to add TESTSERIES when needed. Also enable adding EXECUTED_BY configuration when needed. (From OE-Core rev: 651d8d371e78e77599d56681228d5782664f7743) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-12resulttool/store: Enable add EXECUTED_BY config to resultsYeoh Ee Peng
Current results stored does not have information needed to trace who executed the tests. Enable store to add EXECUTED_BY configuration to results file in order to track who executed the tests. (From OE-Core rev: dca2a57d54163a2e63b06e2f140fea3bd49cef0d) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-12resulttool/resultutils: Enable add extra configurations to resultsYeoh Ee Peng
Current resultutils library always add "TESTSERIES" configuration to results. Enhance this to allow control of adding "TESTSERIES" configuration as well as allow adding extra configurations when needed. (From OE-Core rev: 443c0acc14ef2451b10878fc83dd11b46805daf0) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-11wic/plugins: kernel image refer to KERNEL_IMAGETYPEChee Yang Lee
replaced hardcoded kernel image with KERNEL_IMAGETYPE. set kernel image to "bzImage" incase KERNEL_IMAGETYPE not set. (From OE-Core rev: 88a9fef761c5e67b2964fedc85a7e8ad37067564) Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-10recipetool: add python3 supportMaciej Pijanowski
Add support for generating python3 recipes using the recipetool / devtool. Drop python2 support at the same time. Tested with: oe-selftest -r recipetool.RecipetoolTest [YOCTO #13264] (From OE-Core rev: d8b2f58974482b3b1ccc65c5f93104d0d7ba87bc) Signed-off-by: Maciej Pijanowski <maciej.pijanowski@3mdeb.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-08wic/filemap: handle FIGETBSZ failingRoss Burton
Some file systems don't support fetching the block size (notably the file system Docker uses for containers), so handle the iotctl() failing and raise the expected error. (From OE-Core rev: 3757073726a00c5250556aae3d0daac76b88085e) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-07wic/engine: include .wks.in in wic search and listChee Yang Lee
allow wic to list and search for kickstart file in .wks.in extension. basename show by wic list images to fully exclude extension. (From OE-Core rev: 2c0a292a790ad069648e37b1b29fcea656fcf3e4) Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04resulttool: Remove prints if no tests occurJon Mason
Printing the lack of a test is not necessary (per feedback). Remove this from the template to quieten it. (From OE-Core rev: b1fe6ae66360e160eeaeafe456536f335a0eab60) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04resulttool/manualexecution: Enable creation of test case configurationsangeeta jain
Allow the creation of test case configuration file based on user inputs. Where this testcase configuration file will be used by the the manual execution to run selected test cases for a module rather than compulsory run all test cases in manual json file. (From OE-Core rev: 73d2a747c17779da0ca972da776b3cf02c2e1cbc) Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-06-04resulttool: modify to be multi-machineJon Mason
Currently, the code will sum all of the different machine results into a single report of the tests results. This can lead to confusion as to which machine may be experiencing issues. Modify the code to store the results in a per machine basis and report them accordingly. (From OE-Core rev: 16d4031ea5df8a4ddfdb937d35464c09e1abd10e) Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-30wic: bootimg-efi: add label source parameterChee Yang Lee
Add new source parameter label to allow custom boot.conf/grub.cfg label, so far it's hardcoded to "Boot". Default label to "Boot" for systemd-boot and blank for grub-efi when source parameter label are not set. (From OE-Core rev: 7a0aab1aa31e66e6bc94c04c2f6c1043b64a8967) Signed-off-by: Chee Yang Lee <chee.yang.lee at intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-15wic/bootimg-efi: replace hardcoded volume name with labelChee Yang Lee
volume name should refer to --label in .wks. Replace the hardcoded volume name with label. set "ESP" as default name when no lable specified. (From OE-Core rev: 5621aceaf39ef0dc097b16c83e73b9882c987a7c) Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-09meta/lib+scripts: Convert to SPDX license headersRichard Purdie
This adds SPDX license headers in place of the wide assortment of things currently in our script headers. We default to GPL-2.0-only except for the oeqa code where it was clearly submitted and marked as MIT on the most part or some scripts which had the "or later" GPL versioning. The patch also drops other obsolete bits of file headers where they were encoountered such as editor modelines, obsolete maintainer information or the phrase "All rights reserved" which is now obsolete and not required in copyright headers (in this case its actually confusing for licensing as all rights were not reserved). More work is needed for OE-Core but this takes care of the bulk of the scripts and meta/lib directories. The top level LICENSE files are tweaked to match the new structure and the SPDX naming. (From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08resulttool: Add option to dump all ptest logsJoshua Watt
Adds an option to dump all the ptest logs to individual files in a specified directory. If multiple test runs are present, the '--prepend-run' argument will create separate directories for each test run under the target directory and put the logs there to prevent each test run from clobbering the others. [YOCTO #13331] (From OE-Core rev: f262b9e7482e38f04dfa162f0aaf65b17e86f8b2) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-05-08recipetool: fix unbound variable when fixed SRCREV can't be foundAlex Kiernan
If attempting to find a fixed SRCREV fails because the directory doesn't exit, avoid failing with: Traceback (most recent call last): File "/home/vagrant/poky/scripts/recipetool", line 121, in <module> ret = main() File "/home/vagrant/poky/scripts/recipetool", line 110, in main ret = args.func(args) File "/home/vagrant/poky/scripts/lib/recipetool/create.py", line 707, in create_recipe srcrev = stdout.rstrip() UnboundLocalError: local variable 'stdout' referenced before assignment Fixes: 000480c42797 ("recipetool / devtool: set a fixed SRCREV by default when fetching from git") (From OE-Core rev: 8e4b301573071ea04911b0df6533c25bc4596b93) Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-29resulttool: add LTP compliance sectionArmin Kuster
(From OE-Core rev: a680d7d15fafbecf4edce9a29cc4eda16c11fd94) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-29resulttool: add ltp test supportArmin Kuster
(From OE-Core rev: 966795aa2c6960aca11a04e87f415256faf26957) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26resulttool/manualexecution: Enable test case configuration optionsangeeta jain
Current manualexecution required user to exceute all test cases defined inside a "modulename.json" file in oeqa/manual There are cases when all test cases all not required to run for a module on specific DUT. Enable manualexecution to have the optional feature where it will use pre-defined json format test case configuration file where user will be able to select test cases from the "modulename.json" instead of running all of them. This will help in reducing testing time and reporting unneccesary skip or failures. Example pre-defined json format test case configuration file (for build-applince): { "testcases" : [ "build-appliance.build-appliance.Create_core-image-sato-sdk_using_build_appliance", "build-appliance.build-appliance.Build_a_image_without_error_(added_recipe)" ] } (From OE-Core rev: 3950c28a34f94b5907d37b579bdaee5a59794652) Signed-off-by: sangeeta jain <sangeeta.jain@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26Revert "wic: Set a miniumum FAT16 volume size."Angus Lees
This reverts commit f7dfb4d43247d3c13a4e0a3853007d63b9512b83. FAT16 volumes do not have a minimum size of 8250 blocks (== slightly over 4MB). Exhibit A: floppy disks. The original commit message suggests this was a workaround for a bug in parted - in which case we should fix it there, or use the wic `--fixed-size` option to pass down an enforced minimum from a more context-aware point in the callstack. (From OE-Core rev: e0a1ae3863bc229512d43e4a5248551bf25950e5) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-26resultool/resultutils: Fix module import errorRichard Purdie
Fix AttributeError: module 'urllib' has no attribute 'request' when using remote http urls. (From OE-Core rev: 9acc47236e2a9ac7f97f80040f5e508f23f01611) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-25resulttool: Add log subcommandJoshua Watt
Adds a subcommand for dumping various logs from test results (From OE-Core rev: 454b8d2cdc6f79a51e610dae92c22352850c3f7c) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2019-04-25resulttool: Load results from URLJoshua Watt
Adds support for resulttool to load JSON files directly from a http:// or https:// URL (From OE-Core rev: 235bcf0c504e7ea253ccfb28d600898117c64c1f) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>