summaryrefslogtreecommitdiffstats
path: root/scripts/lib/wic
AgeCommit message (Collapse)Author
2021-11-10wic: Use os.rename instead of bb.utils.renameKhem Raj
This is not invoked with bitbake context as a result bb.utils is not visible when this function is called during image creation and builds fail e.g. NameError: name 'bb' is not defined (From OE-Core rev: df9dca9fe4dd1abfe5f3986389a8e8ff524da5d7) Signed-off-by: Khem Raj <raj.khem@gmail.com> Cc: Joshua Watt <JPEWhacker@gmail.com> Cc: Devendra Tewari <devendra.tewari@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-10-04wic/bootimg-efi: Add Unified Kernel Image optionKristian Klausen
"A unified kernel image is a single EFI PE executable combining an EFI stub loader, a kernel image, an initramfs image, and the kernel command line. [...] Images of this type have the advantage that all metadata and payload that makes up the boot entry is monopolized in a single PE file that can be signed cryptographically as one for the purpose of EFI SecureBoot."[1] This commit adds a create-unified-kernel-image=true option to the bootimg-efi plugin for creating a Unified Kernel Image[1] and installing it into $BOOT/EFI/Linux/ with a .efi extension per the the Boot Loader Specification[1][2]. This is useful for implementing Secure Boot. systemd-boot is the only mainstream bootloader implementing the specification, but GRUB should be able to boot the EFI binary, this commit however doesn't implement the necessary changes to the GRUB config generation logic to boot the Unified Kernel Image. [1] https://systemd.io/BOOT_LOADER_SPECIFICATION/#type-2-efi-unified-kernel-images [2] https://systemd.io/BOOT_LOADER_SPECIFICATION/ (From OE-Core rev: b0573f240525df561ddef6e47cb285b217d38487) Signed-off-by: Kristian Klausen <kristian@klausen.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-26wic:direct.py: ignore invalid mountpoints during fstab updateMarkus Volk
wic fstab-update creates invalid entries for partitons that are not supposed to be mounted from userspace eg u-boot partitions. The following lines were added to fstab on a rock-pi-4: /dev/mmcblk1p1 loader1 vfat defaults 0 0 /dev/mmcblk1p2 reserved1 vfat defaults 0 0 /dev/mmcblk1p3 reserved2 vfat defaults 0 0 /dev/mmcblk1p4 loader2 vfat defaults 0 0 /dev/mmcblk1p5 atf vfat defaults 0 0 /dev/mmcblk1p6 /boot vfat defaults 0 0 With this patch only valid entries should be added /dev/mmcblk1p6 /boot vfat defaults 0 0 (From OE-Core rev: 7aa678ce804c21dc1dc51b9be442671bc33c4041) Signed-off-by: MarkusVolk <f_l_k@t-online.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-19wic: keep rootfs_size as integerAlexander Kanavin
The corrected line accidentally converted it to float, which causes problems later on with python 3.10: | File "/home/alex/development/poky/scripts/lib/wic/partition.py", line 278, in prepare_rootfs_ext | os.ftruncate(sparse.fileno(), rootfs_size * 1024) | TypeError: 'float' object cannot be interpreted as an integer (From OE-Core rev: d1d260dd2d196d10379ed9e238bcb34f39f3a3b7) Signed-off-by: Alexander Kanavin <alex@linutronix.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-09-06wic: Add extra-space argumentKristian Klausen
This allows extra space to be added after the last partition and is especially useful when free space is needed for ex: adding partitions on first boot with ex: systemd-repart[1] and the image is tested in QEMU. [1] https://www.freedesktop.org/software/systemd/man/systemd-repart.html (From OE-Core rev: f81b188bcf5aa18746fd622eb7b5c0dcb0b5c93d) Signed-off-by: Kristian Klausen <kristian@klausen.dk> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-20wic: Add --no-fstab-update part optionDaniel Gomez
When embedding a rootfs image (e.g. 'rootfs-dir') as a partition we might want to keep the stock fstab for that image. In such a case, use this option to not update the fstab and use the stock one instead. This option allows you to specify which partitions get the fstab updated and which get the stock fstab. The option matches the argument you can pass to wic itself where the same action is performed but for all the partitions. Example: part /export --source rootfs --rootfs-dir=hockeycam-image --fstype=ext4 --label export --align 1024 --no-fstab-update part / --source rootfs --fstype=ext4 --label rootfs --align 1024 (From OE-Core rev: ab4c95af8ecd15dc136194ab761afae756db5803) Signed-off-by: Daniel Gomez <daniel@qtec.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-17scripts/lib/wic/help/py: "Redhat" -> "Red Hat"Robert P. J. Day
Company name is properly spelled as "Red Hat". (From OE-Core rev: cd60f20fe40bba5b80ab604ee34fbf4ec2b0b4a2) Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-08-12wic: don't forcibly pass -T defaultRoss Burton
This reverts part of oe-core eecbe62555, which was a previous attempt to solve the Y2038 problem. This is now solved centrally in e2fsprogs, so doesn't need to be dealt with in wic. We don't revert the commit entirely, to retain the warning if a filesystem has small inodes. (From OE-Core rev: 7e8017208bed98b6c90735cb641fc9d7aedf9140) Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-06-29Use the label provided when formating a dos partitionjbouchard
Previously the bootimg-pcbios wic plugin was not respecting the --label option provided from the wks file. The plugin was setting the label to 'boot'. With this fix, the --label option is use. If no option are specified, then the default is 'boot'. (From OE-Core rev: 0fd7a73c1bd2486b7a022f0f69bbcb2e0d9cb141) Signed-off-by: jbouchard <jeanbouch418@gmail.com> Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-31Add support for erofs filesystemsRichard Weinberger
Since Linux 5.4 a new read-only filesystem is available, erofs. Compared to squashfs it offers much better read performance with and without compression enabled. It suppports two optional compressors, lz4 and lz4hc. >From the mkfs.erofs man page: EROFS is a new enhanced lightweight linux read-only filesystem with modern designs (eg. no buffer head, reduced metadata, inline xattrs/data, etc.) for scenarios which need high-performance read-only requirements, e.g. Android OS for smartphones and LIVECDs. It also provides fixed-sized output compression support, which improves storage density, keeps relatively higher compression ratios, which is more useful to achieve high performance for embedded devices with limited memory since it has unnoticable memory overhead and page cache thrashing. This commit adds support for three new filesystem targets: erofs: erofs without compression erofs-lz4: erofs with lz4 compresssion enabled erofs-lz4hc: erofs with lz4hc compression enabled (From OE-Core rev: 41dead1ff8ccc49e6cd6e6f5d41a59d164693e0d) Signed-off-by: Richard Weinberger <richard@nod.at> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-05-06classes/lib/scripts: Use bb.utils.rename() instead of os.rename()Devendra Tewari
Incremental build in Docker fails with: OSError: [Errno 18] Invalid cross-device link when source and destination are on different overlay filesystems. Rather than adding fallback code to every call site, use a new wrapper in bitbake which detects this case and falls back to shutil.move which is slower but will handtle the overlay docker filesystems correctly. [YOCTO #14301] (From OE-Core rev: 656a65b2b84e7d529b89cf5de7eb838f902d84a2) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-03-02scripts/lib/wic/partition.py: do not set FAT sizeAlexander Kanavin
Modern dosfstools automatically determines the appropriate size and will error out if something that doesn't make sense is supplied on the command line. (From OE-Core rev: b85a09ea450a5e8f49418f4a930805fbb88dc83b) Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-26wic: Warn if an ext filesystem affected by the Y2038 problem is usedFlorian Bezdeka
We are getting closer and closer to the year 2038 where the 32 bit time_t overflow will happen. While products (= embedded systems) with an expected life time of 15 years are still save the situation may change if your system has to survive the next 20 years. ext2 and ext3 filesystems are always affected by the time overflow, so let's warn the user if these filesystems are still being used. If ext4 is affected depends on the inode size chosen during filesystem creation. At least 256 bytes are necessary to be safe. As ext4 is used very often (and partitions may be created small first and extended later) this might be an issue for many users. Some filesystems created during CI runs were already affected by the Y2038 problem. By using `--mkfs-extraopts "-T default"` we tell mke2fs not to auto-detect the usage type based on the filesystem size. mke2fs will use the default values for tuning parameters instead. The inode size is one of these parameters. (From OE-Core rev: eecbe625558406680121d2a7e84917fea45ea9dc) Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-02-09wic: debug mode to keep tmp directoryLee Chee Yang
files in wic tmp directory can be usefull for debugging, so do not remove tmp directory when wic create run with debugging mode (-D or --debug). also update wic.Wic.test_debug_short and wic.Wic.test_debug_long to check for tmp directory. [YOCTO#14216] (From OE-Core rev: a122e2418b67d38f691edcf8dd846c167d6b4fa9) Signed-off-by: Lee Chee Yang <Chee.Yang.Lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-20wic: Optimise fstab modification for ext2/3/4 and msdos partitionsPaul Barker
The fix for [Yocto #13994] required the rootfs directory to be copied (using hardlinks if possible) when modifying the fstab file under wic. We can optimise this copy away for filesystems where we have the tools to modify the contents of the partition image after it is created. For ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat filesystems we have the mcopy tool. So for any of these filesystems we skip the modification of the fstab file in the rootfs directory (and skip the associated copy unless it is otherwise necessary) and update the contents of fstab directly in the partition image. (From OE-Core rev: 5fb8ae0e9159597d7eaa9307a3a8543800bf9405) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-20wic: Copy rootfs dir if fstab needs updatingPaul Barker
By default, wic updates the /etc/fstab in the rootfs to include details of additional partitions described in the selected wks file. If this modification is performed in place, other tasks which create an image file from the rootfs directory (e.g. do_image_tar and do_image_ext4) will pick up the modified fstab file which would not be appropriate for those images as they do not include the additional partitions described in the wks file. wic does undo modifications to the fstab file once it has finished creating the filesystem image, however this leaves open a race condition if one of the other tasks reads the contents of the fstab file from the rootfs directory between the point where wic modifies the fstab file and the point where wic restores the files original content. This could be solved by adding a lockfile for tasks which use the rootfs directory to ensure that no other such task is reading the rootfs directory while do_image_wic is running. This would serialize several do_image_* tasks and result in slower builds, especially for large images. Another drawback of this solution is that it is hard to selectively optimise - adding lockfiles to do_image_* tasks would result in these tasks always being serialized even if no fstab modification will take place. An alternative solution is to copy the rootfs directory when fstab needs to be modified. The code to do this in wic already exists as it is needed when including or excluding content in the rootfs. This still results in an impact on build times but the copy uses hardlinks if possible (so little data is actually copied) and we can make selective optimisations to improve things. The rootfs copy will only take place if fstab modification is required (or if it was already needed to include or exclude rootfs content). We can also follow up with further optimisations after this commit. So this second solution is chosen. Fixes [Yocto #13994] (From OE-Core rev: ce682a73b7447652f898ce1d1d0416a456df5416) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-20wic: Update pseudo db when excluding content from rootfsPaul Barker
To exclude content from the rootfs, wic makes a copy (using hardlinks if possible) of the rootfs directory and associated pseudo db, then removes files & directories as needed. However if these files and directories are removed using the python functions os.remove and shutil.rmtree, the copied pseudo db will not be updated correctly. For files copied from the original rootfs, if hardlinks were used successfully when copying the rootfs this should mean that the relevant inodes can't be reused and so the risk of pseudo aborts should be avoided. However, this logic doesn't apply for directories (as they can't be hardlinked) or for files added via the '--include-path' argument (as they weren't present in the original rootfs) and so there remains some risk of inodes being reused and the pseudo db becoming corrupted. To fix this, use the 'rm' command under pseudo when removing files & directories from the copied rootfs to ensure that the copied pseudo db is updated. (From OE-Core rev: d5db7e268947f0392c2126137571a44acd29ccd6) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-20wic: Ensure internal workdir is not reusedPaul Barker
If a path is specified for the internal wic working directory using the -w/--workdir argument then it must not already exist. Re-using a previous workdir could easily result in rootfs and intermediate files from a previous build being added to the current image. (From OE-Core rev: 2e40c8d4109024ff704c5ce40d98050ca7f34dd5) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-13wic: Allow exec_native_cmd to run HOSTTOOLSPaul Barker
This allows programs from HOSTTOOLS (e.g. 'install', 'rm', 'mv', etc) to be more easily executed by wic. Without this change only programs from an actual *-native recipe built by bitbake can be executed by wic. (From OE-Core rev: 8eb186acdecfbb3151c9a0ab148358e3fe5cce39) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-13wic: Add workdir argumentPaul Barker
This allows the path for the temporary workdir used by wic to be set when running wic from bitbake or directly from the command line. (From OE-Core rev: 2e7314ac3a3cab89de93c932e8efc75de0feb0a7) Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2021-01-01wic/direct/kparser: ensure fsuuid for vfat and msdos align with formatLee Chee Yang
vfat/msdos filesystem should have fsuuid in format 0xYYYYYYYY where "0x" in front follow with 8 hexadecimal number in uppercase. In wic, when using custom fsuuid for vfat/msdos partition in wks, it is able to set the value in any length, with or without leading "0x". This can cause fsuuid missaligned when fstab updates, fstab expect exactly 10 character fsuuid for vfat/msdos partition and all in uppercase. if custom fsuuid for vfat/msdos is set, check the length and format, error if it exceed the format size. Amend it so it is align with format 0xYYYYYYYY. This is done before image create and fstab update to ensure the fsuuid are same in all followup process. if custom fsuuid length less than expected, fill in "0". [YOCTO #14161] (From OE-Core rev: d9686ae511ef10a504becfd81bfe296b788e1456) Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-20wic: Pass canonicalized paths in PSEUDO_IGNORE_PATHSPeter Kjellerstedt
(From OE-Core rev: 55618cbd58d6784a82e773f323723be6f722232f) Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-12-15wic: Introduce empty plugin to create unformatted empty partitionsDiego Sueiro
The empty wic plugin is used to create unformatted empty partitions for wic images. To use it you must pass "empty" as argument for the "--source" parameter in the wks file. For example: part foo --source empty --ondisk sda --size="1024" --align 1024 Also adds a selftest for this plugin where the 'Fstype' column from 'wic ls' should be empty for the second partition as listed in test_empty_plugin.wks. (From OE-Core rev: 77d174fc80663403ef76c5b808aafc1117d3545c) Signed-off-by: Diego Sueiro <diego.sueiro@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-11-08isoimage-isohybrid.py: Support adding files/dirsChaitanya Vadrevu
Add "payload_dir" parameter so user can add other files/dirs to the ISO. (From OE-Core rev: bbe84a33c81066fee96dd28a4009d8404b51e642) Signed-off-by: Chaitanya Vadrevu <chaitanya.vadrevu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-10-07wic: Handle new PSEUDO_IGNORE_PATHS variableRichard Purdie
Adjust wic to correctly handle the new PSEUDO_IGNORE_PATH variable and avoid inode corruption issues. (From OE-Core rev: 13500f5234361385c365c7c35e83f99435500481) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-15wic/bootimg-efi: IMAGE_EFI_BOOT_FILES variable added to separate bootimg-efi ↵Khairul Rohaizzat Jamaluddin
and bootimg-partition Due to recent changes in bootimg-efi to include IMAGE_BOOT_FILES, when both bootimg-partition and bootimg-efi occur in a single .wks and IMAGE_BOOT_FILES are defined, files listed in IMAGE_BOOT_FILES will be duplicated in both partition. Since IMAGE_BOOT_FILES are crucial for bootimg-partition, but optional for bootimg-efi, hence allowing bootimg-efi to have the option to ignore it. The new variable, IMAGE_EFI_BOOT_FILES, was added to help handle this issue. Its basic usage is the same as IMAGE_BOOT_FILES. Usage example: ${IMGDEPLOYDIR}/${IMAGE_BASENAME}-${MACHINE}.ext4;rootfs.img \ This commit is also squashed with the updated testcase to cover for this change. [YOCTO #14011] (From OE-Core rev: 945339e06b273df1935cfd784f548ef57e0b7f4c) Signed-off-by: Khairul Rohaizzat Jamaluddin <khairul.rohaizzat.jamaluddin@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-09-02wic: misc: Add /bin to the list of searchpathsVijai Kumar K
/bin is also a valid path where one can find executables. Add that to the search path. (From OE-Core rev: ca0a6025351cb2135e87cecf828633cf12aa34c6) Signed-off-by: Vijai Kumar K <vijaikumar.kanagarajan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-22wic: Add 512 Byte alignment to --offsetJoshua Watt
Allows the --offset argument to use the "s" or "S" suffix to specify that it is reporting the number of 512 byte sectors. This is required for some SoCs where the mask ROM looks for an item at a sector that isn't aligned to a 1KB boundary. (From OE-Core rev: 938595d1dc4abaf5f7f3a7900add3f0492b805d0) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-08-06wic: fix typoDaniel Ammann
(From OE-Core rev: 9ab4c4571d899d2eefcd2fc53af0851863f29008) Signed-off-by: Daniel Ammann <daniel.ammann@bytesatwork.ch> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-25wic/bootimg-efi: Add support for IMAGE_BOOT_FILESKhasim Mohammed
List of files defined using IMAGE_BOOT_FILES are installed into the boot partition when preparing an image using the wic tool with the bootimg-efi source plugin. The code snippet introduced is taken as is from bootimg-partition.py Change-Id: I8dbb6b4e7c24870f587a6f31e6ae4a87d7033782 Issue-Id: PLATFORMS-3134 (From OE-Core rev: a44ab3a4ee5b3c57812909c6194456f299d6ba7f) Signed-off-by: Khasim Syed Mohammed <khasim.mohammed@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-18wic/filemap: Fall back to standard copy when no way to get the block mapKevin Hao
For some filesystems, such as aufs which may be used by docker container, don't support either the SEEK_DATA/HOLE or FIEMAP to get the block map. So add a FileNobmap class to fall back to standard copy when there is no way to get the block map. [Yocto #12988] (From OE-Core rev: 7934ed49179242f15b413c0275040a3bb6b68876) Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-18wic/filemap: Drop the unused get_unmapped_ranges()Kevin Hao
This method is not used by any code, so drop it. (From OE-Core rev: 6513fd9302b9989f97fc9d95e76e06ad5d266774) Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-07-18wic/filemap: Drop the unused block_is_unmapped()Kevin Hao
This method is not used by any code, so drop it. (From OE-Core rev: fb62a15349597ee026c67a0bb0a6ca2cc9bfe420) Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-16wic: Fix error message when reporting invalid offsetJoshua Watt
The error message was reporting the calculated offset instead of the current offset, which made it confusing. (From OE-Core rev: 2be775cfe1b49ce3889b5dc326e2b67a9667f18a) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-12wic: Fix --extra-space argument handlingJoshua Watt
467f84e12b ("wic: Add --offset argument for partitions") broke the --extra-space argument handling in wic. Fix the option and add a unit test for the argument. (From OE-Core rev: 87722a92c18f94917c8f70afc8cd0763462a5c25) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-06-04wic: Add --offset argument for partitionsJoshua Watt
Add support for an --offset argument when defining a partition. Many SoCs require that boot partitions be located at specific offsets. Prior to this argument, most WKS files were using the --align attribute to specify the location of these fixed partitions but this is not ideal because in the event that the partition couldn't be placed in the specified location, wic would move it to the next sector with that alignment, often preventing the device from booting. Unlike the --align argument, wic will fail if a partition cannot be placed at the exact offset specified with --offset. Changes in V2: * Fixed a small typo that prevented test_fixed_size_error from passing (From OE-Core rev: 467f84e12b96bc977d57575023517dd6f8ef7f29) Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: rootfs: Combine path_validation in one functionRicardo Ribalda Delgado
Combine all the common path validation in a function to avoid code duplication. Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: ebd12b10d17db0b4176b0188407d7e9b8420eab1) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: root: Add an opt. destination on include-pathRicardo Ribalda Delgado
Allow specifying an optional destination to include-path and make the option aware of permissions and owners. It is very useful for making a partition that contains the rootfs for a host and a target Eg: / -> Roofs for the host /export/ -> Rootfs for the target (which will netboot) Although today we support making a partition for "/export" this might not be compatible with some upgrade systems, or we might be limited by the number of partitions. With this patch we can use something like: part / --source rootfs --fstype=ext4 --include-path core-image-minimal-mtdutils export/ --include-path hello on the .wks file. Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: e8c21c6ebaebde88151697381bdb2452f1171090) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: misc: Do not find for executables in ASSUME_PROVIDEDRicardo Ribalda Delgado
Executables like tar won't be available on the native sysroot, as they are part of the ASSUME_PROVIDED variable. Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: 2f574d535f8665b26dab65c14668cf8fc7b751c0) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: Avoid creating invalid pseudo directoryRicardo Ribalda Delgado
If the source of the rootfs is not a bitbake cooked image, or it is not pointing to the root of one, we call pseudo again, which will produce a new pseudo folder at rootfs/../pseudo Eg: part /etc --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/home --fstype=ext4 Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: 799a24ae78655f7a3eda7456b1a0ffaf3e43ec16) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: Continue if excluded_path does not existRicardo Ribalda Delgado
If an excuded path does not exist, continue without an error. This allows to seamleasly reuse .wks among different projects. Eg: part / --source rootfs --fstype=ext4 --exclude-path=opt/private_keys Where /opt/private_keys in only populated by some of the image.bb files. Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: 2dbd692b8e563cf991fb4ae1ef6129fda0d7e3c4) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: Add --change-directory argumentRicardo Ribalda Delgado
This option allows to specify which part of a rootfs is going to be included, the same way the -C argument on tar. Thanks to this option we can make sure the permissions and usernames on the target partition are respected, and also simplify the creation of splitted partitons, not neeting to invoke external vars or using .wks.in files. Eg: part / --source rootfs --ondisk sda --fstype=ext4 --exclude-path=etc/    part /etc --source rootfs --fstype=ext4 --change-directory=etc Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: 2265d089a58e1f78f26d623ee667c420cb1c3bd4) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: Fix multi images .wks with bitbakeRicardo Ribalda Delgado
In order to support .wks files with multiple images inside bitbake we need to explicitly set the pseudo database in use. Eg: If we try this .mks: part / --source rootfs --ondisk sda --fstype=ext4 part /export --source rootfs --rootfs=core-image-minimal-mtdutils --fstype=ext4 The username for all the files under /export will be set to the runner of bitbake (usually UID 1000). Before we run wic, we need to make sure that the pseudo database will be flushed, and contains all the data needed. Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: dde90a5dd2b22a539095d1bac82acc15c6380ac8) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: Fix permissions when using exclude or include pathRicardo Ribalda Delgado
When parameters include_path or exclude_path are passed to the rootfs plugin, it will copy the partition content into a folder and make all the modifications there. This is done using copyhardlinktree(), which does not take into consideration the content of the pseudo folder, which contains the information about the right permissions and ownership of the folders. This results in a rootfs owned by the user that is running the wic command (usually UID 1000), which makes some rootfs unbootable. This bug can be easily triggerd with the following .wks part / --source rootfs --fstype=ext4 --exclude-path=home And this sequence: $ wic create test-permissions -e core-image-minimal -o test/ $ sudo mount test/test-permissions-202004080823-sda.direct.p1 /mnt $ ls -la /mnt/etc/shadow To fix this we copy the content of the pseudo folders to the new folder and modify the pseudo database using the "pseudo -B" command. If the rootfs is not a rootfs generated by bitbake a warning is shown making the user aware that the permissions on the target might not match what he expects. WARNING: /tmp/test/../pseudo folder does not exist. Usernames and permissions will be invalid Cc: Paul Barker <pbarker@konsulko.com> (From OE-Core rev: fd739c15cdba221d9d497d3402b7d929c0be2ca4) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-26wic: use Filesystem UUID when expand swap partitionLee Chee Yang
part.get("uuid") return the Partition UUID from the partition table instead of Filesystem UUID. This lead to swap partition UUID not match/change when wic write expand swap partition. change it to read the filesystem UUID using blkid. The output from blkid should looks like this: wic-partvzhiwq3s: LABEL="swap1" UUID="04e55c19-3f3f-4491-9e32-44eea8daa827" VERSION="1" TYPE="swap" USAGE="other" [YOCTO #13313] (From OE-Core rev: 683297eefebe83f848daad9927871242ab28ef91) Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07Revert "wic: Fix permissions when using exclude or include path"Richard Purdie
This reverts commit 36993eea89d1c011397b7692b9b8d61b499d0171. After discussion on the mailing list it was felt these changes were not ready yet. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-07Revert "wic: Add --embed-rootfs argument"Richard Purdie
This reverts commit efdcf94801f6abe8e4099e324d9a3deccd8d4384. After discussion on the mailing list it was felt these changes were not ready yet. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06wic: Add --embed-rootfs argumentRicardo Ribalda Delgado
This option adds the content of a rootfs on a specific location on the rootfs. It is very useful for making a partition that contains the rootfs for a host and a target Eg: / -> Roofs for the host /export/ -> Rootfs for the target (which will netboot) Although today we support making a partition for "/export" this might not be compatible with some upgrade systems, or we might be limited by the number of partitions. With this patch we can use something like: part / --source rootfs --embed-rootfs target-image /export --embed-rootfs target-image2 /export2 on the .wks file. (From OE-Core rev: efdcf94801f6abe8e4099e324d9a3deccd8d4384) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-06wic: Fix permissions when using exclude or include pathRicardo Ribalda Delgado
When parameters include_path or exclude_path are passed to the rootfs plugin, it will copy the partition content into a folder and make all the modifications there. This is done using copyhardlinktree(), which does not take into consideration the content of the pseudo folder, which contains the information about the right permissions and ownership of the folders. This results in a rootfs owned by the user that is running the wic command (usually UID 1000), which makes some rootfs unbootable. To fix this we copy the content of the pseudo folders to the new folder and modify the pseudo database using the "pseudo -B" command. (From OE-Core rev: 36993eea89d1c011397b7692b9b8d61b499d0171) Signed-off-by: Ricardo Ribalda Delgado <ricardo@ribalda.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2020-04-02wic: align layer plugin pathLee Chee Yang
wic look for external layer source plugin under scripts/lib/ while other tools look for lib/. allow wic to check for source plugin at both scripts/lib/ and lib/ to align with other tools while avoid breaking any existing source plugin in external layer. [YOCTO #13056] (From OE-Core rev: 986baff26bd96a6265f5fe2d631818fff9f66374) Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>