Age | Commit message (Collapse) | Author |
|
There are scenarios in which these are useful for use by other
parts of this system, not least of all the next patch which uses
them to compose a manifest for bundle-based images.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
The code in 5b48b103b9b940232cf65699db2adcab9c224124 fails to ever
delete the manifest symlinks, which causes errors on builds beyond
the first. The reason for this is that os.path.exists() on a broken
symlink returns False.
Fix this by trying to remove the manifest symlink first and then
removing the linked to file.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
Add the oe-swupd-helpers base dependency to
SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS in order to prevent the bash recipes
architecture taining the sstate of allarch oe-swupd-helpers.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
We now have HOWTO.md to describe usage of the layer in detail.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
HOWTO.md describes how to use the layer, what the known limitations
are and some of the design decisions we made when implementing the
class.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
As we don't actually generate any images for the bundle images
(only the rootfs) writing an image manifest to the deploy directory
isn't very useful.
Add a task to remove the image manifests, ideally we'd just unset
IMAGE_MANIFEST in swupdbundle.bbclass but that isn't fully supported
in write_image_manifest() of rootfs-postcommands.bbclass in OE-Core.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
tar's option --preserve-permissions must be used on both sides
of the pipe: when creating an archive and when extracting it.
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
|
|
Pseudo does not handle xattrs on hardlinks correctly. We work around
that for now by replacing hardlinks with symlinks. Only affects a few
entries, for example /sbin/e2fsck.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
|
|
To keep the "os-core" bundle small, the base image should be as
minimal as possible. It might not even allow logging in. In such a
setup, actual images have to have additional software installed, which
in a swupd-based image implies installing additional bundles.
The new SWUPD_IMAGES variable can be used to request the creation of
additional virtual image recipes which, if building them gets
explicitly requested, create a suitable rootfs by copying from the
mega image rootfs and then applying the normal image creation.
In contrast to the internal bundle-<baseimage>-<bundle> recipes, the
SWUPD_IMAGES are user-visible, so they are named
<baseimage>-<imageextension> where <imageextension> is the name given
in SWUPD_IMAGES.
To make this work, some changes are required in the code:
- the special swupd methods need to ignore the virtual image
recipes which produce images
- MEGA_IMAGE_ROOTFS needs to be set for all image recipes
- /usr/share/clear/bundles is populated consistently in all
rootfs directories (including os-core and also for the mega
rootfs)
- manipulating the rootfs must be split into code which sets
up the clear/bundles directory (can and should be in do_rootfs)
and copying files (should better be in do_image, because it needs
to replace work done there) and the latter must be available
as an utility function because the new virtual image recipes
replace the entire do_image with it
- the shared pseudo database can no longer be removed, because the
image creation also depends on it, and it is unknown when that is
done (might even be in a separate, future bitbake invocation)
The "swupdimage.bbclass" name is consistent with
"swupdbundle.bbclass", but can easily be confused with
"swupd-image.bbclass". The goal is to rename "swupd-image.bbclass" to
just "swupd.bbclass".
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
|
|
All that we need from virtual image recipes is the output of do_image.
That will run regardless whether there are image types defined.
Instead of producing image files and removing them later (which, besides
wasting time and IO bandwidth, was also not done completely when using
this class in Ostro OS) we can skip the entire image file creation
by unsetting IMAGE_FSTYPES.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
|
|
Having virtual recipes named "${PN}-<bundle>" (example:
ostro-image-iotivity for ostro-image.bb) is potentially problematic,
because there might also be a real recipe with that name
(ostro-image-iotivity.bb).
Now the virtual recipes use "bundle-" as prefix in their name.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
|
|
Convert do_copy_bundle_contents to python and use the same
functions for creating the swupd bundle manifest and copying
a tree with xattrs included.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
We should also call this when copying bundle directories to ensure
behaviour is consistent.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
This prevents swupd-server failing with an error when it can't find
groups.ini if SWUPD_BUNDLES is unset.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
Backport a patch from swupd-server upstream to ensure that xattrs
are always used when calculating file hashes. This is the behaviour
swupd-client expects.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
Create a custom copy of oe.path.copytree() which also copies any
extended attributes (xattr) on files and directories in the tree.
Use this new copyxattrtree() function to copy the mega rootfs
contents in do_rootfs_append ().
To ensure that we have access to a version of GNU tar which supports
including xattr (1.27 or newer) we add a DEPENDS on
tar-replacement-native.
Note: this change removes the postfunc to delete the pseudo database
as the database is required in order to know what the xattr are and
life-cycles aren't as clear cut as we'd like (i.e. if the build is
run in stages by calling individual tasks).
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
This parameter became mandatory in bitbake commit
fab717d303df0bcef737661f6917f275f35215a4
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
These changes don't affect the functionality but improve the code
and improve maintainability.
* remove a redundant comment
* keep the do_swupd_update task and its varflags together in the code
* remove a spurious double copy of the os-core bundle contents. This is now
handled by the do_copy_bundle_contents task.
* move os-core pack generation into the loop which handles other bundles,
this ensures all calls are the same regardless of bundle and aides future
refactoring.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
|
|
If bundles haven't been defined for an image which inherits
swupd-image there's no point in creating a mega image, which should
be the same as the base image. Add some logic to avoid creating the
mega image when SWUPD_BUNDLES is unset to save time and space.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
Fix reporting of the previous release which the update is being
generated against in a debug statement by correctly expanding the
PREVREL shell variable.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
It should be possible to use swupd-image to create images with an
update stream regardless of whether bundles have been defined for
the image or not.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
The last push included some commented out code which should have
been removed before pushing, tidy that up now.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
The bundle contents and the base image's root filesystem should be populated
from the mega image's rootfs in order to ensure that fully populated
versions of any files which undergo post-processing during rootfs/image
construction (i.e. /etc/passwd) are included.
[YOCTO #9193]
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
If a user were to define a bundle named 'mega' the logic for
creating the mega-image (base image + all bundles) would become
confused. Therefore raise an error if a user-defined bundle is
named mega.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
The virtual images are created as a side-effect of the swupd
update — we don't need the base image + bundle contents images
any more once the bundle contents have been extracted into the
swupd bundle directories, therefore delete them in order to
reduce disk footprint.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
We need to use a shared pseudo database whilst all of the virtual
images are created and the swupd post-processing is performed so
that the permissions are correct across recipes, however we also
need to be careful about the database's lifecycle to ensure that
operations run outside of pseudo control don't cause the database
to become invalidated.
As the do_swupd_image task only runs for the base image *and* runs
after all of the virtual images have been created we can safely
clean up the database once it's finished — add a task to do so.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|
|
This initial layer version provides an initial set of metadata to enable
integration of the swupd (https://clearlinux.org/features/software-update)
software updater into an image.
Approach:
An image that inherits the swupd-image bbclass will automatically have
bundle 'chroots' created which contain the filesystem contents of the
specified bundles, with the contents of the inheriting image forming the
default os-core bundle.
The mechanism to achieve this is that several virtual image recipes are
created using the swupdbundle class, one for each defined bundle plus a
'mega' image recipe. The 'mega' image contains the base image plus the
contents of all of the bundles, whilst bundle images contain only the
base image plus the contents of a single bundle.
We build the mega image first, then the base image (the one which
inherits this class) and finally all of the bundle images. Each
non-mega image has a manifest generated that lists the file contents
of the image.
We took the approach of building images, rather than populating the
chroot-like bundle directories with a package manager, because various
layers and recipes make changes to the rootfs contents outside of the
package manager, particularly with IMAGE_POSTPROCESS_COMMAND, etc.
Once the images and their manifests have been created each bundle image
manifest is compared to the base image manifest in order to generate a list
of files in the bundle image which don't exist in the base image.
Files in this list are then preserved in the bundle directory for
processing by swupd-server in order to generate update artefacts.
Finally the binaries from swupd-server are called on the bundle directories
to generate the artefacts for consumption by a swupd client.
How to:
* inherit the swupd-image class in your core OS image. swupd-based OS's use
bundles, the primary one of which, os-core, is defined as the contents of
this image.
* Assign a list of names for bundles you wish to generate to the
SWUPD_BUNDLES variable i.e.
SWUPD_BUNDLES = "feature_one feature_two"
* Assign a list of packages for which their content should be included in a
bundle to a varflag of BUNDLE_CONTENTS which matches the bundle name i.e.
BUNDLE_CONTENTS[feature_one] = "package_one package_three package_six"
* Ensure the OS_VERSION variable is assigned an integer value and increased
before each image build which should generate swupd update artefacts.
This variable must echo the same version number as is used to set the
VERSION_ID field of os-release as swupd-client will use it to check for
updates.
* Publish the contents of
${DEPLOY_DIR}/swupd/${MACHINE}/${IMAGE_BASENAME}/www on a server for
consumption by swupd-client
* Use swupd client sub-commands with the -u argument pointing to the
contents published above
Known issues:
* shared pseudo database: the bundle chroot-like directories are generated
per-recipe and processed by a task of the inheriting recipe. In order for
the files generated outside of the base recipe to have correct
permissions when processed by swupd-server we need to share a pseudo
database across the recipes.
This database is currently never cleaned up, which is likely to cause
headaches due to the way pseudo operates on inodes that could be reused
outside of pseudo's influence. We have yet to determine an appropriate
time to perform housekeeping on this database (we essentially need the
database to be removed when DEPLOY_DIR_SWUPD is removed).
* oe-swupd-helpers: this recipe provides stub implementations only of some
swupd-client helpers. Anyone wishing to utilise swupd in a deployed image
will need to at least override kernel_updater.sh and
systemdboot_updater.sh.
* hard-coded paths: swupd assumes Clear Linux as a host OS and hard-codes
several paths to directories, programs and configuration files on Clear.
* builds a lot of images: due to the approach taken there are n+2 images
built, where n is the number of bundles defined in SWUPD_BUNDLES.
* creates a lot of duplicate files: due to the way swupd works by
processing chroot-like bundle directories on each os release we
potentially end up carrying a lot of duplicate files in DEPLOY_DIR_SWUPD.
We intend to look at using the hardlink program to replace duplicate
files in that directory with hard links in order to save disk space.
* requires far more testing: there are a lot of combinations of bundle
contents, rootfs modification commands (IMAGE_POSTPROCESS_COMMAND,
IMAGE_PREPROCESS_COMMAND, etc) that can affect the inputs to swupd and
our testing has likely missed areas of issue.
* OS_VERSION: introduces a new variable for the OS version number when we
already have a DISTRO_VERSION variable. This was done because swupd makes
various assumptions about the version number which aren't necessarily
true for traditional DISTRO_VERSION values in OE et al.
Co-authored-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
|