aboutsummaryrefslogtreecommitdiffstats
path: root/recipes-core
AgeCommit message (Collapse)Author
2016-12-08swupd-server: allow directory -> symlink transitionPatrick Ohly
A local patch is needed for updating Ostro OS 1.0 until upstream officially supports this. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08do_swupd_update: support format changesPatrick Ohly
There are reasons for format changes, the upcoming tools update being one of them. When the format changes, swupd-image.bbclass must build two OS versions from the same rootfs: once with the old format, once with the new format. OS_VERSION is used as number for the new format, OS_VERSION - 1 for the old one. OS_VERSION must be high enough such that OS_VERSION - 1 is still available. Usually it is, but there's also a sanity check for that. When changing the format because of a change in the tools, then both old and new swupd-server are needed, so now recipe and installed files include the tool format version. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-server/client: remove 2.x versionsPatrick Ohly
They are neither used nor supported. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-server: fix build issue with libmagicPatrick Ohly
libmagic is provided by file-native and required by swupd-server-native, but because file-native is usually assumed to be provided, it won't get compiled unless we depend on the special file-replacement-native. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-client: speed up download of large files when using IMAPatrick Ohly
When IMA is active, the kernel ended up updating the file hash each time swupd wrote a chunk, because files were getting opened and closed for each chunk. Now they get opened before downloading and closed when done. Fixes: clearlinux/swupd-client/#41 Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd_create_pack: enable delta computationPatrick Ohly
The previous approaches all relied on somehow carrying additional data across from one build to the next (sstate or additional archives in the deploy directory). The new approach replaces that with downloading required content on a file-by-file basis from the normal update repo when (and not sooner) it is needed by swupd_create_pack. That works for meta-swupd because the format of the files (compressed archive created with bsdtar) is expected to be stable. If a change ever becomes necessary, some backward compatibilty mode would have to be added or deltas simply would be skipped again. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-server: avoid pseudo xattr removal bugPatrick Ohly
Not removing the directories is okay: typically we don't build incrementally, and we can remove any remaining ones before invoking swupd. Not removing a tempory directory tree may also have performance benefits, but the even better solution will be to not write the tree in the first place by calling libarchive directly. Related-to: https://bugzilla.yoctoproject.org/show_bug.cgi?id=10623 Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-server: fix delta computation with xattrsPatrick Ohly
In the case that a diff against a previous build is computed and both old and new files have the same xattr, the server failed because an internal sanity check was implemented incorrectly. Not relevant at the moment for Ostro OS because changing file content implies changing the ima.security xattr, in which case current swupd-server skips diffing entirely (changing xattrs via patching not supported). It's more relevant for Ostro OS XT, which has Smack, but not IMA. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-server: avoid segfault when nothing changedPatrick Ohly
swupd_create_fullfiles segfaulted when no new files were needed for the current build because nothing changed. Very unlikely, but can happen during testing. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-client: fix Manifest hash mismatch errorPatrick Ohly
The IMA and Smack xattrs of the downloaded Manifest files are set on the downloaded and unpacked Manifest files, while the server doesn't have them at all. They need to be ignored. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08meta-swupd: per-image swupd client configurationPatrick Ohly
The settings affecting the swupd client belong to the image, not the swupd client recipe. That way, different images can use different settings while sharing the same swupd client. Creating the bundles directory was broken in the swupd-client recipe and also not needed because swupd-image.bbclass does it, too. This will also allow implementing better update repo generation (incremental, supporting format changes, etc.) because now swupd-image.bbclass has access to the settings. The installed swupd client must match the format of the update repo for that OS_VERSION. To ensure that, swupd-image.bbclass now adds a dependency on a virtual "swupd-client-format<format number>" and suitable swupd client recipe(s) provide that. Distros then have two ways of choosing a swupd client version, should that ever be necessary: - first they need to override the per-image format default value - then set the preferred swupd client version, if there is more than one for that format TODO: installing the SSL pubkey into the image after a file change does not work. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-server: support logging to stdoutPatrick Ohly
When a swupd command fails, bitbake doesn't show what the error was because the tools only write it into an internal log file. Logging it to stderr will capture the error also in the logs shown by bitbake and thus the Jenkins CI. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-server: enable support for single input rootfsPatrick Ohly
Splitting up the "mega" image just so that the original swupd-create-update can be used unmodified creates lots of redundant file operations, which are noticably slow under pseudo. This path is meant to go upstream. For now it is included here as POC. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-server: update to 3.2.5 and use libarchive directlyPatrick Ohly
Using libarchive directly avoids one fork/exec per file in swupd-make-fullfiles, which improves performance. Several regressions in the new upstream version had to be fixed as part of the version update. The version got updated to make it easier to upstream the libarchive patch. The latest upstream version actually is 3.2.7, but that version introduces a format change. Updating to that will require further work and preparations. Luckily, the source code patches apply cleanly to 3.2.5 and 3.2.7. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08swupd-client: don't unconditionally depend on bashAndré Draszik
The swupd client itself does not depend on bash anymore since version 3.3.0. Any posix shell is fine. So let's move the runtime dependency to the appropriate place. If some layer's oe-swupd-helpers.bbappend does introduce a bash dependency, it should just state that dependency itself. As the shell now be provided by bash or busybox, also add an appropriate entry to SIGGEN_EXCLUDE_SAFE_RECIPE_DEPS. Signed-off-by: André Draszik <adraszik@tycoint.com> Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-12-08oe-swupd-helpers: convert scripts to posix shellAndré Draszik
These scripts don't do much and there's no reason for them to require bash as interpreter. Signed-off-by: André Draszik <adraszik@tycoint.com> Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-08-30swupd-client_git.bb: Make pinned pubkey configurablejoshuagl/nextDmitry Rozhkov
SWUPD server may move to a new location where a different pubkey needs to be used and the hardcoded one won't work. This makes pinned pubkey configurable. Changes in v2: add explicit 'else' clause to the last statement of do_install_append() to avoid returning exit code 1. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-08-19swupd-client_git.bb: fix typo in config files creationDmitry Rozhkov
The patch puts intended values to the config files with swupd-client's default values. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-08-10core-image-minimal-swupd: add example imageJoshua Lock
Add a simple image which turns core-image-minimal into a swupd-image, this should make it easier to start investigating and understanding meta-swupd. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-08-04swupd-server: upgrade to 3.2.5Joshua Lock
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-07-12Bump swupd-client recipe to 3.6.0Igor Stoppa
The most recent version of swupd-client should now support signed manifests. For more details, refer to https://github.com/clearlinux/swupd-client Signed-off-by: Igor Stoppa <igor.stoppa@intel.com>
2016-05-13swupd-server: fix fullfiles patch for swupd-server-2.53Joshua Lock
Rebasing the fullfiles.c-work-around-pseudo-bug.patch patch for newer swupd 3.x branches prevents it from being applied cleanly to the 2.53 version. Resolve this by keeping a separate copy of the original patch for swupd-server-2.53 Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-05-13swupd-client: update to 3.5.3Joshua Lock
Add some logic to do_install which writes the default settings files for content URL, version URL and format. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-05-13swupd-server: bump to 3.2.3Joshua Lock
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18swupd-client: recognise and update changed config files in v3.xJoshua Lock
Also apply a version of the patch from 6d5555c1 to the reciep for swupd-client v3.x Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18swupd-client: backport tar protection fix for 3.x recipeJoshua Lock
c6fc2a adds a backport of a patch submitted upstream to protect the tar command against special characters, backport the submitted patch for use in the 3.x recipes to keep our recipes as functionaly equivalent as possible. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18swupd-client: update Upstream-Status for bundle removal fixJoshua Lock
0001-manifest.c-Always-initialize-preserver-pointer-of-fi.patch is available in v3.0.0 and newer. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18swupd-client_2.87.bb: Protect tar command against special charactersDmitry Rozhkov
This patch adds escaping for special characters in file names for the tar commands used to install files into target file system. Particularly it fixes installation of nodejs libs having '#' and '@' in file names. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-04-17swupd-client: recognise and update changed config filesJoshua Lock
swupd upstream expects its host OS, i.e. Clear Linux, to be stateless and thus by default ignores configuration file changes in the manifests. Add a configure option to swupd-client to not ignore these files, matching a similar change in swupd-server to ensure config files are included in the manifests. Do not enable this option by default in our recipe as we have several files in /etc which are generated/modified on first-boot and will cause a swupd verify to fail. [YOCTO #9199] Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-17swupd-client_2.87.bb: Fix bundle removalDmitry Rozhkov
The patch makes the function deduplicate_files_from_manifest() return correct value always. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-04-17swupd-client_2.87.bb: Avoid using real distro URL as defaultDmitry Rozhkov
The layer's users are supposed to create their own update repos. Not setting --url may lead to downloading updates from the default repo which is http://downloads.clearlinux.com/update Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-04-14swupd-client_2.87.bb: improve file downloadingPatrick Ohly
Errors are now printed including the "staging" filename (and without segfaulting, which was already fixed by the previous patch), and download performance is a lot better on Ostro OS where IMA is active and affects the performance of the close() syscall. The download performance patch was also ported to current swupd master, see https://github.com/clearlinux/swupd-client/pull/42. It does not get included here because there is a chance to get it via an upstream update, whereas that is unlikely for 2.87. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-04-13swupd-client: Check filename for NULL before logging itDmitry Rozhkov
In many error situations a fake file struct is passed to the logger. The filename field of the struct needs to be checked for NULL before printing it to a log to avoid dereference of a NULL-pointer. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-04-12swupd-server: inherit pkgconfigJoshua Lock
The configure.ac uses PKG_CHECK, inherit pkgconfig to ensure we have pkg-config built before swupd-server. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-07swupd-client: move 2.87 specific patches to 2.87 directoryJoshua Lock
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-06swupd-client: better align 3.3.0 recipe with 2.87 recipeJoshua Lock
Switch 3.3.0 recipe to using bsdtar, align DEPENDS, RDEPENDS and EXTRA_OECONF settings. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-06swupd-server: better align 3.1.2 recipe with 2.53 recipeJoshua Lock
Switch 3.1.2 recipe to using bsdtar and include the patch to work around hardlink extended attribute issues when running under pseudo. Move 2.53 specific patches to the swupd-server-2.53 directory. Unify DEPENDS, RDEPENDS and EXTRA_OECONF settings. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-04swupd-client: add recipe for 3.3.0Joshua Lock
Add a new recipe to track the latest version, 3.3.0, from github. We'll keep the older version around for now whilst we do some testing. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-04swupd-server: add recipe for 3.1.2Joshua Lock
Add a new recipe to track the latest version, 3.1.2, from github. We'll keep the older version around for now whilst we do some testing. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-04os-release: patch in OS_VERSION during rootfs creationPatrick Ohly
Changing os-release in a .bbappend is undesirable for two reasons: 1. it changes the content also for images which do not use swupd 2. it causes full rebuilds even when the only thing that changed is the OS_VERSION The latter may or may not be desired, so we should better leave that choice to the user, which is possible by setting VERSION_ID to OS_VERSION in local.conf. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-03-30swupd-server: work around pseudo xattr hardlink bugPatrick Ohly
swupd-server relies on hardlinks to create a temporary directory with the original file for the external tar command. Creating hardlinks under pseudo leads to copies which (incorrectly) do not share the original file's xattrs. To work around this bug, we explicitly copy the xattrs. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-03-30swupd-client: Update the client and server to use bsdtarDmitry Rozhkov
This patch adds bsdtar support to swupd-client and swupd-server and enables it. The reason why it's done this way is that: - bsdtar works better with IMA (opens files only once and then updates content and xattrs together); - swupd remains fully functional, including xattrs support, even when a distro disables GPLv3 licensed code. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-03-30swupd-client: Update patch fixing quotes in os-releaseDmitry Rozhkov
Update 0001-Tolerate-quotes-in-os-release-files.patch to the final version that got accepted upstream. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-03-30swupd-client: Update Upstream-Status for Fix-build-failure-on-Yocto.patchDmitry Rozhkov
The patch has been accepted upstream Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-03-30swupd-client: Tweak os-release to conform ClearLinux formatDmitry Rozhkov
swupd-client checks VERSION_ID, which must match the OS_VERSION used for generating swupd bundles in the current build. Also move os-release from /etc to /usr/lib because swupd-client doesn't update files in /etc. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com> Replaced "ln -s ../usr/lib/...." with "lnr ${D}/usr/lib/..." for the sake of consistency with OE-core common practices. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-03-21swupd-server: always use xattrs when calculating hashesJoshua Lock
Backport a patch from swupd-server upstream to ensure that xattrs are always used when calculating file hashes. This is the behaviour swupd-client expects. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-03-21Remove editor backup filesJoshua Lock
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-03-14swupd-server: Add patch fixing freeing dangling pointersDmitry Rozhkov
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-02-25Initial populationJoshua Lock
This initial layer version provides an initial set of metadata to enable integration of the swupd (https://clearlinux.org/features/software-update) software updater into an image. Approach: An image that inherits the swupd-image bbclass will automatically have bundle 'chroots' created which contain the filesystem contents of the specified bundles, with the contents of the inheriting image forming the default os-core bundle. The mechanism to achieve this is that several virtual image recipes are created using the swupdbundle class, one for each defined bundle plus a 'mega' image recipe. The 'mega' image contains the base image plus the contents of all of the bundles, whilst bundle images contain only the base image plus the contents of a single bundle. We build the mega image first, then the base image (the one which inherits this class) and finally all of the bundle images. Each non-mega image has a manifest generated that lists the file contents of the image. We took the approach of building images, rather than populating the chroot-like bundle directories with a package manager, because various layers and recipes make changes to the rootfs contents outside of the package manager, particularly with IMAGE_POSTPROCESS_COMMAND, etc. Once the images and their manifests have been created each bundle image manifest is compared to the base image manifest in order to generate a list of files in the bundle image which don't exist in the base image. Files in this list are then preserved in the bundle directory for processing by swupd-server in order to generate update artefacts. Finally the binaries from swupd-server are called on the bundle directories to generate the artefacts for consumption by a swupd client. How to: * inherit the swupd-image class in your core OS image. swupd-based OS's use bundles, the primary one of which, os-core, is defined as the contents of this image. * Assign a list of names for bundles you wish to generate to the SWUPD_BUNDLES variable i.e. SWUPD_BUNDLES = "feature_one feature_two" * Assign a list of packages for which their content should be included in a bundle to a varflag of BUNDLE_CONTENTS which matches the bundle name i.e. BUNDLE_CONTENTS[feature_one] = "package_one package_three package_six" * Ensure the OS_VERSION variable is assigned an integer value and increased before each image build which should generate swupd update artefacts. This variable must echo the same version number as is used to set the VERSION_ID field of os-release as swupd-client will use it to check for updates. * Publish the contents of ${DEPLOY_DIR}/swupd/${MACHINE}/${IMAGE_BASENAME}/www on a server for consumption by swupd-client * Use swupd client sub-commands with the -u argument pointing to the contents published above Known issues: * shared pseudo database: the bundle chroot-like directories are generated per-recipe and processed by a task of the inheriting recipe. In order for the files generated outside of the base recipe to have correct permissions when processed by swupd-server we need to share a pseudo database across the recipes. This database is currently never cleaned up, which is likely to cause headaches due to the way pseudo operates on inodes that could be reused outside of pseudo's influence. We have yet to determine an appropriate time to perform housekeeping on this database (we essentially need the database to be removed when DEPLOY_DIR_SWUPD is removed). * oe-swupd-helpers: this recipe provides stub implementations only of some swupd-client helpers. Anyone wishing to utilise swupd in a deployed image will need to at least override kernel_updater.sh and systemdboot_updater.sh. * hard-coded paths: swupd assumes Clear Linux as a host OS and hard-codes several paths to directories, programs and configuration files on Clear. * builds a lot of images: due to the approach taken there are n+2 images built, where n is the number of bundles defined in SWUPD_BUNDLES. * creates a lot of duplicate files: due to the way swupd works by processing chroot-like bundle directories on each os release we potentially end up carrying a lot of duplicate files in DEPLOY_DIR_SWUPD. We intend to look at using the hardlink program to replace duplicate files in that directory with hard links in order to save disk space. * requires far more testing: there are a lot of combinations of bundle contents, rootfs modification commands (IMAGE_POSTPROCESS_COMMAND, IMAGE_PREPROCESS_COMMAND, etc) that can affect the inputs to swupd and our testing has likely missed areas of issue. * OS_VERSION: introduces a new variable for the OS version number when we already have a DISTRO_VERSION variable. This was done because swupd makes various assumptions about the version number which aren't necessarily true for traditional DISTRO_VERSION values in OE et al. Co-authored-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>