AgeCommit message (Collapse)Author
2016-10-03swupd-image: prevent buildhistory failurepre-sstateJoshua Lock
When buildstats detects TaskSucceeded for a do_rootfs task it will try and determine the size of the rootfs using du, if the rootfs directory isn't present the call to du fails which triggers a bb.error. Since e587c50c2639989d02d282c7a91134d5934eb042 do_rootfs for swumpdimage based virtual images has been an empty function which simply returns as soon as it's invoked, as we populate the rootfs from the mega rootfs contents in do_image. Due to this the rootfs directory wasn't yet present at the time buildstats detects the TaskSucceeded and the subsequent call to du fails. Work around this by creating an empty rootfs directory during do_rootfs in swupdimage. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-09-12swupdimage.bbclass: ensure that do_rootfs gets executedPatrick Ohly
OE-core commit 6d969bacc718e changed do_rootfs so that it creates IMGDEPLOYDIR. That change broke the creation of additional swupd images, because setting do_rootfs to empty caused the entire task to be skipped, including the evaluation of the 'cleandirs' task attribute. It remains to be seen whether that's really the desired behavior (see https://bugzilla.yoctoproject.org/show_bug.cgi?id=10256), but as it is what it is right now, we need to avoid the situation by overwriting do_rootfs with non-empty code that doesn't do anything. That way, the directory gets created. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-08-30swupd-client_git.bb: Make pinned pubkey configurableDmitry Rozhkov
SWUPD server may move to a new location where a different pubkey needs to be used and the hardcoded one won't work. This makes pinned pubkey configurable. Changes in v2: add explicit 'else' clause to the last statement of do_install_append() to avoid returning exit code 1. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-08-19swupd-client_git.bb: fix typo in config files creationDmitry Rozhkov
The patch puts intended values to the config files with swupd-client's default values. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-07-26swupd-image.bbclass: fix special code path for "no swupd bundles"Patrick Ohly
When there are no additional bundles, there's no need to create a separate "mega bundle" rootfs and several copy operations can be skipped because the rootfs created for the main image recipe already represents the final content. That particular code path is taken for SWUPD_BUNDLES = "". In some functions, the special handling of that configuration was missing, leading to: Exception: subprocess.CalledProcessError: Command 'tar --xattrs --xattrs-include='' -cf - -C None -p . | tar -p --xattrs --xattrs-include='' -xf - -C .../ostro-image-swupd/1.0-r0/rootfs' returned non-zero exit status 2 Fixes: IOTOS-1725 Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-07-12Bump swupd-client recipe to 3.6.0Igor Stoppa
The most recent version of swupd-client should now support signed manifests. For more details, refer to https://github.com/clearlinux/swupd-client Signed-off-by: Igor Stoppa <igor.stoppa@intel.com>
2016-07-12Drop use of oe.path.check_output()Joshua Lock
This was a copy-and-paste of the check_output() method of the subprocess module in order to support Python versions prior to 2.7 -- we should just use the method from subprocess directly. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-07-12swupd-image: fix swupd_replace_hardlinks() for python3Joshua Lock
Dictionaries don't have an iteritems() method in Python 3, use the items() method instead. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-06-01swupd-image: also write os-core to groups.ini fileJoshua Lock
Based on observation of Clear Linux's Mixer and the fact that swupd-server, since the addition of bundle includes in 3.20, segfaults when loading a manifest for a previous OS release that includes the os-core (the default behaviour since the addition of includes) we infer that it's previously expected and now required to include the os-core bundle in the groups.ini file. With this change generating updates beyond the first using swupd-server v3.x no longer causes swupd_create_update to segfault. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-05-13swupd-server: fix fullfiles patch for swupd-server-2.53Joshua Lock
Rebasing the fullfiles.c-work-around-pseudo-bug.patch patch for newer swupd 3.x branches prevents it from being applied cleanly to the 2.53 version. Resolve this by keeping a separate copy of the original patch for swupd-server-2.53 Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-05-13swupd-client: update to 3.5.3Joshua Lock
Add some logic to do_install which writes the default settings files for content URL, version URL and format. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-05-13swupd-server: bump to 3.2.3Joshua Lock
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-05-04swupdbundle.class: fix image features of mega bundlePatrick Ohly
The looping checking the individual bundles used the wrong variable name and thus failed to gather all additional image features from the individual bundles. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-04-27swupdbundle.bbclass: support arbitrary bundle featuresPatrick Ohly
Choosing bundle features based on some (rather arbitrary) bundle name suffix does not scale. Currently it does not support ptest-pkgs and dbg-pkgs. Adding support for those via bundle name suffix would become rather awkward. So instead we now allow (and require!) that bundle features for a bundle named "foo" are set explicitly in BUNDLE_FEATURES[foo]. The corresponding content has to be in BUNDLE_CONTENTS[foo]. There's no sanity checking that features listed there are really suitable for a bundle. Features adding content (like ptest-pkgs, dbg-pkgs, dev-pkgs) are okay, features changing the os-core are not (like debug-tweaks). This cannot be checked automatically by meta-swupd because there is no definite list of acceptable features. This is an API change. Users of meta-swupd need to be adapted. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-04-26swupd-image: remove unrequired utilities from IMAGE_INSTALLkrogothJoshua Lock
Now that we are using libarchive and bsdtar it is not necessary to ensure the full versions of gzip, bzip2, tar and xz are included in all swupd using images. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-26swupd-image: ensure bsdiff-native is in the native sysrootJoshua Lock
When using swupd-server 3.x bsdiff native is required at runtime, ensure it is available in the native sysroot as a dependency of the do_swupd_update task. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-22swupd-image: weak assign default value for SWUPD_FORMATJoshua Lock
This makes it easier to override for consumers of the class. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-21swupd-image: set weak default for SWUPD_IMAGE_SANITY_CHECKSJoshua Lock
This makes it simpler for a consumer of the layer to override or append to the value (i.e. using += as is commonly done). Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-19Update and improve documentationJoshua Lock
Move implementation details and known issues into separate files. Enhance the How to guide to discuss some of the swupd concepts and how they impact the way we construct a custom distro using swupd. Specifically discuss likely pain points and how to address them, notably: * update-alternatives * stateless vs. stateful (updating files in /etc) Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18swupd-client: recognise and update changed config files in v3.xJoshua Lock
Also apply a version of the patch from 6d5555c1 to the reciep for swupd-client v3.x Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18swupd-client: backport tar protection fix for 3.x recipeJoshua Lock
c6fc2a adds a backport of a patch submitted upstream to protect the tar command against special characters, backport the submitted patch for use in the 3.x recipes to keep our recipes as functionaly equivalent as possible. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18swupd-client: update Upstream-Status for bundle removal fixJoshua Lock
0001-manifest.c-Always-initialize-preserver-pointer-of-fi.patch is available in v3.0.0 and newer. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18swupd-client_2.87.bb: Protect tar command against special charactersDmitry Rozhkov
This patch adds escaping for special characters in file names for the tar commands used to install files into target file system. Particularly it fixes installation of nodejs libs having '#' and '@' in file names. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-04-18swupdbundle: don't try and build bootimgJoshua Lock
We already unset IMAGE_FSTYPES to prevent any images being written for the transient bundle images. As a logical follow on from that change, and to prevent issues when trying to build for a MACHINE which supports live images (due to the automatic inference of live image support via build_live() in image.bbclass), delete the do_bootimg task for bundle images. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-18Revert "swupdbundle: disable image construction differently"Joshua Lock
This reverts commit a7dddfbfb8f45989856c93c6f82f46d8a96e6f0b. We can't completely disable the do_image task because we need to allow various tasks that run during image creation which affect the rootfs to run.
2016-04-17swupd-client: recognise and update changed config filesJoshua Lock
swupd upstream expects its host OS, i.e. Clear Linux, to be stateless and thus by default ignores configuration file changes in the manifests. Add a configure option to swupd-client to not ignore these files, matching a similar change in swupd-server to ensure config files are included in the manifests. Do not enable this option by default in our recipe as we have several files in /etc which are generated/modified on first-boot and will cause a swupd verify to fail. [YOCTO #9199] Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-17swupd-client_2.87.bb: Fix bundle removalDmitry Rozhkov
The patch makes the function deduplicate_files_from_manifest() return correct value always. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-04-17swupd-client_2.87.bb: Avoid using real distro URL as defaultDmitry Rozhkov
The layer's users are supposed to create their own update repos. Not setting --url may lead to downloading updates from the default repo which is http://downloads.clearlinux.com/update Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-04-14swupdbundle: disable image construction differentlyJoshua Lock
Mark do_image and do_bootimg as noexec tasks so that they don't run but their task dependency chain is preserved. This prevents an error in swupdbundle image building when an image for the target MACHINE would result in a live image being built. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-14swupd-client_2.87.bb: improve file downloadingPatrick Ohly
Errors are now printed including the "staging" filename (and without segfaulting, which was already fixed by the previous patch), and download performance is a lot better on Ostro OS where IMA is active and affects the performance of the close() syscall. The download performance patch was also ported to current swupd master, see https://github.com/clearlinux/swupd-client/pull/42. It does not get included here because there is a chance to get it via an upstream update, whereas that is unlikely for 2.87. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-04-13swupd-image: also reserve the bundle name fullJoshua Lock
swupd generates and uses a Manifest.full which is a complete manifest of all files in all bundles of the OS. Therefore we should prevent users defining a bundle named 'full' as this would also generate a Manifest.full file and confuse future use of swupd. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-13swupd-client: Check filename for NULL before logging itDmitry Rozhkov
In many error situations a fake file struct is passed to the logger. The filename field of the struct needs to be checked for NULL before printing it to a log to avoid dereference of a NULL-pointer. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
2016-04-13docs: move HOWTO.md to docs/GUIDE.mdJoshua Lock
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-13swupd-image: implement image qa checksJoshua Lock
Implement a task which runs after image creation and executes the functions defined in SWUPD_IMAGE_SANITY_CHECKS to validate the constructed image. do_swupd_sanity_check_image() is influenced heavily by the sanity_check_conffiles method in OE-Core's sanity.bbclass. Include a single QA function to start with, disabled by default, which checks whether there are any dangling symlinks in the image — this check is based on code from the Ostro Project by Patrick Ohly. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-12swupd-server: inherit pkgconfigJoshua Lock
The configure.ac uses PKG_CHECK, inherit pkgconfig to ensure we have pkg-config built before swupd-server. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-12rsync: remove bbappend, the native variant is provided by OE-CoreJoshua Lock
We no longer need to add a BBCLASSEXTEND = "native" as this change has been integrated in the rsync recipe in OE-Core. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-07swupd-client: move 2.87 specific patches to 2.87 directoryJoshua Lock
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-06swupd-image: make the swupd format number a variableJoshua Lock
The format version isn't static, and indeed will be bumped in the next swupd release. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-06swupd-client: better align 3.3.0 recipe with 2.87 recipeJoshua Lock
Switch 3.3.0 recipe to using bsdtar, align DEPENDS, RDEPENDS and EXTRA_OECONF settings. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-06swupd-server: better align 3.1.2 recipe with 2.53 recipeJoshua Lock
Switch 3.1.2 recipe to using bsdtar and include the patch to work around hardlink extended attribute issues when running under pseudo. Move 2.53 specific patches to the swupd-server-2.53 directory. Unify DEPENDS, RDEPENDS and EXTRA_OECONF settings. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-06swupd-image.bbclass: fix remove of symlinks to directoriesPatrick Ohly
remove_unlisted_files_from_directory() failed to remove unwanted symlinks to directories because it only looked at files, whereas Python lists symlinks to directories as directories. In Ostro OS, that affected for example /usr/lib/perl5 -> perl and was found when adding a QA check for dangling symlinks. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-04-06swupd-image.bbclass: always update rootfs of base imagePatrick Ohly
As a special optimization, updating the rootfs of the base image was skipped if actual image creation was disabled for the base image. However, there are reasons beyond image creation for updating the rootfs, for example to run QA checks on it. Therefore the change gets reverted and the code now only gets skipped for the virtual bundle recipes. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-04-04swupd-client: add recipe for 3.3.0Joshua Lock
Add a new recipe to track the latest version, 3.3.0, from github. We'll keep the older version around for now whilst we do some testing. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-04swupd-server: add recipe for 3.1.2Joshua Lock
Add a new recipe to track the latest version, 3.1.2, from github. We'll keep the older version around for now whilst we do some testing. Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-04bsdiff: add new recipeJoshua Lock
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-04-04os-release: patch in OS_VERSION during rootfs creationPatrick Ohly
Changing os-release in a .bbappend is undesirable for two reasons: 1. it changes the content also for images which do not use swupd 2. it causes full rebuilds even when the only thing that changed is the OS_VERSION The latter may or may not be desired, so we should better leave that choice to the user, which is possible by setting VERSION_ID to OS_VERSION in local.conf. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-04-04swupd-image: make debugging function for long-running tasks changeableJoshua Lock
We had a request to print more of the debugging around long-running tasks to the console when using knotty using bbplain. This is undesirable as a default but this patch enables a user to opt-in to this behaviour by setting SWUPD_LOG_FN = "bbplain" producing output like: NOTE: Executing RunQueue Tasks jgl-image-swupd-1.0-r0 do_swupd_update: New OS_VERSION is 20 jgl-image-swupd-1.0-r0 do_swupd_update: Generating update from 0 to 20 jgl-image-swupd-1.0-r0 do_swupd_update: Generating fullfiles for 20 jgl-image-swupd-1.0-r0 do_swupd_update: Generating zero packs, this can take some time. jgl-image-swupd-1.0-r0 do_swupd_update: Generating zero pack for os-core jgl-image-swupd-1.0-r0 do_swupd_update: Generating zero pack for connman jgl-image-swupd-1.0-r0 do_swupd_update: Generating zero pack for server jgl-image-swupd-1.0-r0 do_swupd_update: Generating zero pack for developer NOTE: Tasks Summary: Attempted 2857 tasks of which 2582 didn't need to be rerun and all succeeded. NOTE: Writing buildhistory Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
2016-03-30swupd-image.bbclass: remove assumption about OS version numberingPatrick Ohly
It is not guaranteed that OS versions always increment in steps of 10 and that deltas are desired for "current version - 10, -20, etc.". For example, previous versions might be 10, 20, and the current one 21 because it is a minor update. In such a case no delta packs would be generated because OS versions 11 and 1 do not exist. Ostro OS outside of the CI derives the OS version from datetime, which leads to high version numbers like 33014500. Counting down from that in steps of ten in shell under pseudo was surprisingly slow and unusable, running for minutes without even getting close to completion when it does not hit existing version numbers. So now the simplified approach is to generate delta packs for exactly the last SWUPD_N_DELTAPACK older versions, regardless what their numbering is. If a more sophisticated approach is needed, it can still be added later on, for example as part of YOCTO #9321. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-03-30swupd-server: work around pseudo xattr hardlink bugPatrick Ohly
swupd-server relies on hardlinks to create a temporary directory with the original file for the external tar command. Creating hardlinks under pseudo leads to copies which (incorrectly) do not share the original file's xattrs. To work around this bug, we explicitly copy the xattrs. Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
2016-03-30swupd-client: Update the client and server to use bsdtarDmitry Rozhkov
This patch adds bsdtar support to swupd-client and swupd-server and enables it. The reason why it's done this way is that: - bsdtar works better with IMA (opens files only once and then updates content and xattrs together); - swupd remains fully functional, including xattrs support, even when a distro disables GPLv3 licensed code. Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>