diff options
Diffstat (limited to 'documentation/dev-manual/upgrading-recipes.rst')
| -rw-r--r-- | documentation/dev-manual/upgrading-recipes.rst | 397 |
1 files changed, 397 insertions, 0 deletions
diff --git a/documentation/dev-manual/upgrading-recipes.rst b/documentation/dev-manual/upgrading-recipes.rst new file mode 100644 index 0000000000..4fac78bdfb --- /dev/null +++ b/documentation/dev-manual/upgrading-recipes.rst @@ -0,0 +1,397 @@ +.. SPDX-License-Identifier: CC-BY-SA-2.0-UK + +Upgrading Recipes +***************** + +Over time, upstream developers publish new versions for software built +by layer recipes. It is recommended to keep recipes up-to-date with +upstream version releases. + +While there are several methods to upgrade a recipe, you might +consider checking on the upgrade status of a recipe first. You can do so +using the ``devtool check-upgrade-status`` command. See the +":ref:`devtool-checking-on-the-upgrade-status-of-a-recipe`" +section in the Yocto Project Reference Manual for more information. + +The remainder of this section describes three ways you can upgrade a +recipe. You can use the Automated Upgrade Helper (AUH) to set up +automatic version upgrades. Alternatively, you can use +``devtool upgrade`` to set up semi-automatic version upgrades. Finally, +you can manually upgrade a recipe by editing the recipe itself. + +Using the Auto Upgrade Helper (AUH) +=================================== + +The AUH utility works in conjunction with the OpenEmbedded build system +in order to automatically generate upgrades for recipes based on new +versions being published upstream. Use AUH when you want to create a +service that performs the upgrades automatically and optionally sends +you an email with the results. + +AUH allows you to update several recipes with a single use. You can also +optionally perform build and integration tests using images with the +results saved to your hard drive and emails of results optionally sent +to recipe maintainers. Finally, AUH creates Git commits with appropriate +commit messages in the layer's tree for the changes made to recipes. + +.. note:: + + In some conditions, you should not use AUH to upgrade recipes + and should instead use either ``devtool upgrade`` or upgrade your + recipes manually: + + - When AUH cannot complete the upgrade sequence. This situation + usually results because custom patches carried by the recipe + cannot be automatically rebased to the new version. In this case, + ``devtool upgrade`` allows you to manually resolve conflicts. + + - When for any reason you want fuller control over the upgrade + process. For example, when you want special arrangements for + testing. + +The following steps describe how to set up the AUH utility: + +#. *Be Sure the Development Host is Set Up:* You need to be sure that + your development host is set up to use the Yocto Project. For + information on how to set up your host, see the + ":ref:`dev-manual/start:Preparing the Build Host`" section. + +#. *Make Sure Git is Configured:* The AUH utility requires Git to be + configured because AUH uses Git to save upgrades. Thus, you must have + Git user and email configured. The following command shows your + configurations:: + + $ git config --list + + If you do not have the user and + email configured, you can use the following commands to do so:: + + $ git config --global user.name some_name + $ git config --global user.email username@domain.com + +#. *Clone the AUH Repository:* To use AUH, you must clone the repository + onto your development host. The following command uses Git to create + a local copy of the repository on your system:: + + $ git clone git://git.yoctoproject.org/auto-upgrade-helper + Cloning into 'auto-upgrade-helper'... remote: Counting objects: 768, done. + remote: Compressing objects: 100% (300/300), done. + remote: Total 768 (delta 499), reused 703 (delta 434) + Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done. + Resolving deltas: 100% (499/499), done. + Checking connectivity... done. + + AUH is not part of the :term:`OpenEmbedded-Core (OE-Core)` or + :term:`Poky` repositories. + +#. *Create a Dedicated Build Directory:* Run the :ref:`structure-core-script` + script to create a fresh :term:`Build Directory` that you use exclusively + for running the AUH utility:: + + $ cd poky + $ source oe-init-build-env your_AUH_build_directory + + Re-using an existing :term:`Build Directory` and its configurations is not + recommended as existing settings could cause AUH to fail or behave + undesirably. + +#. *Make Configurations in Your Local Configuration File:* Several + settings are needed in the ``local.conf`` file in the build + directory you just created for AUH. Make these following + configurations: + + - If you want to enable :ref:`Build + History <dev-manual/build-quality:maintaining build output quality>`, + which is optional, you need the following lines in the + ``conf/local.conf`` file:: + + INHERIT =+ "buildhistory" + BUILDHISTORY_COMMIT = "1" + + With this configuration and a successful + upgrade, a build history "diff" file appears in the + ``upgrade-helper/work/recipe/buildhistory-diff.txt`` file found in + your :term:`Build Directory`. + + - If you want to enable testing through the :ref:`ref-classes-testimage` + class, which is optional, you need to have the following set in + your ``conf/local.conf`` file:: + + IMAGE_CLASSES += "testimage" + + .. note:: + + If your distro does not enable by default ptest, which Poky + does, you need the following in your ``local.conf`` file:: + + DISTRO_FEATURES:append = " ptest" + + +#. *Optionally Start a vncserver:* If you are running in a server + without an X11 session, you need to start a vncserver:: + + $ vncserver :1 + $ export DISPLAY=:1 + +#. *Create and Edit an AUH Configuration File:* You need to have the + ``upgrade-helper/upgrade-helper.conf`` configuration file in your + :term:`Build Directory`. You can find a sample configuration file in the + :yocto_git:`AUH source repository </auto-upgrade-helper/tree/>`. + + Read through the sample file and make configurations as needed. For + example, if you enabled build history in your ``local.conf`` as + described earlier, you must enable it in ``upgrade-helper.conf``. + + Also, if you are using the default ``maintainers.inc`` file supplied + with Poky and located in ``meta-yocto`` and you do not set a + "maintainers_whitelist" or "global_maintainer_override" in the + ``upgrade-helper.conf`` configuration, and you specify "-e all" on + the AUH command-line, the utility automatically sends out emails to + all the default maintainers. Please avoid this. + +This next set of examples describes how to use the AUH: + +- *Upgrading a Specific Recipe:* To upgrade a specific recipe, use the + following form:: + + $ upgrade-helper.py recipe_name + + For example, this command upgrades the ``xmodmap`` recipe:: + + $ upgrade-helper.py xmodmap + +- *Upgrading a Specific Recipe to a Particular Version:* To upgrade a + specific recipe to a particular version, use the following form:: + + $ upgrade-helper.py recipe_name -t version + + For example, this command upgrades the ``xmodmap`` recipe to version 1.2.3:: + + $ upgrade-helper.py xmodmap -t 1.2.3 + +- *Upgrading all Recipes to the Latest Versions and Suppressing Email + Notifications:* To upgrade all recipes to their most recent versions + and suppress the email notifications, use the following command:: + + $ upgrade-helper.py all + +- *Upgrading all Recipes to the Latest Versions and Send Email + Notifications:* To upgrade all recipes to their most recent versions + and send email messages to maintainers for each attempted recipe as + well as a status email, use the following command:: + + $ upgrade-helper.py -e all + +Once you have run the AUH utility, you can find the results in the AUH +:term:`Build Directory`:: + + ${BUILDDIR}/upgrade-helper/timestamp + +The AUH utility +also creates recipe update commits from successful upgrade attempts in +the layer tree. + +You can easily set up to run the AUH utility on a regular basis by using +a cron job. See the +:yocto_git:`weeklyjob.sh </auto-upgrade-helper/tree/weeklyjob.sh>` +file distributed with the utility for an example. + +Using ``devtool upgrade`` +========================= + +As mentioned earlier, an alternative method for upgrading recipes to +newer versions is to use +:doc:`devtool upgrade </ref-manual/devtool-reference>`. +You can read about ``devtool upgrade`` in general in the +":ref:`sdk-manual/extensible:use \`\`devtool upgrade\`\` to create a version of the recipe that supports a newer version of the software`" +section in the Yocto Project Application Development and the Extensible +Software Development Kit (eSDK) Manual. + +To see all the command-line options available with ``devtool upgrade``, +use the following help command:: + + $ devtool upgrade -h + +If you want to find out what version a recipe is currently at upstream +without any attempt to upgrade your local version of the recipe, you can +use the following command:: + + $ devtool latest-version recipe_name + +As mentioned in the previous section describing AUH, ``devtool upgrade`` +works in a less-automated manner than AUH. Specifically, +``devtool upgrade`` only works on a single recipe that you name on the +command line, cannot perform build and integration testing using images, +and does not automatically generate commits for changes in the source +tree. Despite all these "limitations", ``devtool upgrade`` updates the +recipe file to the new upstream version and attempts to rebase custom +patches contained by the recipe as needed. + +.. note:: + + AUH uses much of ``devtool upgrade`` behind the scenes making AUH somewhat + of a "wrapper" application for ``devtool upgrade``. + +A typical scenario involves having used Git to clone an upstream +repository that you use during build operations. Because you have built the +recipe in the past, the layer is likely added to your +configuration already. If for some reason, the layer is not added, you +could add it easily using the +":ref:`bitbake-layers <bsp-guide/bsp:creating a new bsp layer using the \`\`bitbake-layers\`\` script>`" +script. For example, suppose you use the ``nano.bb`` recipe from the +``meta-oe`` layer in the ``meta-openembedded`` repository. For this +example, assume that the layer has been cloned into following area:: + + /home/scottrif/meta-openembedded + +The following command from your :term:`Build Directory` adds the layer to +your build configuration (i.e. ``${BUILDDIR}/conf/bblayers.conf``):: + + $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe + NOTE: Starting bitbake server... + Parsing recipes: 100% |##########################################| Time: 0:00:55 + Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. + Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00 + Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00 + Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00 + Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00 + +For this example, assume that the ``nano.bb`` recipe that +is upstream has a 2.9.3 version number. However, the version in the +local repository is 2.7.4. The following command from your build +directory automatically upgrades the recipe for you:: + + $ devtool upgrade nano -V 2.9.3 + NOTE: Starting bitbake server... + NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace + Parsing recipes: 100% |##########################################| Time: 0:00:46 + Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. + NOTE: Extracting current version source... + NOTE: Resolving any missing task queue dependencies + . + . + . + NOTE: Executing SetScene Tasks + NOTE: Executing RunQueue Tasks + NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded. + Adding changed files: 100% |#####################################| Time: 0:00:00 + NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano + NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb + +.. note:: + + Using the ``-V`` option is not necessary. Omitting the version number causes + ``devtool upgrade`` to upgrade the recipe to the most recent version. + +Continuing with this example, you can use ``devtool build`` to build the +newly upgraded recipe:: + + $ devtool build nano + NOTE: Starting bitbake server... + Loading cache: 100% |################################################################################################| Time: 0:00:01 + Loaded 2040 entries from dependency cache. + Parsing recipes: 100% |##############################################################################################| Time: 0:00:00 + Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. + NOTE: Resolving any missing task queue dependencies + . + . + . + NOTE: Executing SetScene Tasks + NOTE: Executing RunQueue Tasks + NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano + NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded. + +Within the ``devtool upgrade`` workflow, you can +deploy and test your rebuilt software. For this example, +however, running ``devtool finish`` cleans up the workspace once the +source in your workspace is clean. This usually means using Git to stage +and submit commits for the changes generated by the upgrade process. + +Once the tree is clean, you can clean things up in this example with the +following command from the ``${BUILDDIR}/workspace/sources/nano`` +directory:: + + $ devtool finish nano meta-oe + NOTE: Starting bitbake server... + Loading cache: 100% |################################################################################################| Time: 0:00:00 + Loaded 2040 entries from dependency cache. + Parsing recipes: 100% |##############################################################################################| Time: 0:00:01 + Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. + NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch + NOTE: Updating recipe nano_2.9.3.bb + NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb + NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano + NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually + + +Using the ``devtool finish`` command cleans up the workspace and creates a patch +file based on your commits. The tool puts all patch files back into the +source directory in a sub-directory named ``nano`` in this case. + +Manually Upgrading a Recipe +=========================== + +If for some reason you choose not to upgrade recipes using +:ref:`dev-manual/upgrading-recipes:Using the Auto Upgrade Helper (AUH)` or +by :ref:`dev-manual/upgrading-recipes:Using \`\`devtool upgrade\`\``, +you can manually edit the recipe files to upgrade the versions. + +.. note:: + + Manually updating multiple recipes scales poorly and involves many + steps. The recommendation to upgrade recipe versions is through AUH + or ``devtool upgrade``, both of which automate some steps and provide + guidance for others needed for the manual process. + +To manually upgrade recipe versions, follow these general steps: + +#. *Change the Version:* Rename the recipe such that the version (i.e. + the :term:`PV` part of the recipe name) + changes appropriately. If the version is not part of the recipe name, + change the value as it is set for :term:`PV` within the recipe itself. + +#. *Update* :term:`SRCREV` *if Needed*: If the source code your recipe builds + is fetched from Git or some other version control system, update + :term:`SRCREV` to point to the + commit hash that matches the new version. + +#. *Build the Software:* Try to build the recipe using BitBake. Typical + build failures include the following: + + - License statements were updated for the new version. For this + case, you need to review any changes to the license and update the + values of :term:`LICENSE` and + :term:`LIC_FILES_CHKSUM` + as needed. + + .. note:: + + License changes are often inconsequential. For example, the + license text's copyright year might have changed. + + - Custom patches carried by the older version of the recipe might + fail to apply to the new version. For these cases, you need to + review the failures. Patches might not be necessary for the new + version of the software if the upgraded version has fixed those + issues. If a patch is necessary and failing, you need to rebase it + into the new version. + +#. *Optionally Attempt to Build for Several Architectures:* Once you + successfully build the new software for a given architecture, you + could test the build for other architectures by changing the + :term:`MACHINE` variable and + rebuilding the software. This optional step is especially important + if the recipe is to be released publicly. + +#. *Check the Upstream Change Log or Release Notes:* Checking both these + reveals if there are new features that could break + backwards-compatibility. If so, you need to take steps to mitigate or + eliminate that situation. + +#. *Optionally Create a Bootable Image and Test:* If you want, you can + test the new software by booting it onto actual hardware. + +#. *Create a Commit with the Change in the Layer Repository:* After all + builds work and any testing is successful, you can create commits for + any changes in the layer holding your upgraded recipe. + |