diff options
Diffstat (limited to 'documentation/dev-manual/dev-manual-common-tasks.xml')
| -rw-r--r-- | documentation/dev-manual/dev-manual-common-tasks.xml | 16081 |
1 files changed, 0 insertions, 16081 deletions
diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml deleted file mode 100644 index d29fe6262d..0000000000 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ /dev/null @@ -1,16081 +0,0 @@ -<!DOCTYPE chapter PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" -"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd" -[<!ENTITY % poky SYSTEM "../poky.ent"> %poky; ] > -<!--SPDX-License-Identifier: CC-BY-2.0-UK--> - -<chapter id='extendpoky'> - -<title>Common Tasks</title> - <para> - This chapter describes fundamental procedures such as creating layers, - adding new software packages, extending or customizing images, - porting work to new hardware (adding a new machine), and so forth. - You will find that the procedures documented here occur often in the - development cycle using the Yocto Project. - </para> - - <section id="understanding-and-creating-layers"> - <title>Understanding and Creating Layers</title> - - <para> - The OpenEmbedded build system supports organizing - <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink> into - multiple layers. - Layers allow you to isolate different types of customizations from - each other. - For introductory information on the Yocto Project Layer Model, - see the - "<ulink url='&YOCTO_DOCS_OM_URL;#the-yocto-project-layer-model'>The Yocto Project Layer Model</ulink>" - section in the Yocto Project Overview and Concepts Manual. - </para> - - <section id='creating-your-own-layer'> - <title>Creating Your Own Layer</title> - - <para> - It is very easy to create your own layers to use with the - OpenEmbedded build system. - The Yocto Project ships with tools that speed up creating - layers. - This section describes the steps you perform by hand to create - layers so that you can better understand them. - For information about the layer-creation tools, see the - "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script'>Creating a New BSP Layer Using the <filename>bitbake-layers</filename> Script</ulink>" - section in the Yocto Project Board Support Package (BSP) - Developer's Guide and the - "<link linkend='creating-a-general-layer-using-the-bitbake-layers-script'>Creating a General Layer Using the <filename>bitbake-layers</filename> Script</link>" - section further down in this manual. - </para> - - <para> - Follow these general steps to create your layer without using - tools: - <orderedlist> - <listitem><para> - <emphasis>Check Existing Layers:</emphasis> - Before creating a new layer, you should be sure someone - has not already created a layer containing the Metadata - you need. - You can see the - <ulink url='http://layers.openembedded.org/layerindex/layers/'>OpenEmbedded Metadata Index</ulink> - for a list of layers from the OpenEmbedded community - that can be used in the Yocto Project. - You could find a layer that is identical or close to - what you need. - </para></listitem> - <listitem><para> - <emphasis>Create a Directory:</emphasis> - Create the directory for your layer. - When you create the layer, be sure to create the - directory in an area not associated with the - Yocto Project - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - (e.g. the cloned <filename>poky</filename> repository). - </para> - - <para>While not strictly required, prepend the name of - the directory with the string "meta-". - For example: - <literallayout class='monospaced'> - meta-mylayer - meta-GUI_xyz - meta-mymachine - </literallayout> - With rare exceptions, a layer's name follows this - form: - <literallayout class='monospaced'> - meta-<replaceable>root_name</replaceable> - </literallayout> - Following this layer naming convention can - save you trouble later when tools, components, or - variables "assume" your layer name begins with "meta-". - A notable example is in configuration files as - shown in the following step where layer names without - the "meta-" string are appended - to several variables used in the configuration. - </para></listitem> - <listitem><para id='dev-layer-config-file-description'> - <emphasis>Create a Layer Configuration File:</emphasis> - Inside your new layer folder, you need to create a - <filename>conf/layer.conf</filename> file. - It is easiest to take an existing layer configuration - file and copy that to your layer's - <filename>conf</filename> directory and then modify the - file as needed.</para> - - <para>The - <filename>meta-yocto-bsp/conf/layer.conf</filename> file - in the Yocto Project - <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta-yocto-bsp/conf'>Source Repositories</ulink> - demonstrates the required syntax. - For your layer, you need to replace "yoctobsp" with - a unique identifier for your layer (e.g. "machinexyz" - for a layer named "meta-machinexyz"): - <literallayout class='monospaced'> - # We have a conf and classes directory, add to BBPATH - BBPATH .= ":${LAYERDIR}" - - # We have recipes-* directories, add to BBFILES - BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ - ${LAYERDIR}/recipes-*/*/*.bbappend" - - BBFILE_COLLECTIONS += "yoctobsp" - BBFILE_PATTERN_yoctobsp = "^${LAYERDIR}/" - BBFILE_PRIORITY_yoctobsp = "5" - LAYERVERSION_yoctobsp = "4" - LAYERSERIES_COMPAT_yoctobsp = "&DISTRO_NAME_NO_CAP;" - </literallayout> - Following is an explanation of the layer configuration - file: - <itemizedlist> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>: - Adds the layer's root directory to BitBake's - search path. - Through the use of the - <filename>BBPATH</filename> variable, BitBake - locates class files - (<filename>.bbclass</filename>), - configuration files, and files that are - included with <filename>include</filename> and - <filename>require</filename> statements. - For these cases, BitBake uses the first file - that matches the name found in - <filename>BBPATH</filename>. - This is similar to the way the - <filename>PATH</filename> variable is used for - binaries. - It is recommended, therefore, that you use - unique class and configuration filenames in - your custom layer. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'><filename>BBFILES</filename></ulink>: - Defines the location for all recipes in the - layer. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_COLLECTIONS'><filename>BBFILE_COLLECTIONS</filename></ulink>: - Establishes the current layer through a - unique identifier that is used throughout the - OpenEmbedded build system to refer to the layer. - In this example, the identifier "yoctobsp" is - the representation for the container layer - named "meta-yocto-bsp". - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PATTERN'><filename>BBFILE_PATTERN</filename></ulink>: - Expands immediately during parsing to - provide the directory of the layer. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink>: - Establishes a priority to use for - recipes in the layer when the OpenEmbedded build - finds recipes of the same name in different - layers. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERVERSION'><filename>LAYERVERSION</filename></ulink>: - Establishes a version number for the layer. - You can use this version number to specify this - exact version of the layer as a dependency when - using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDEPENDS'><filename>LAYERDEPENDS</filename></ulink> - variable. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERDEPENDS'><filename>LAYERDEPENDS</filename></ulink>: - Lists all layers on which this layer depends (if any). - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-LAYERSERIES_COMPAT'><filename>LAYERSERIES_COMPAT</filename></ulink>: - Lists the - <ulink url='&YOCTO_WIKI_URL;/wiki/Releases'>Yocto Project</ulink> - releases for which the current version is - compatible. - This variable is a good way to indicate if - your particular layer is current. - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <emphasis>Add Content:</emphasis> - Depending on the type of layer, add the content. - If the layer adds support for a machine, add the machine - configuration in a <filename>conf/machine/</filename> - file within the layer. - If the layer adds distro policy, add the distro - configuration in a <filename>conf/distro/</filename> - file within the layer. - If the layer introduces new recipes, put the recipes - you need in <filename>recipes-*</filename> - subdirectories within the layer. - <note> - For an explanation of layer hierarchy that - is compliant with the Yocto Project, see - the - "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-filelayout'>Example Filesystem Layout</ulink>" - section in the Yocto Project Board - Support Package (BSP) Developer's Guide. - </note> - </para></listitem> - <listitem><para> - <emphasis>Optionally Test for Compatibility:</emphasis> - If you want permission to use the Yocto Project - Compatibility logo with your layer or application that - uses your layer, perform the steps to apply for - compatibility. - See the - "<link linkend='making-sure-your-layer-is-compatible-with-yocto-project'>Making Sure Your Layer is Compatible With Yocto Project</link>" - section for more information. - </para></listitem> - </orderedlist> - </para> - </section> - - <section id='best-practices-to-follow-when-creating-layers'> - <title>Following Best Practices When Creating Layers</title> - - <para> - To create layers that are easier to maintain and that will - not impact builds for other machines, you should consider the - information in the following list: - <itemizedlist> - <listitem><para> - <emphasis>Avoid "Overlaying" Entire Recipes from Other Layers in Your Configuration:</emphasis> - In other words, do not copy an entire recipe into your - layer and then modify it. - Rather, use an append file - (<filename>.bbappend</filename>) to override only those - parts of the original recipe you need to modify. - </para></listitem> - <listitem><para> - <emphasis>Avoid Duplicating Include Files:</emphasis> - Use append files (<filename>.bbappend</filename>) - for each recipe that uses an include file. - Or, if you are introducing a new recipe that requires - the included file, use the path relative to the - original layer directory to refer to the file. - For example, use - <filename>require recipes-core/</filename><replaceable>package</replaceable><filename>/</filename><replaceable>file</replaceable><filename>.inc</filename> - instead of - <filename>require </filename><replaceable>file</replaceable><filename>.inc</filename>. - If you're finding you have to overlay the include file, - it could indicate a deficiency in the include file in - the layer to which it originally belongs. - If this is the case, you should try to address that - deficiency instead of overlaying the include file. - For example, you could address this by getting the - maintainer of the include file to add a variable or - variables to make it easy to override the parts needing - to be overridden. - </para></listitem> - <listitem><para> - <emphasis>Structure Your Layers:</emphasis> - Proper use of overrides within append files and - placement of machine-specific files within your layer - can ensure that a build is not using the wrong Metadata - and negatively impacting a build for a different - machine. - Following are some examples: - <itemizedlist> - <listitem><para> - <emphasis>Modify Variables to Support a - Different Machine:</emphasis> - Suppose you have a layer named - <filename>meta-one</filename> that adds support - for building machine "one". - To do so, you use an append file named - <filename>base-files.bbappend</filename> and - create a dependency on "foo" by altering the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> - variable: - <literallayout class='monospaced'> - DEPENDS = "foo" - </literallayout> - The dependency is created during any build that - includes the layer - <filename>meta-one</filename>. - However, you might not want this dependency - for all machines. - For example, suppose you are building for - machine "two" but your - <filename>bblayers.conf</filename> file has the - <filename>meta-one</filename> layer included. - During the build, the - <filename>base-files</filename> for machine - "two" will also have the dependency on - <filename>foo</filename>.</para> - <para>To make sure your changes apply only when - building machine "one", use a machine override - with the <filename>DEPENDS</filename> statement: - <literallayout class='monospaced'> - DEPENDS_one = "foo" - </literallayout> - You should follow the same strategy when using - <filename>_append</filename> and - <filename>_prepend</filename> operations: - <literallayout class='monospaced'> - DEPENDS_append_one = " foo" - DEPENDS_prepend_one = "foo " - </literallayout> - As an actual example, here's a snippet from the - generic kernel include file - <filename>linux-yocto.inc</filename>, - wherein the kernel compile and link options are - adjusted in the case of a subset of the supported - architectures: - <literallayout class='monospaced'> - DEPENDS_append_aarch64 = " libgcc" - KERNEL_CC_append_aarch64 = " ${TOOLCHAIN_OPTIONS}" - KERNEL_LD_append_aarch64 = " ${TOOLCHAIN_OPTIONS}" - - DEPENDS_append_nios2 = " libgcc" - KERNEL_CC_append_nios2 = " ${TOOLCHAIN_OPTIONS}" - KERNEL_LD_append_nios2 = " ${TOOLCHAIN_OPTIONS}" - - DEPENDS_append_arc = " libgcc" - KERNEL_CC_append_arc = " ${TOOLCHAIN_OPTIONS}" - KERNEL_LD_append_arc = " ${TOOLCHAIN_OPTIONS}" - - KERNEL_FEATURES_append_qemuall=" features/debug/printk.scc" - </literallayout> - <note> - Avoiding "+=" and "=+" and using - machine-specific - <filename>_append</filename> - and <filename>_prepend</filename> operations - is recommended as well. - </note> - </para></listitem> - <listitem><para> - <emphasis>Place Machine-Specific Files in - Machine-Specific Locations:</emphasis> - When you have a base recipe, such as - <filename>base-files.bb</filename>, that - contains a - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - statement to a file, you can use an append file - to cause the build to use your own version of - the file. - For example, an append file in your layer at - <filename>meta-one/recipes-core/base-files/base-files.bbappend</filename> - could extend - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink> - using - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink> - as follows: - <literallayout class='monospaced'> - FILESEXTRAPATHS_prepend := "${THISDIR}/${BPN}:" - </literallayout> - The build for machine "one" will pick up your - machine-specific file as long as you have the - file in - <filename>meta-one/recipes-core/base-files/base-files/</filename>. - However, if you are building for a different - machine and the - <filename>bblayers.conf</filename> file includes - the <filename>meta-one</filename> layer and - the location of your machine-specific file is - the first location where that file is found - according to <filename>FILESPATH</filename>, - builds for all machines will also use that - machine-specific file.</para> - <para>You can make sure that a machine-specific - file is used for a particular machine by putting - the file in a subdirectory specific to the - machine. - For example, rather than placing the file in - <filename>meta-one/recipes-core/base-files/base-files/</filename> - as shown above, put it in - <filename>meta-one/recipes-core/base-files/base-files/one/</filename>. - Not only does this make sure the file is used - only when building for machine "one", but the - build process locates the file more quickly.</para> - <para>In summary, you need to place all files - referenced from <filename>SRC_URI</filename> - in a machine-specific subdirectory within the - layer in order to restrict those files to - machine-specific builds. - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <emphasis>Perform Steps to Apply for Yocto Project Compatibility:</emphasis> - If you want permission to use the - Yocto Project Compatibility logo with your layer - or application that uses your layer, perform the - steps to apply for compatibility. - See the - "<link linkend='making-sure-your-layer-is-compatible-with-yocto-project'>Making Sure Your Layer is Compatible With Yocto Project</link>" - section for more information. - </para></listitem> - <listitem><para> - <emphasis>Follow the Layer Naming Convention:</emphasis> - Store custom layers in a Git repository that use the - <filename>meta-<replaceable>layer_name</replaceable></filename> - format. - </para></listitem> - <listitem><para> - <emphasis>Group Your Layers Locally:</emphasis> - Clone your repository alongside other cloned - <filename>meta</filename> directories from the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='making-sure-your-layer-is-compatible-with-yocto-project'> - <title>Making Sure Your Layer is Compatible With Yocto Project</title> - - <para> - When you create a layer used with the Yocto Project, it is - advantageous to make sure that the layer interacts well with - existing Yocto Project layers (i.e. the layer is compatible - with the Yocto Project). - Ensuring compatibility makes the layer easy to be consumed - by others in the Yocto Project community and could allow you - permission to use the Yocto Project Compatible Logo. - <note> - Only Yocto Project member organizations are permitted to - use the Yocto Project Compatible Logo. - The logo is not available for general use. - For information on how to become a Yocto Project member - organization, see the - <ulink url='&YOCTO_HOME_URL;'>Yocto Project Website</ulink>. - </note> - </para> - - <para> - The Yocto Project Compatibility Program consists of a layer - application process that requests permission to use the Yocto - Project Compatibility Logo for your layer and application. - The process consists of two parts: - <orderedlist> - <listitem><para> - Successfully passing a script - (<filename>yocto-check-layer</filename>) that - when run against your layer, tests it against - constraints based on experiences of how layers have - worked in the real world and where pitfalls have been - found. - Getting a "PASS" result from the script is required for - successful compatibility registration. - </para></listitem> - <listitem><para> - Completion of an application acceptance form, which - you can find at - <ulink url='https://www.yoctoproject.org/webform/yocto-project-compatible-registration'></ulink>. - </para></listitem> - </orderedlist> - </para> - - <para> - To be granted permission to use the logo, you need to satisfy - the following: - <itemizedlist> - <listitem><para> - Be able to check the box indicating that you - got a "PASS" when running the script against your - layer. - </para></listitem> - <listitem><para> - Answer "Yes" to the questions on the form or have an - acceptable explanation for any questions answered "No". - </para></listitem> - <listitem><para> - Be a Yocto Project Member Organization. - </para></listitem> - </itemizedlist> - </para> - - <para> - The remainder of this section presents information on the - registration form and on the - <filename>yocto-check-layer</filename> script. - </para> - - <section id='yocto-project-compatible-program-application'> - <title>Yocto Project Compatible Program Application</title> - - <para> - Use the form to apply for your layer's approval. - Upon successful application, you can use the Yocto - Project Compatibility Logo with your layer and the - application that uses your layer. - </para> - - <para> - To access the form, use this link: - <ulink url='https://www.yoctoproject.org/webform/yocto-project-compatible-registration'></ulink>. - Follow the instructions on the form to complete your - application. - </para> - - <para> - The application consists of the following sections: - <itemizedlist> - <listitem><para> - <emphasis>Contact Information:</emphasis> - Provide your contact information as the fields - require. - Along with your information, provide the - released versions of the Yocto Project for which - your layer is compatible. - </para></listitem> - <listitem><para> - <emphasis>Acceptance Criteria:</emphasis> - Provide "Yes" or "No" answers for each of the - items in the checklist. - Space exists at the bottom of the form for any - explanations for items for which you answered "No". - </para></listitem> - <listitem><para> - <emphasis>Recommendations:</emphasis> - Provide answers for the questions regarding Linux - kernel use and build success. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='yocto-check-layer-script'> - <title><filename>yocto-check-layer</filename> Script</title> - - <para> - The <filename>yocto-check-layer</filename> script - provides you a way to assess how compatible your layer is - with the Yocto Project. - You should run this script prior to using the form to - apply for compatibility as described in the previous - section. - You need to achieve a "PASS" result in order to have - your application form successfully processed. - </para> - - <para> - The script divides tests into three areas: COMMON, BSP, - and DISTRO. - For example, given a distribution layer (DISTRO), the - layer must pass both the COMMON and DISTRO related tests. - Furthermore, if your layer is a BSP layer, the layer must - pass the COMMON and BSP set of tests. - </para> - - <para> - To execute the script, enter the following commands from - your build directory: - <literallayout class='monospaced'> - $ source oe-init-build-env - $ yocto-check-layer <replaceable>your_layer_directory</replaceable> - </literallayout> - Be sure to provide the actual directory for your layer - as part of the command. - </para> - - <para> - Entering the command causes the script to determine the - type of layer and then to execute a set of specific - tests against the layer. - The following list overviews the test: - <itemizedlist> - <listitem><para> - <filename>common.test_readme</filename>: - Tests if a <filename>README</filename> file - exists in the layer and the file is not empty. - </para></listitem> - <listitem><para> - <filename>common.test_parse</filename>: - Tests to make sure that BitBake can parse the - files without error (i.e. - <filename>bitbake -p</filename>). - </para></listitem> - <listitem><para> - <filename>common.test_show_environment</filename>: - Tests that the global or per-recipe environment - is in order without errors (i.e. - <filename>bitbake -e</filename>). - </para></listitem> - <listitem><para> - <filename>common.test_world</filename>: - Verifies that <filename>bitbake world</filename> works. - </para></listitem> - <listitem><para> - <filename>common.test_signatures</filename>: - Tests to be sure that BSP and DISTRO layers do not - come with recipes that change signatures. - </para></listitem> - <listitem><para> - <filename>common.test_layerseries_compat</filename>: - Verifies layer compatibility is set properly. - </para></listitem> - <listitem><para> - <filename>bsp.test_bsp_defines_machines</filename>: - Tests if a BSP layer has machine configurations. - </para></listitem> - <listitem><para> - <filename>bsp.test_bsp_no_set_machine</filename>: - Tests to ensure a BSP layer does not set the - machine when the layer is added. - </para></listitem> - <listitem><para> - <filename>bsp.test_machine_world</filename>: - Verifies that <filename>bitbake world</filename> - works regardless of which machine is selected. - </para></listitem> - <listitem><para> - <filename>bsp.test_machine_signatures</filename>: - Verifies that building for a particular machine - affects only the signature of tasks specific to that - machine. - </para></listitem> - <listitem><para> - <filename>distro.test_distro_defines_distros</filename>: - Tests if a DISTRO layer has distro configurations. - </para></listitem> - <listitem><para> - <filename>distro.test_distro_no_set_distros</filename>: - Tests to ensure a DISTRO layer does not set the - distribution when the layer is added. - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id='enabling-your-layer'> - <title>Enabling Your Layer</title> - - <para> - Before the OpenEmbedded build system can use your new layer, - you need to enable it. - To enable your layer, simply add your layer's path to the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'>BBLAYERS</ulink></filename> - variable in your <filename>conf/bblayers.conf</filename> file, - which is found in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - The following example shows how to enable a layer named - <filename>meta-mylayer</filename>: - <literallayout class='monospaced'> - # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf - # changes incompatibly - POKY_BBLAYERS_CONF_VERSION = "2" - - BBPATH = "${TOPDIR}" - BBFILES ?= "" - - BBLAYERS ?= " \ - /home/<replaceable>user</replaceable>/poky/meta \ - /home/<replaceable>user</replaceable>/poky/meta-poky \ - /home/<replaceable>user</replaceable>/poky/meta-yocto-bsp \ - /home/<replaceable>user</replaceable>/poky/meta-mylayer \ - " - </literallayout> - </para> - - <para> - BitBake parses each <filename>conf/layer.conf</filename> file - from the top down as specified in the - <filename>BBLAYERS</filename> variable - within the <filename>conf/bblayers.conf</filename> file. - During the processing of each - <filename>conf/layer.conf</filename> file, BitBake adds the - recipes, classes and configurations contained within the - particular layer to the source directory. - </para> - </section> - - <section id='using-bbappend-files'> - <title>Using .bbappend Files in Your Layer</title> - - <para> - A recipe that appends Metadata to another recipe is called a - BitBake append file. - A BitBake append file uses the <filename>.bbappend</filename> - file type suffix, while the corresponding recipe to which - Metadata is being appended uses the <filename>.bb</filename> - file type suffix. - </para> - - <para> - You can use a <filename>.bbappend</filename> file in your - layer to make additions or changes to the content of another - layer's recipe without having to copy the other layer's - recipe into your layer. - Your <filename>.bbappend</filename> file resides in your layer, - while the main <filename>.bb</filename> recipe file to - which you are appending Metadata resides in a different layer. - </para> - - <para> - Being able to append information to an existing recipe not only - avoids duplication, but also automatically applies recipe - changes from a different layer into your layer. - If you were copying recipes, you would have to manually merge - changes as they occur. - </para> - - <para> - When you create an append file, you must use the same root - name as the corresponding recipe file. - For example, the append file - <filename>someapp_&DISTRO;.bbappend</filename> must apply to - <filename>someapp_&DISTRO;.bb</filename>. - This means the original recipe and append file names are - version number-specific. - If the corresponding recipe is renamed to update to a newer - version, you must also rename and possibly update - the corresponding <filename>.bbappend</filename> as well. - During the build process, BitBake displays an error on starting - if it detects a <filename>.bbappend</filename> file that does - not have a corresponding recipe with a matching name. - See the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_DANGLINGAPPENDS_WARNONLY'><filename>BB_DANGLINGAPPENDS_WARNONLY</filename></ulink> - variable for information on how to handle this error. - </para> - - <para> - As an example, consider the main formfactor recipe and a - corresponding formfactor append file both from the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. - Here is the main formfactor recipe, which is named - <filename>formfactor_0.0.bb</filename> and located in the - "meta" layer at - <filename>meta/recipes-bsp/formfactor</filename>: - <literallayout class='monospaced'> - SUMMARY = "Device formfactor information" - SECTION = "base" - LICENSE = "MIT" - LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" - PR = "r45" - - SRC_URI = "file://config file://machconfig" - S = "${WORKDIR}" - - PACKAGE_ARCH = "${MACHINE_ARCH}" - INHIBIT_DEFAULT_DEPS = "1" - - do_install() { - # Install file only if it has contents - install -d ${D}${sysconfdir}/formfactor/ - install -m 0644 ${S}/config ${D}${sysconfdir}/formfactor/ - if [ -s "${S}/machconfig" ]; then - install -m 0644 ${S}/machconfig ${D}${sysconfdir}/formfactor/ - fi - } </literallayout> - In the main recipe, note the - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - variable, which tells the OpenEmbedded build system where to - find files during the build. - </para> - - <para> - Following is the append file, which is named - <filename>formfactor_0.0.bbappend</filename> and is from the - Raspberry Pi BSP Layer named - <filename>meta-raspberrypi</filename>. - The file is in the layer at - <filename>recipes-bsp/formfactor</filename>: - <literallayout class='monospaced'> - FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:" - </literallayout> - </para> - - <para> - By default, the build system uses the - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink> - variable to locate files. - This append file extends the locations by setting the - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESEXTRAPATHS'><filename>FILESEXTRAPATHS</filename></ulink> - variable. - Setting this variable in the <filename>.bbappend</filename> - file is the most reliable and recommended method for adding - directories to the search path used by the build system - to find files. - </para> - - <para> - The statement in this example extends the directories to - include - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-THISDIR'><filename>THISDIR</filename></ulink><filename>}/${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>, - which resolves to a directory named - <filename>formfactor</filename> in the same directory - in which the append file resides (i.e. - <filename>meta-raspberrypi/recipes-bsp/formfactor</filename>. - This implies that you must have the supporting directory - structure set up that will contain any files or patches you - will be including from the layer. - </para> - - <para> - Using the immediate expansion assignment operator - <filename>:=</filename> is important because of the reference - to <filename>THISDIR</filename>. - The trailing colon character is important as it ensures that - items in the list remain colon-separated. - <note> - <para> - BitBake automatically defines the - <filename>THISDIR</filename> variable. - You should never set this variable yourself. - Using "_prepend" as part of the - <filename>FILESEXTRAPATHS</filename> ensures your path - will be searched prior to other paths in the final - list. - </para> - - <para> - Also, not all append files add extra files. - Many append files simply exist to add build options - (e.g. <filename>systemd</filename>). - For these cases, your append file would not even - use the <filename>FILESEXTRAPATHS</filename> statement. - </para> - </note> - </para> - </section> - - <section id='prioritizing-your-layer'> - <title>Prioritizing Your Layer</title> - - <para> - Each layer is assigned a priority value. - Priority values control which layer takes precedence if there - are recipe files with the same name in multiple layers. - For these cases, the recipe file from the layer with a higher - priority number takes precedence. - Priority values also affect the order in which multiple - <filename>.bbappend</filename> files for the same recipe are - applied. - You can either specify the priority manually, or allow the - build system to calculate it based on the layer's dependencies. - </para> - - <para> - To specify the layer's priority manually, use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink> - variable and append the layer's root name: - <literallayout class='monospaced'> - BBFILE_PRIORITY_mylayer = "1" - </literallayout> - </para> - - <note> - <para>It is possible for a recipe with a lower version number - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> - in a layer that has a higher priority to take precedence.</para> - <para>Also, the layer priority does not currently affect the - precedence order of <filename>.conf</filename> - or <filename>.bbclass</filename> files. - Future versions of BitBake might address this.</para> - </note> - </section> - - <section id='managing-layers'> - <title>Managing Layers</title> - - <para> - You can use the BitBake layer management tool - <filename>bitbake-layers</filename> to provide a view - into the structure of recipes across a multi-layer project. - Being able to generate output that reports on configured layers - with their paths and priorities and on - <filename>.bbappend</filename> files and their applicable - recipes can help to reveal potential problems. - </para> - - <para> - For help on the BitBake layer management tool, use the - following command: - <literallayout class='monospaced'> - $ bitbake-layers --help - NOTE: Starting bitbake server... - usage: bitbake-layers [-d] [-q] [-F] [--color COLOR] [-h] <subcommand> ... - - BitBake layers utility - - optional arguments: - -d, --debug Enable debug output - -q, --quiet Print only errors - -F, --force Force add without recipe parse verification - --color COLOR Colorize output (where COLOR is auto, always, never) - -h, --help show this help message and exit - - subcommands: - <subcommand> - show-layers show current configured layers. - show-overlayed list overlayed recipes (where the same recipe exists - in another layer) - show-recipes list available recipes, showing the layer they are - provided by - show-appends list bbappend files and recipe files they apply to - show-cross-depends Show dependencies between recipes that cross layer - boundaries. - add-layer Add one or more layers to bblayers.conf. - remove-layer Remove one or more layers from bblayers.conf. - flatten flatten layer configuration into a separate output - directory. - layerindex-fetch Fetches a layer from a layer index along with its - dependent layers, and adds them to conf/bblayers.conf. - layerindex-show-depends - Find layer dependencies from layer index. - create-layer Create a basic layer - - Use bitbake-layers <subcommand> --help to get help on a specific command - </literallayout> - </para> - - <para> - The following list describes the available commands: - <itemizedlist> - <listitem><para> - <emphasis><filename>help:</filename></emphasis> - Displays general help or help on a specified command. - </para></listitem> - <listitem><para> - <emphasis><filename>show-layers:</filename></emphasis> - Shows the current configured layers. - </para></listitem> - <listitem><para> - <emphasis><filename>show-overlayed:</filename></emphasis> - Lists overlayed recipes. - A recipe is overlayed when a recipe with the same name - exists in another layer that has a higher layer - priority. - </para></listitem> - <listitem><para> - <emphasis><filename>show-recipes:</filename></emphasis> - Lists available recipes and the layers that provide them. - </para></listitem> - <listitem><para> - <emphasis><filename>show-appends:</filename></emphasis> - Lists <filename>.bbappend</filename> files and the - recipe files to which they apply. - </para></listitem> - <listitem><para> - <emphasis><filename>show-cross-depends:</filename></emphasis> - Lists dependency relationships between recipes that - cross layer boundaries. - </para></listitem> - <listitem><para> - <emphasis><filename>add-layer:</filename></emphasis> - Adds a layer to <filename>bblayers.conf</filename>. - </para></listitem> - <listitem><para> - <emphasis><filename>remove-layer:</filename></emphasis> - Removes a layer from <filename>bblayers.conf</filename> - </para></listitem> - <listitem><para> - <emphasis><filename>flatten:</filename></emphasis> - Flattens the layer configuration into a separate output - directory. - Flattening your layer configuration builds a "flattened" - directory that contains the contents of all layers, - with any overlayed recipes removed and any - <filename>.bbappend</filename> files appended to the - corresponding recipes. - You might have to perform some manual cleanup of the - flattened layer as follows: - <itemizedlist> - <listitem><para> - Non-recipe files (such as patches) - are overwritten. - The flatten command shows a warning for these - files. - </para></listitem> - <listitem><para> - Anything beyond the normal layer - setup has been added to the - <filename>layer.conf</filename> file. - Only the lowest priority layer's - <filename>layer.conf</filename> is used. - </para></listitem> - <listitem><para> - Overridden and appended items from - <filename>.bbappend</filename> files need to be - cleaned up. - The contents of each - <filename>.bbappend</filename> end up in the - flattened recipe. - However, if there are appended or changed - variable values, you need to tidy these up - yourself. - Consider the following example. - Here, the <filename>bitbake-layers</filename> - command adds the line - <filename>#### bbappended ...</filename> so that - you know where the following lines originate: - <literallayout class='monospaced'> - ... - DESCRIPTION = "A useful utility" - ... - EXTRA_OECONF = "--enable-something" - ... - - #### bbappended from meta-anotherlayer #### - - DESCRIPTION = "Customized utility" - EXTRA_OECONF += "--enable-somethingelse" - </literallayout> - Ideally, you would tidy up these utilities as - follows: - <literallayout class='monospaced'> - ... - DESCRIPTION = "Customized utility" - ... - EXTRA_OECONF = "--enable-something --enable-somethingelse" - ... - </literallayout> - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <emphasis><filename>layerindex-fetch</filename>:</emphasis> - Fetches a layer from a layer index, along with its - dependent layers, and adds the layers to the - <filename>conf/bblayers.conf</filename> file. - </para></listitem> - <listitem><para> - <emphasis><filename>layerindex-show-depends</filename>:</emphasis> - Finds layer dependencies from the layer index. - </para></listitem> - <listitem><para> - <emphasis><filename>create-layer</filename>:</emphasis> - Creates a basic layer. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='creating-a-general-layer-using-the-bitbake-layers-script'> - <title>Creating a General Layer Using the <filename>bitbake-layers</filename> Script</title> - - <para> - The <filename>bitbake-layers</filename> script with the - <filename>create-layer</filename> subcommand simplifies - creating a new general layer. - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - For information on BSP layers, see the - "<ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP Layers</ulink>" - section in the Yocto Project Board Specific (BSP) - Developer's Guide. - </para></listitem> - <listitem><para> - In order to use a layer with the OpenEmbedded - build system, you need to add the layer to your - <filename>bblayers.conf</filename> configuration - file. - See the - "<link linkend='adding-a-layer-using-the-bitbake-layers-script'>Adding a Layer Using the <filename>bitbake-layers</filename> Script</link>" - section for more information. - </para></listitem> - </itemizedlist> - </note> - The default mode of the script's operation with this - subcommand is to create a layer with the following: - <itemizedlist> - <listitem><para>A layer priority of 6. - </para></listitem> - <listitem><para>A <filename>conf</filename> - subdirectory that contains a - <filename>layer.conf</filename> file. - </para></listitem> - <listitem><para> - A <filename>recipes-example</filename> subdirectory - that contains a further subdirectory named - <filename>example</filename>, which contains - an <filename>example.bb</filename> recipe file. - </para></listitem> - <listitem><para>A <filename >COPYING.MIT</filename>, - which is the license statement for the layer. - The script assumes you want to use the MIT license, - which is typical for most layers, for the contents of - the layer itself. - </para></listitem> - <listitem><para> - A <filename>README</filename> file, which is a file - describing the contents of your new layer. - </para></listitem> - </itemizedlist> - </para> - - <para> - In its simplest form, you can use the following command form - to create a layer. - The command creates a layer whose name corresponds to - <replaceable>your_layer_name</replaceable> in the current - directory: - <literallayout class='monospaced'> - $ bitbake-layers create-layer <replaceable>your_layer_name</replaceable> - </literallayout> - As an example, the following command creates a layer named - <filename>meta-scottrif</filename> in your home directory: - <literallayout class='monospaced'> - $ cd /usr/home - $ bitbake-layers create-layer meta-scottrif - NOTE: Starting bitbake server... - Add your new layer with 'bitbake-layers add-layer meta-scottrif' - </literallayout> - </para> - - <para> - If you want to set the priority of the layer to other than the - default value of "6", you can either use the - <filename>‐‐priority</filename> option or you can - edit the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILE_PRIORITY'><filename>BBFILE_PRIORITY</filename></ulink> - value in the <filename>conf/layer.conf</filename> after the - script creates it. - Furthermore, if you want to give the example recipe file - some name other than the default, you can - use the - <filename>‐‐example-recipe-name</filename> option. - </para> - - <para> - The easiest way to see how the - <filename>bitbake-layers create-layer</filename> command - works is to experiment with the script. - You can also read the usage information by entering the - following: - <literallayout class='monospaced'> - $ bitbake-layers create-layer --help - NOTE: Starting bitbake server... - usage: bitbake-layers create-layer [-h] [--priority PRIORITY] - [--example-recipe-name EXAMPLERECIPE] - layerdir - - Create a basic layer - - positional arguments: - layerdir Layer directory to create - - optional arguments: - -h, --help show this help message and exit - --priority PRIORITY, -p PRIORITY - Layer directory to create - --example-recipe-name EXAMPLERECIPE, -e EXAMPLERECIPE - Filename of the example recipe - </literallayout> - </para> - </section> - - <section id='adding-a-layer-using-the-bitbake-layers-script'> - <title>Adding a Layer Using the <filename>bitbake-layers</filename> Script</title> - - <para> - Once you create your general layer, you must add it to your - <filename>bblayers.conf</filename> file. - Adding the layer to this configuration file makes the - OpenEmbedded build system aware of your layer so that it can - search it for metadata. - </para> - - <para> - Add your layer by using the - <filename>bitbake-layers add-layer</filename> command: - <literallayout class='monospaced'> - $ bitbake-layers add-layer <replaceable>your_layer_name</replaceable> - </literallayout> - Here is an example that adds a layer named - <filename>meta-scottrif</filename> to the configuration file. - Following the command that adds the layer is another - <filename>bitbake-layers</filename> command that shows the - layers that are in your <filename>bblayers.conf</filename> - file: - <literallayout class='monospaced'> - $ bitbake-layers add-layer meta-scottrif - NOTE: Starting bitbake server... - Parsing recipes: 100% |##########################################################| Time: 0:00:49 - Parsing of 1441 .bb files complete (0 cached, 1441 parsed). 2055 targets, 56 skipped, 0 masked, 0 errors. - $ bitbake-layers show-layers - NOTE: Starting bitbake server... - layer path priority - ========================================================================== - meta /home/scottrif/poky/meta 5 - meta-poky /home/scottrif/poky/meta-poky 5 - meta-yocto-bsp /home/scottrif/poky/meta-yocto-bsp 5 - workspace /home/scottrif/poky/build/workspace 99 - meta-scottrif /home/scottrif/poky/build/meta-scottrif 6 - </literallayout> - Adding the layer to this file enables the build system to - locate the layer during the build. - <note> - During a build, the OpenEmbedded build system looks in - the layers from the top of the list down to the bottom - in that order. - </note> - </para> - </section> - </section> - - <section id='usingpoky-extend-customimage'> - <title>Customizing Images</title> - - <para> - You can customize images to satisfy particular requirements. - This section describes several methods and provides guidelines for each. - </para> - - <section id='usingpoky-extend-customimage-localconf'> - <title>Customizing Images Using <filename>local.conf</filename></title> - - <para> - Probably the easiest way to customize an image is to add a - package by way of the <filename>local.conf</filename> - configuration file. - Because it is limited to local use, this method generally only - allows you to add packages and is not as flexible as creating - your own customized image. - When you add packages using local variables this way, you need - to realize that these variable changes are in effect for every - build and consequently affect all images, which might not - be what you require. - </para> - - <para> - To add a package to your image using the local configuration - file, use the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename> - variable with the <filename>_append</filename> operator: - <literallayout class='monospaced'> - IMAGE_INSTALL_append = " strace" - </literallayout> - Use of the syntax is important - specifically, the space between - the quote and the package name, which is - <filename>strace</filename> in this example. - This space is required since the <filename>_append</filename> - operator does not add the space. - </para> - - <para> - Furthermore, you must use <filename>_append</filename> instead - of the <filename>+=</filename> operator if you want to avoid - ordering issues. - The reason for this is because doing so unconditionally appends - to the variable and avoids ordering problems due to the - variable being set in image recipes and - <filename>.bbclass</filename> files with operators like - <filename>?=</filename>. - Using <filename>_append</filename> ensures the operation takes - affect. - </para> - - <para> - As shown in its simplest use, - <filename>IMAGE_INSTALL_append</filename> affects all images. - It is possible to extend the syntax so that the variable - applies to a specific image only. - Here is an example: - <literallayout class='monospaced'> - IMAGE_INSTALL_append_pn-core-image-minimal = " strace" - </literallayout> - This example adds <filename>strace</filename> to the - <filename>core-image-minimal</filename> image only. - </para> - - <para> - You can add packages using a similar approach through the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL'>CORE_IMAGE_EXTRA_INSTALL</ulink></filename> - variable. - If you use this variable, only - <filename>core-image-*</filename> images are affected. - </para> - </section> - - <section id='usingpoky-extend-customimage-imagefeatures'> - <title>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and - <filename>EXTRA_IMAGE_FEATURES</filename></title> - - <para> - Another method for customizing your image is to enable or - disable high-level image features by using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> - and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink> - variables. - Although the functions for both variables are nearly equivalent, - best practices dictate using <filename>IMAGE_FEATURES</filename> - from within a recipe and using - <filename>EXTRA_IMAGE_FEATURES</filename> from within - your <filename>local.conf</filename> file, which is found in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - </para> - - <para> - To understand how these features work, the best reference is - <filename>meta/classes/core-image.bbclass</filename>. - This class lists out the available - <filename>IMAGE_FEATURES</filename> of which most map to - package groups while some, such as - <filename>debug-tweaks</filename> and - <filename>read-only-rootfs</filename>, resolve as general - configuration settings. - </para> - - <para> - In summary, the file looks at the contents of the - <filename>IMAGE_FEATURES</filename> variable and then maps - or configures the feature accordingly. - Based on this information, the build system automatically - adds the appropriate packages or configurations to the - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink> - variable. - Effectively, you are enabling extra features by extending the - class or creating a custom class for use with specialized image - <filename>.bb</filename> files. - </para> - - <para> - Use the <filename>EXTRA_IMAGE_FEATURES</filename> variable - from within your local configuration file. - Using a separate area from which to enable features with - this variable helps you avoid overwriting the features in the - image recipe that are enabled with - <filename>IMAGE_FEATURES</filename>. - The value of <filename>EXTRA_IMAGE_FEATURES</filename> is added - to <filename>IMAGE_FEATURES</filename> within - <filename>meta/conf/bitbake.conf</filename>. - </para> - - <para> - To illustrate how you can use these variables to modify your - image, consider an example that selects the SSH server. - The Yocto Project ships with two SSH servers you can use - with your images: Dropbear and OpenSSH. - Dropbear is a minimal SSH server appropriate for - resource-constrained environments, while OpenSSH is a - well-known standard SSH server implementation. - By default, the <filename>core-image-sato</filename> image - is configured to use Dropbear. - The <filename>core-image-full-cmdline</filename> and - <filename>core-image-lsb</filename> images both - include OpenSSH. - The <filename>core-image-minimal</filename> image does not - contain an SSH server. - </para> - - <para> - You can customize your image and change these defaults. - Edit the <filename>IMAGE_FEATURES</filename> variable - in your recipe or use the - <filename>EXTRA_IMAGE_FEATURES</filename> in your - <filename>local.conf</filename> file so that it configures the - image you are working with to include - <filename>ssh-server-dropbear</filename> or - <filename>ssh-server-openssh</filename>. - </para> - - <note> - See the - "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" - section in the Yocto Project Reference Manual for a complete - list of image features that ship with the Yocto Project. - </note> - </section> - - <section id='usingpoky-extend-customimage-custombb'> - <title>Customizing Images Using Custom .bb Files</title> - - <para> - You can also customize an image by creating a custom recipe - that defines additional software as part of the image. - The following example shows the form for the two lines you need: - <literallayout class='monospaced'> - IMAGE_INSTALL = "packagegroup-core-x11-base package1 package2" - - inherit core-image - </literallayout> - </para> - - <para> - Defining the software using a custom recipe gives you total - control over the contents of the image. - It is important to use the correct names of packages in the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename> - variable. - You must use the OpenEmbedded notation and not the Debian notation for the names - (e.g. <filename>glibc-dev</filename> instead of <filename>libc6-dev</filename>). - </para> - - <para> - The other method for creating a custom image is to base it on an existing image. - For example, if you want to create an image based on <filename>core-image-sato</filename> - but add the additional package <filename>strace</filename> to the image, - copy the <filename>meta/recipes-sato/images/core-image-sato.bb</filename> to a - new <filename>.bb</filename> and add the following line to the end of the copy: - <literallayout class='monospaced'> - IMAGE_INSTALL += "strace" - </literallayout> - </para> - </section> - - <section id='usingpoky-extend-customimage-customtasks'> - <title>Customizing Images Using Custom Package Groups</title> - - <para> - For complex custom images, the best approach for customizing - an image is to create a custom package group recipe that is - used to build the image or images. - A good example of a package group recipe is - <filename>meta/recipes-core/packagegroups/packagegroup-base.bb</filename>. - </para> - - <para> - If you examine that recipe, you see that the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename> - variable lists the package group packages to produce. - The <filename>inherit packagegroup</filename> statement - sets appropriate default values and automatically adds - <filename>-dev</filename>, <filename>-dbg</filename>, and - <filename>-ptest</filename> complementary packages for each - package specified in the <filename>PACKAGES</filename> - statement. - <note> - The <filename>inherit packagegroup</filename> line should be - located near the top of the recipe, certainly before - the <filename>PACKAGES</filename> statement. - </note> - </para> - - <para> - For each package you specify in <filename>PACKAGES</filename>, - you can use - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'>RDEPENDS</ulink></filename> - and - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'>RRECOMMENDS</ulink></filename> - entries to provide a list of packages the parent task package - should contain. - You can see examples of these further down in the - <filename>packagegroup-base.bb</filename> recipe. - </para> - - <para> - Here is a short, fabricated example showing the same basic - pieces for a hypothetical packagegroup defined in - <filename>packagegroup-custom.bb</filename>, where the - variable <filename>PN</filename> is the standard way to - abbreviate the reference to the full packagegroup name - <filename>packagegroup-custom</filename>: - <literallayout class='monospaced'> - DESCRIPTION = "My Custom Package Groups" - - inherit packagegroup - - PACKAGES = "\ - ${PN}-apps \ - ${PN}-tools \ - " - - RDEPENDS_${PN}-apps = "\ - dropbear \ - portmap \ - psplash" - - RDEPENDS_${PN}-tools = "\ - oprofile \ - oprofileui-server \ - lttng-tools" - - RRECOMMENDS_${PN}-tools = "\ - kernel-module-oprofile" - </literallayout> - </para> - - <para> - In the previous example, two package group packages are created with their dependencies and their - recommended package dependencies listed: <filename>packagegroup-custom-apps</filename>, and - <filename>packagegroup-custom-tools</filename>. - To build an image using these package group packages, you need to add - <filename>packagegroup-custom-apps</filename> and/or - <filename>packagegroup-custom-tools</filename> to - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'>IMAGE_INSTALL</ulink></filename>. - For other forms of image dependencies see the other areas of this section. - </para> - </section> - - <section id='usingpoky-extend-customimage-image-name'> - <title>Customizing an Image Hostname</title> - - <para> - By default, the configured hostname (i.e. - <filename>/etc/hostname</filename>) in an image is the - same as the machine name. - For example, if - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> - equals "qemux86", the configured hostname written to - <filename>/etc/hostname</filename> is "qemux86". - </para> - - <para> - You can customize this name by altering the value of the - "hostname" variable in the - <filename>base-files</filename> recipe using either - an append file or a configuration file. - Use the following in an append file: - <literallayout class='monospaced'> - hostname="myhostname" - </literallayout> - Use the following in a configuration file: - <literallayout class='monospaced'> - hostname_pn-base-files = "myhostname" - </literallayout> - </para> - - <para> - Changing the default value of the variable "hostname" can be - useful in certain situations. - For example, suppose you need to do extensive testing on an - image and you would like to easily identify the image - under test from existing images with typical default - hostnames. - In this situation, you could change the default hostname to - "testme", which results in all the images using the name - "testme". - Once testing is complete and you do not need to rebuild the - image for test any longer, you can easily reset the default - hostname. - </para> - - <para> - Another point of interest is that if you unset the variable, - the image will have no default hostname in the filesystem. - Here is an example that unsets the variable in a - configuration file: - <literallayout class='monospaced'> - hostname_pn-base-files = "" - </literallayout> - Having no default hostname in the filesystem is suitable for - environments that use dynamic hostnames such as virtual - machines. - </para> - </section> - </section> - - <section id='new-recipe-writing-a-new-recipe'> - <title>Writing a New Recipe</title> - - <para> - Recipes (<filename>.bb</filename> files) are fundamental components - in the Yocto Project environment. - Each software component built by the OpenEmbedded build system - requires a recipe to define the component. - This section describes how to create, write, and test a new - recipe. - <note> - For information on variables that are useful for recipes and - for information about recipe naming issues, see the - "<ulink url='&YOCTO_DOCS_REF_URL;#ref-varlocality-recipe-required'>Required</ulink>" - section of the Yocto Project Reference Manual. - </note> - </para> - - <section id='new-recipe-overview'> - <title>Overview</title> - - <para> - The following figure shows the basic process for creating a - new recipe. - The remainder of the section provides details for the steps. - <imagedata fileref="figures/recipe-workflow.png" width="6in" depth="7in" align="center" scalefit="1" /> - </para> - </section> - - <section id='new-recipe-locate-or-automatically-create-a-base-recipe'> - <title>Locate or Automatically Create a Base Recipe</title> - - <para> - You can always write a recipe from scratch. - However, three choices exist that can help you quickly get a - start on a new recipe: - <itemizedlist> - <listitem><para> - <emphasis><filename>devtool add</filename>:</emphasis> - A command that assists in creating a recipe and - an environment conducive to development. - </para></listitem> - <listitem><para> - <emphasis><filename>recipetool create</filename>:</emphasis> - A command provided by the Yocto Project that automates - creation of a base recipe based on the source - files. - </para></listitem> - <listitem><para> - <emphasis>Existing Recipes:</emphasis> - Location and modification of an existing recipe that is - similar in function to the recipe you need. - </para></listitem> - </itemizedlist> - <note> - For information on recipe syntax, see the - "<link linkend='recipe-syntax'>Recipe Syntax</link>" - section. - </note> - </para> - - <section id='new-recipe-creating-the-base-recipe-using-devtool'> - <title>Creating the Base Recipe Using <filename>devtool add</filename></title> - - <para> - The <filename>devtool add</filename> command uses the same - logic for auto-creating the recipe as - <filename>recipetool create</filename>, which is listed - below. - Additionally, however, <filename>devtool add</filename> - sets up an environment that makes it easy for you to - patch the source and to make changes to the recipe as - is often necessary when adding a recipe to build a new - piece of software to be included in a build. - </para> - - <para> - You can find a complete description of the - <filename>devtool add</filename> command in the - "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-a-closer-look-at-devtool-add'>A Closer Look at <filename>devtool</filename> add</ulink>" - section in the Yocto Project Application Development - and the Extensible Software Development Kit (eSDK) manual. - </para> - </section> - - <section id='new-recipe-creating-the-base-recipe-using-recipetool'> - <title>Creating the Base Recipe Using <filename>recipetool create</filename></title> - - <para> - <filename>recipetool create</filename> automates creation - of a base recipe given a set of source code files. - As long as you can extract or point to the source files, - the tool will construct a recipe and automatically - configure all pre-build information into the recipe. - For example, suppose you have an application that builds - using Autotools. - Creating the base recipe using - <filename>recipetool</filename> results in a recipe - that has the pre-build dependencies, license requirements, - and checksums configured. - </para> - - <para> - To run the tool, you just need to be in your - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - and have sourced the build environment setup script - (i.e. - <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>oe-init-build-env</filename></ulink>). - To get help on the tool, use the following command: - <literallayout class='monospaced'> - $ recipetool -h - NOTE: Starting bitbake server... - usage: recipetool [-d] [-q] [--color COLOR] [-h] <subcommand> ... - - OpenEmbedded recipe tool - - options: - -d, --debug Enable debug output - -q, --quiet Print only errors - --color COLOR Colorize output (where COLOR is auto, always, never) - -h, --help show this help message and exit - - subcommands: - create Create a new recipe - newappend Create a bbappend for the specified target in the specified - layer - setvar Set a variable within a recipe - appendfile Create/update a bbappend to replace a target file - appendsrcfiles Create/update a bbappend to add or replace source files - appendsrcfile Create/update a bbappend to add or replace a source file - Use recipetool <subcommand> --help to get help on a specific command - </literallayout> - </para> - - <para> - Running - <filename>recipetool create -o</filename> <replaceable>OUTFILE</replaceable> - creates the base recipe and locates it properly in the - layer that contains your source files. - Following are some syntax examples: - </para> - - <para> - Use this syntax to generate a recipe based on - <replaceable>source</replaceable>. - Once generated, the recipe resides in the existing source - code layer: - <literallayout class='monospaced'> - recipetool create -o <replaceable>OUTFILE</replaceable> <replaceable>source</replaceable> - </literallayout> - Use this syntax to generate a recipe using code that you - extract from <replaceable>source</replaceable>. - The extracted code is placed in its own layer defined - by <replaceable>EXTERNALSRC</replaceable>. - <literallayout class='monospaced'> - recipetool create -o <replaceable>OUTFILE</replaceable> -x <replaceable>EXTERNALSRC</replaceable> <replaceable>source</replaceable> - </literallayout> - Use this syntax to generate a recipe based on - <replaceable>source</replaceable>. - The options direct <filename>recipetool</filename> to - generate debugging information. - Once generated, the recipe resides in the existing source - code layer: - <literallayout class='monospaced'> - recipetool create -d -o <replaceable>OUTFILE</replaceable> <replaceable>source</replaceable> - </literallayout> - </para> - </section> - - <section id='new-recipe-locating-and-using-a-similar-recipe'> - <title>Locating and Using a Similar Recipe</title> - - <para> - Before writing a recipe from scratch, it is often useful to - discover whether someone else has already written one that - meets (or comes close to meeting) your needs. - The Yocto Project and OpenEmbedded communities maintain many - recipes that might be candidates for what you are doing. - You can find a good central index of these recipes in the - <ulink url='http://layers.openembedded.org'>OpenEmbedded Layer Index</ulink>. - </para> - - <para> - Working from an existing recipe or a skeleton recipe is the - best way to get started. - Here are some points on both methods: - <itemizedlist> - <listitem><para><emphasis>Locate and modify a recipe that - is close to what you want to do:</emphasis> - This method works when you are familiar with the - current recipe space. - The method does not work so well for those new to - the Yocto Project or writing recipes.</para> - <para>Some risks associated with this method are - using a recipe that has areas totally unrelated to - what you are trying to accomplish with your recipe, - not recognizing areas of the recipe that you might - have to add from scratch, and so forth. - All these risks stem from unfamiliarity with the - existing recipe space.</para></listitem> - <listitem><para><emphasis>Use and modify the following - skeleton recipe:</emphasis> - If for some reason you do not want to use - <filename>recipetool</filename> and you cannot - find an existing recipe that is close to meeting - your needs, you can use the following structure to - provide the fundamental areas of a new recipe. - <literallayout class='monospaced'> - DESCRIPTION = "" - HOMEPAGE = "" - LICENSE = "" - SECTION = "" - DEPENDS = "" - LIC_FILES_CHKSUM = "" - - SRC_URI = "" - </literallayout> - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id='new-recipe-storing-and-naming-the-recipe'> - <title>Storing and Naming the Recipe</title> - - <para> - Once you have your base recipe, you should put it in your - own layer and name it appropriately. - Locating it correctly ensures that the OpenEmbedded build - system can find it when you use BitBake to process the - recipe. - </para> - - <itemizedlist> - <listitem><para><emphasis>Storing Your Recipe:</emphasis> - The OpenEmbedded build system locates your recipe - through the layer's <filename>conf/layer.conf</filename> - file and the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBFILES'><filename>BBFILES</filename></ulink> - variable. - This variable sets up a path from which the build system can - locate recipes. - Here is the typical use: - <literallayout class='monospaced'> - BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \ - ${LAYERDIR}/recipes-*/*/*.bbappend" - </literallayout> - Consequently, you need to be sure you locate your new recipe - inside your layer such that it can be found.</para> - <para>You can find more information on how layers are - structured in the - "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" - section.</para></listitem> - <listitem><para><emphasis>Naming Your Recipe:</emphasis> - When you name your recipe, you need to follow this naming - convention: - <literallayout class='monospaced'> - <replaceable>basename</replaceable>_<replaceable>version</replaceable>.bb - </literallayout> - Use lower-cased characters and do not include the reserved - suffixes <filename>-native</filename>, - <filename>-cross</filename>, <filename>-initial</filename>, - or <filename>-dev</filename> casually (i.e. do not use them - as part of your recipe name unless the string applies). - Here are some examples: - <literallayout class='monospaced'> - cups_1.7.0.bb - gawk_4.0.2.bb - irssi_0.8.16-rc1.bb - </literallayout></para></listitem> - </itemizedlist> - </section> - - <section id='new-recipe-running-a-build-on-the-recipe'> - <title>Running a Build on the Recipe</title> - - <para> - Creating a new recipe is usually an iterative process that - requires using BitBake to process the recipe multiple times in - order to progressively discover and add information to the - recipe file. - </para> - - <para> - Assuming you have sourced the build environment setup script (i.e. - <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>) - and you are in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>, - use BitBake to process your recipe. - All you need to provide is the - <filename><replaceable>basename</replaceable></filename> of the recipe as described - in the previous section: - <literallayout class='monospaced'> - $ bitbake <replaceable>basename</replaceable> - </literallayout> - - </para> - - <para> - During the build, the OpenEmbedded build system creates a - temporary work directory for each recipe - (<filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename>) - where it keeps extracted source files, log files, intermediate - compilation and packaging files, and so forth. - </para> - - <para> - The path to the per-recipe temporary work directory depends - on the context in which it is being built. - The quickest way to find this path is to have BitBake return it - by running the following: - <literallayout class='monospaced'> - $ bitbake -e <replaceable>basename</replaceable> | grep ^WORKDIR= - </literallayout> - As an example, assume a Source Directory top-level folder named - <filename>poky</filename>, a default Build Directory at - <filename>poky/build</filename>, and a - <filename>qemux86-poky-linux</filename> machine target system. - Furthermore, suppose your recipe is named - <filename>foo_1.3.0.bb</filename>. - In this case, the work directory the build system uses to - build the package would be as follows: - <literallayout class='monospaced'> - poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 - </literallayout> - Inside this directory you can find sub-directories such as - <filename>image</filename>, <filename>packages-split</filename>, - and <filename>temp</filename>. - After the build, you can examine these to determine how well - the build went. - <note> - You can find log files for each task in the recipe's - <filename>temp</filename> directory (e.g. - <filename>poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0/temp</filename>). - Log files are named <filename>log.<replaceable>taskname</replaceable></filename> - (e.g. <filename>log.do_configure</filename>, - <filename>log.do_fetch</filename>, and - <filename>log.do_compile</filename>). - </note> - </para> - - <para> - You can find more information about the build process in - "<ulink url='&YOCTO_DOCS_OM_URL;#overview-development-environment'>The Yocto Project Development Environment</ulink>" - chapter of the Yocto Project Overview and Concepts Manual. - </para> - </section> - - <section id='new-recipe-fetching-code'> - <title>Fetching Code</title> - - <para> - The first thing your recipe must do is specify how to fetch - the source files. - Fetching is controlled mainly through the - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - variable. - Your recipe must have a <filename>SRC_URI</filename> variable - that points to where the source is located. - For a graphical representation of source locations, see the - "<ulink url='&YOCTO_DOCS_OM_URL;#sources-dev-environment'>Sources</ulink>" - section in the Yocto Project Overview and Concepts Manual. - </para> - - <para> - The - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink> - task uses the prefix of each entry in the - <filename>SRC_URI</filename> variable value to determine which - <ulink url='&YOCTO_DOCS_BB_URL;#bb-fetchers'>fetcher</ulink> - to use to get your source files. - It is the <filename>SRC_URI</filename> variable that triggers - the fetcher. - The - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> - task uses the variable after source is fetched to apply - patches. - The OpenEmbedded build system uses - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESOVERRIDES'><filename>FILESOVERRIDES</filename></ulink> - for scanning directory locations for local files in - <filename>SRC_URI</filename>. - </para> - - <para> - The <filename>SRC_URI</filename> variable in your recipe must - define each unique location for your source files. - It is good practice to not hard-code version numbers in a URL used - in <filename>SRC_URI</filename>. - Rather than hard-code these values, use - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink><filename>}</filename>, - which causes the fetch process to use the version specified in - the recipe filename. - Specifying the version in this manner means that upgrading the - recipe to a future version is as simple as renaming the recipe - to match the new version. - </para> - - <para> - Here is a simple example from the - <filename>meta/recipes-devtools/strace/strace_5.5.bb</filename> - recipe where the source comes from a single tarball. - Notice the use of the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> - variable: - <literallayout class='monospaced'> - SRC_URI = "https://strace.io/files/${PV}/strace-${PV}.tar.xz \ - </literallayout> - </para> - - <para> - Files mentioned in <filename>SRC_URI</filename> whose names end - in a typical archive extension (e.g. <filename>.tar</filename>, - <filename>.tar.gz</filename>, <filename>.tar.bz2</filename>, - <filename>.zip</filename>, and so forth), are automatically - extracted during the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink> - task. - For another example that specifies these types of files, see - the - "<link linkend='new-recipe-autotooled-package'>Autotooled Package</link>" - section. - </para> - - <para> - Another way of specifying source is from an SCM. - For Git repositories, you must specify - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink> - and you should specify - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> - to include the revision with - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>. - Here is an example from the recipe - <filename>meta/recipes-kernel/blktrace/blktrace_git.bb</filename>: - <literallayout class='monospaced'> - SRCREV = "d6918c8832793b4205ed3bfede78c2f915c23385" - - PR = "r6" - PV = "1.0.5+git${SRCPV}" - - SRC_URI = "git://git.kernel.dk/blktrace.git \ - file://ldflags.patch" - </literallayout> - </para> - - <para> - If your <filename>SRC_URI</filename> statement includes - URLs pointing to individual files fetched from a remote server - other than a version control system, BitBake attempts to - verify the files against checksums defined in your recipe to - ensure they have not been tampered with or otherwise modified - since the recipe was written. - Two checksums are used: - <filename>SRC_URI[md5sum]</filename> and - <filename>SRC_URI[sha256sum]</filename>. - </para> - - <para> - If your <filename>SRC_URI</filename> variable points to - more than a single URL (excluding SCM URLs), you need to - provide the <filename>md5</filename> and - <filename>sha256</filename> checksums for each URL. - For these cases, you provide a name for each URL as part of - the <filename>SRC_URI</filename> and then reference that name - in the subsequent checksum statements. - Here is an example combining lines from the files - <filename>git.inc</filename> and - <filename>git_2.24.1.bb</filename>: - <literallayout class='monospaced'> - SRC_URI = "${KERNELORG_MIRROR}/software/scm/git/git-${PV}.tar.gz;name=tarball \ - ${KERNELORG_MIRROR}/software/scm/git/git-manpages-${PV}.tar.gz;name=manpages" - - SRC_URI[tarball.md5sum] = "166bde96adbbc11c8843d4f8f4f9811b" - SRC_URI[tarball.sha256sum] = "ad5334956301c86841eb1e5b1bb20884a6bad89a10a6762c958220c7cf64da02" - SRC_URI[manpages.md5sum] = "31c2272a8979022497ba3d4202df145d" - SRC_URI[manpages.sha256sum] = "9a7ae3a093bea39770eb96ca3e5b40bff7af0b9f6123f089d7821d0e5b8e1230" - </literallayout> - </para> - - <para> - Proper values for <filename>md5</filename> and - <filename>sha256</filename> checksums might be available - with other signatures on the download page for the upstream - source (e.g. <filename>md5</filename>, - <filename>sha1</filename>, <filename>sha256</filename>, - <filename>GPG</filename>, and so forth). - Because the OpenEmbedded build system only deals with - <filename>sha256sum</filename> and <filename>md5sum</filename>, - you should verify all the signatures you find by hand. - </para> - - <para> - If no <filename>SRC_URI</filename> checksums are specified - when you attempt to build the recipe, or you provide an - incorrect checksum, the build will produce an error for each - missing or incorrect checksum. - As part of the error message, the build system provides - the checksum string corresponding to the fetched file. - Once you have the correct checksums, you can copy and paste - them into your recipe and then run the build again to continue. - <note> - As mentioned, if the upstream source provides signatures - for verifying the downloaded source code, you should - verify those manually before setting the checksum values - in the recipe and continuing with the build. - </note> - </para> - - <para> - This final example is a bit more complicated and is from the - <filename>meta/recipes-sato/rxvt-unicode/rxvt-unicode_9.20.bb</filename> - recipe. - The example's <filename>SRC_URI</filename> statement identifies - multiple files as the source files for the recipe: a tarball, a - patch file, a desktop file, and an icon. - <literallayout class='monospaced'> - SRC_URI = "http://dist.schmorp.de/rxvt-unicode/Attic/rxvt-unicode-${PV}.tar.bz2 \ - file://xwc.patch \ - file://rxvt.desktop \ - file://rxvt.png" - </literallayout> - </para> - - <para> - When you specify local files using the - <filename>file://</filename> URI protocol, the build system - fetches files from the local machine. - The path is relative to the - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILESPATH'><filename>FILESPATH</filename></ulink> - variable and searches specific directories in a certain order: - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink><filename>}</filename>, - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink><filename>}</filename>, - and <filename>files</filename>. - The directories are assumed to be subdirectories of the - directory in which the recipe or append file resides. - For another example that specifies these types of files, see the - "<link linkend='new-recipe-single-c-file-package-hello-world'>Single .c File Package (Hello World!)</link>" - section. - </para> - - <para> - The previous example also specifies a patch file. - Patch files are files whose names usually end in - <filename>.patch</filename> or <filename>.diff</filename> but - can end with compressed suffixes such as - <filename>diff.gz</filename> and - <filename>patch.bz2</filename>, for example. - The build system automatically applies patches as described - in the - "<link linkend='new-recipe-patching-code'>Patching Code</link>" section. - </para> - </section> - - <section id='new-recipe-unpacking-code'> - <title>Unpacking Code</title> - - <para> - During the build, the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink> - task unpacks the source with - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename> - pointing to where it is unpacked. - </para> - - <para> - If you are fetching your source files from an upstream source - archived tarball and the tarball's internal structure matches - the common convention of a top-level subdirectory named - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink><filename>}-${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink><filename>}</filename>, - then you do not need to set <filename>S</filename>. - However, if <filename>SRC_URI</filename> specifies to fetch - source from an archive that does not use this convention, - or from an SCM like Git or Subversion, your recipe needs to - define <filename>S</filename>. - </para> - - <para> - If processing your recipe using BitBake successfully unpacks - the source files, you need to be sure that the directory - pointed to by <filename>${S}</filename> matches the structure - of the source. - </para> - </section> - - <section id='new-recipe-patching-code'> - <title>Patching Code</title> - - <para> - Sometimes it is necessary to patch code after it has been - fetched. - Any files mentioned in <filename>SRC_URI</filename> whose - names end in <filename>.patch</filename> or - <filename>.diff</filename> or compressed versions of these - suffixes (e.g. <filename>diff.gz</filename> are treated as - patches. - The - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> - task automatically applies these patches. - </para> - - <para> - The build system should be able to apply patches with the "-p1" - option (i.e. one directory level in the path will be stripped - off). - If your patch needs to have more directory levels stripped off, - specify the number of levels using the "striplevel" option in - the <filename>SRC_URI</filename> entry for the patch. - Alternatively, if your patch needs to be applied in a specific - subdirectory that is not specified in the patch file, use the - "patchdir" option in the entry. - </para> - - <para> - As with all local files referenced in - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - using <filename>file://</filename>, you should place - patch files in a directory next to the recipe either - named the same as the base name of the recipe - (<ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-BPN'><filename>BPN</filename></ulink>) - or "files". - </para> - </section> - - <section id='new-recipe-licensing'> - <title>Licensing</title> - - <para> - Your recipe needs to have both the - <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink> - variables: - <itemizedlist> - <listitem><para><emphasis><filename>LICENSE</filename>:</emphasis> - This variable specifies the license for the software. - If you do not know the license under which the software - you are building is distributed, you should go to the - source code and look for that information. - Typical files containing this information include - <filename>COPYING</filename>, - <filename>LICENSE</filename>, and - <filename>README</filename> files. - You could also find the information near the top of - a source file. - For example, given a piece of software licensed under - the GNU General Public License version 2, you would - set <filename>LICENSE</filename> as follows: - <literallayout class='monospaced'> - LICENSE = "GPLv2" - </literallayout></para> - <para>The licenses you specify within - <filename>LICENSE</filename> can have any name as long - as you do not use spaces, since spaces are used as - separators between license names. - For standard licenses, use the names of the files in - <filename>meta/files/common-licenses/</filename> - or the <filename>SPDXLICENSEMAP</filename> flag names - defined in <filename>meta/conf/licenses.conf</filename>. - </para></listitem> - <listitem><para><emphasis><filename>LIC_FILES_CHKSUM</filename>:</emphasis> - The OpenEmbedded build system uses this variable to - make sure the license text has not changed. - If it has, the build produces an error and it affords - you the chance to figure it out and correct the problem. - </para> - <para>You need to specify all applicable licensing - files for the software. - At the end of the configuration step, the build process - will compare the checksums of the files to be sure - the text has not changed. - Any differences result in an error with the message - containing the current checksum. - For more explanation and examples of how to set the - <filename>LIC_FILES_CHKSUM</filename> variable, see the - "<link link='usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</link>" - section.</para> - - <para>To determine the correct checksum string, you - can list the appropriate files in the - <filename>LIC_FILES_CHKSUM</filename> variable with - incorrect md5 strings, attempt to build the software, - and then note the resulting error messages that will - report the correct md5 strings. - See the - "<link linkend='new-recipe-fetching-code'>Fetching Code</link>" - section for additional information. - </para> - - <para> - Here is an example that assumes the software has a - <filename>COPYING</filename> file: - <literallayout class='monospaced'> - LIC_FILES_CHKSUM = "file://COPYING;md5=xxx" - </literallayout> - When you try to build the software, the build system - will produce an error and give you the correct string - that you can substitute into the recipe file for a - subsequent build. - </para></listitem> - </itemizedlist> - </para> - -<!-- - - <para> - For trying this out I created a new recipe named - <filename>htop_1.0.2.bb</filename> and put it in - <filename>poky/meta/recipes-extended/htop</filename>. - There are two license type statements in my very simple - recipe: - <literallayout class='monospaced'> - LICENSE = "" - - LIC_FILES_CHKSUM = "" - - SRC_URI[md5sum] = "" - SRC_URI[sha256sum] = "" - </literallayout> - Evidently, you need to run a <filename>bitbake -c cleanall htop</filename>. - Next, you delete or comment out the two <filename>SRC_URI</filename> - lines at the end and then attempt to build the software with - <filename>bitbake htop</filename>. - Doing so causes BitBake to report some errors and and give - you the actual strings you need for the last two - <filename>SRC_URI</filename> lines. - Prior to this, you have to dig around in the home page of the - source for <filename>htop</filename> and determine that the - software is released under GPLv2. - You can provide that in the <filename>LICENSE</filename> - statement. - Now you edit your recipe to have those two strings for - the <filename>SRC_URI</filename> statements: - <literallayout class='monospaced'> - LICENSE = "GPLv2" - - LIC_FILES_CHKSUM = "" - - SRC_URI = "${SOURCEFORGE_MIRROR}/htop/htop-${PV}.tar.gz" - SRC_URI[md5sum] = "0d01cca8df3349c74569cefebbd9919e" - SRC_URI[sha256sum] = "ee60657b044ece0df096c053060df7abf3cce3a568ab34d260049e6a37ccd8a1" - </literallayout> - At this point, you can build the software again using the - <filename>bitbake htop</filename> command. - There is just a set of errors now associated with the - empty <filename>LIC_FILES_CHKSUM</filename> variable now. - </para> ---> - - </section> - - <section id='new-dependencies'> - <title>Dependencies</title> - - <para> - Most software packages have a short list of other packages - that they require, which are called dependencies. - These dependencies fall into two main categories: build-time - dependencies, which are required when the software is built; - and runtime dependencies, which are required to be installed - on the target in order for the software to run. - </para> - - <para> - Within a recipe, you specify build-time dependencies using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> - variable. - Although nuances exist, items specified in - <filename>DEPENDS</filename> should be names of other recipes. - It is important that you specify all build-time dependencies - explicitly. - If you do not, due to the parallel nature of BitBake's - execution, you can end up with a race condition where the - dependency is present for one task of a recipe (e.g. - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink>) - and then gone when the next task runs (e.g. - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink>). - </para> - - <para> - Another consideration is that configure scripts might - automatically check for optional dependencies and enable - corresponding functionality if those dependencies are found. - This behavior means that to ensure deterministic results and - thus avoid more race conditions, you need to either explicitly - specify these dependencies as well, or tell the configure - script explicitly to disable the functionality. - If you wish to make a recipe that is more generally useful - (e.g. publish the recipe in a layer for others to use), - instead of hard-disabling the functionality, you can use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG'><filename>PACKAGECONFIG</filename></ulink> - variable to allow functionality and the corresponding - dependencies to be enabled and disabled easily by other - users of the recipe. - </para> - - <para> - Similar to build-time dependencies, you specify runtime - dependencies through a variable - - <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>, - which is package-specific. - All variables that are package-specific need to have the name - of the package added to the end as an override. - Since the main package for a recipe has the same name as the - recipe, and the recipe's name can be found through the - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename> - variable, then you specify the dependencies for the main - package by setting <filename>RDEPENDS_${PN}</filename>. - If the package were named <filename>${PN}-tools</filename>, - then you would set <filename>RDEPENDS_${PN}-tools</filename>, - and so forth. - </para> - - <para> - Some runtime dependencies will be set automatically at - packaging time. - These dependencies include any shared library dependencies - (i.e. if a package "example" contains "libexample" and - another package "mypackage" contains a binary that links to - "libexample" then the OpenEmbedded build system will - automatically add a runtime dependency to "mypackage" on - "example"). - See the - "<ulink url='&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</ulink>" - section in the Yocto Project Overview and Concepts Manual for - further details. - </para> - </section> - - <section id='new-recipe-configuring-the-recipe'> - <title>Configuring the Recipe</title> - - <para> - Most software provides some means of setting build-time - configuration options before compilation. - Typically, setting these options is accomplished by running a - configure script with options, or by modifying a build - configuration file. - <note> - As of Yocto Project Release 1.7, some of the core recipes - that package binary configuration scripts now disable the - scripts due to the scripts previously requiring error-prone - path substitution. - The OpenEmbedded build system uses - <filename>pkg-config</filename> now, which is much more - robust. - You can find a list of the <filename>*-config</filename> - scripts that are disabled list in the - "<ulink url='&YOCTO_DOCS_REF_URL;#migration-1.7-binary-configuration-scripts-disabled'>Binary Configuration Scripts Disabled</ulink>" - section in the Yocto Project Reference Manual. - </note> - </para> - - <para> - A major part of build-time configuration is about checking for - build-time dependencies and possibly enabling optional - functionality as a result. - You need to specify any build-time dependencies for the - software you are building in your recipe's - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> - value, in terms of other recipes that satisfy those - dependencies. - You can often find build-time or runtime - dependencies described in the software's documentation. - </para> - - <para> - The following list provides configuration items of note based - on how your software is built: - <itemizedlist> - <listitem><para><emphasis>Autotools:</emphasis> - If your source files have a - <filename>configure.ac</filename> file, then your - software is built using Autotools. - If this is the case, you just need to worry about - modifying the configuration.</para> - - <para>When using Autotools, your recipe needs to inherit - the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink> - class and your recipe does not have to contain a - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> - task. - However, you might still want to make some adjustments. - For example, you can set - <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECONF'><filename>EXTRA_OECONF</filename></ulink> - or - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></ulink> - to pass any needed configure options that are specific - to the recipe. - </para></listitem> - <listitem><para><emphasis>CMake:</emphasis> - If your source files have a - <filename>CMakeLists.txt</filename> file, then your - software is built using CMake. - If this is the case, you just need to worry about - modifying the configuration.</para> - - <para>When you use CMake, your recipe needs to inherit - the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-cmake'><filename>cmake</filename></ulink> - class and your recipe does not have to contain a - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> - task. - You can make some adjustments by setting - <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OECMAKE'><filename>EXTRA_OECMAKE</filename></ulink> - to pass any needed configure options that are specific - to the recipe. - <note> - If you need to install one or more custom CMake - toolchain files that are supplied by the - application you are building, install the files to - <filename>${D}${datadir}/cmake/</filename> Modules - during - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>. - </note> - </para></listitem> - <listitem><para><emphasis>Other:</emphasis> - If your source files do not have a - <filename>configure.ac</filename> or - <filename>CMakeLists.txt</filename> file, then your - software is built using some method other than Autotools - or CMake. - If this is the case, you normally need to provide a - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> - task in your recipe - unless, of course, there is nothing to configure. - </para> - <para>Even if your software is not being built by - Autotools or CMake, you still might not need to deal - with any configuration issues. - You need to determine if configuration is even a required step. - You might need to modify a Makefile or some configuration file - used for the build to specify necessary build options. - Or, perhaps you might need to run a provided, custom - configure script with the appropriate options.</para> - <para>For the case involving a custom configure - script, you would run - <filename>./configure --help</filename> and look for - the options you need to set.</para></listitem> - </itemizedlist> - </para> - - <para> - Once configuration succeeds, it is always good practice to - look at the <filename>log.do_configure</filename> file to - ensure that the appropriate options have been enabled and no - additional build-time dependencies need to be added to - <filename>DEPENDS</filename>. - For example, if the configure script reports that it found - something not mentioned in <filename>DEPENDS</filename>, or - that it did not find something that it needed for some - desired optional functionality, then you would need to add - those to <filename>DEPENDS</filename>. - Looking at the log might also reveal items being checked for, - enabled, or both that you do not want, or items not being found - that are in <filename>DEPENDS</filename>, in which case - you would need to look at passing extra options to the - configure script as needed. - For reference information on configure options specific to the - software you are building, you can consult the output of the - <filename>./configure --help</filename> command within - <filename>${S}</filename> or consult the software's upstream - documentation. - </para> - </section> - - <section id='new-recipe-using-headers-to-interface-with-devices'> - <title>Using Headers to Interface with Devices</title> - - <para> - If your recipe builds an application that needs to - communicate with some device or needs an API into a custom - kernel, you will need to provide appropriate header files. - Under no circumstances should you ever modify the existing - <filename>meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc</filename> - file. - These headers are used to build <filename>libc</filename> and - must not be compromised with custom or machine-specific - header information. - If you customize <filename>libc</filename> through modified - headers all other applications that use - <filename>libc</filename> thus become affected. - <note><title>Warning</title> - Never copy and customize the <filename>libc</filename> - header file (i.e. - <filename>meta/recipes-kernel/linux-libc-headers/linux-libc-headers.inc</filename>). - </note> - The correct way to interface to a device or custom kernel is - to use a separate package that provides the additional headers - for the driver or other unique interfaces. - When doing so, your application also becomes responsible for - creating a dependency on that specific provider. - </para> - - <para> - Consider the following: - <itemizedlist> - <listitem><para> - Never modify - <filename>linux-libc-headers.inc</filename>. - Consider that file to be part of the - <filename>libc</filename> system, and not something - you use to access the kernel directly. - You should access <filename>libc</filename> through - specific <filename>libc</filename> calls. - </para></listitem> - <listitem><para> - Applications that must talk directly to devices - should either provide necessary headers themselves, - or establish a dependency on a special headers package - that is specific to that driver. - </para></listitem> - </itemizedlist> - </para> - - <para> - For example, suppose you want to modify an existing header - that adds I/O control or network support. - If the modifications are used by a small number programs, - providing a unique version of a header is easy and has little - impact. - When doing so, bear in mind the guidelines in the previous - list. - <note> - If for some reason your changes need to modify the behavior - of the <filename>libc</filename>, and subsequently all - other applications on the system, use a - <filename>.bbappend</filename> to modify the - <filename>linux-kernel-headers.inc</filename> file. - However, take care to not make the changes - machine specific. - </note> - </para> - - <para> - Consider a case where your kernel is older and you need - an older <filename>libc</filename> ABI. - The headers installed by your recipe should still be a - standard mainline kernel, not your own custom one. - </para> - - <para> - When you use custom kernel headers you need to get them from - <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_KERNEL_DIR'><filename>STAGING_KERNEL_DIR</filename></ulink>, - which is the directory with kernel headers that are - required to build out-of-tree modules. - Your recipe will also need the following: - <literallayout class='monospaced'> - do_configure[depends] += "virtual/kernel:do_shared_workdir" - </literallayout> - </para> - </section> - - <section id='new-recipe-compilation'> - <title>Compilation</title> - - <para> - During a build, the <filename>do_compile</filename> task - happens after source is fetched, unpacked, and configured. - If the recipe passes through <filename>do_compile</filename> - successfully, nothing needs to be done. - </para> - - <para> - However, if the compile step fails, you need to diagnose the - failure. - Here are some common issues that cause failures. - <note> - For cases where improper paths are detected for - configuration files or for when libraries/headers cannot - be found, be sure you are using the more robust - <filename>pkg-config</filename>. - See the note in section - "<link linkend='new-recipe-configuring-the-recipe'>Configuring the Recipe</link>" - for additional information. - </note> - <itemizedlist> - <listitem><para><emphasis>Parallel build failures:</emphasis> - These failures manifest themselves as intermittent - errors, or errors reporting that a file or directory - that should be created by some other part of the build - process could not be found. - This type of failure can occur even if, upon inspection, - the file or directory does exist after the build has - failed, because that part of the build process happened - in the wrong order.</para> - <para>To fix the problem, you need to either satisfy - the missing dependency in the Makefile or whatever - script produced the Makefile, or (as a workaround) - set - <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink> - to an empty string: - <literallayout class='monospaced'> - PARALLEL_MAKE = "" - </literallayout></para> - <para> - For information on parallel Makefile issues, see the - "<link linkend='debugging-parallel-make-races'>Debugging Parallel Make Races</link>" - section. - </para></listitem> - <listitem><para><emphasis>Improper host path usage:</emphasis> - This failure applies to recipes building for the target - or <filename>nativesdk</filename> only. - The failure occurs when the compilation process uses - improper headers, libraries, or other files from the - host system when cross-compiling for the target. - </para> - <para>To fix the problem, examine the - <filename>log.do_compile</filename> file to identify - the host paths being used (e.g. - <filename>/usr/include</filename>, - <filename>/usr/lib</filename>, and so forth) and then - either add configure options, apply a patch, or do both. - </para></listitem> - <listitem><para><emphasis>Failure to find required - libraries/headers:</emphasis> - If a build-time dependency is missing because it has - not been declared in - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>, - or because the dependency exists but the path used by - the build process to find the file is incorrect and the - configure step did not detect it, the compilation - process could fail. - For either of these failures, the compilation process - notes that files could not be found. - In these cases, you need to go back and add additional - options to the configure script as well as possibly - add additional build-time dependencies to - <filename>DEPENDS</filename>.</para> - <para>Occasionally, it is necessary to apply a patch - to the source to ensure the correct paths are used. - If you need to specify paths to find files staged - into the sysroot from other recipes, use the variables - that the OpenEmbedded build system provides - (e.g. - <filename>STAGING_BINDIR</filename>, - <filename>STAGING_INCDIR</filename>, - <filename>STAGING_DATADIR</filename>, and so forth). -<!-- - (e.g. - <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_BINDIR'><filename>STAGING_BINDIR</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_INCDIR'><filename>STAGING_INCDIR</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DATADIR'><filename>STAGING_DATADIR</filename></ulink>, - and so forth). ---> - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='new-recipe-installing'> - <title>Installing</title> - - <para> - During <filename>do_install</filename>, the task copies the - built files along with their hierarchy to locations that - would mirror their locations on the target device. - The installation process copies files from the - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>, - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-B'><filename>B</filename></ulink><filename>}</filename>, - and - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}</filename> - directories to the - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename> - directory to create the structure as it should appear on the - target system. - </para> - - <para> - How your software is built affects what you must do to be - sure your software is installed correctly. - The following list describes what you must do for installation - depending on the type of build system used by the software - being built: - <itemizedlist> - <listitem><para><emphasis>Autotools and CMake:</emphasis> - If the software your recipe is building uses Autotools - or CMake, the OpenEmbedded build - system understands how to install the software. - Consequently, you do not have to have a - <filename>do_install</filename> task as part of your - recipe. - You just need to make sure the install portion of the - build completes with no issues. - However, if you wish to install additional files not - already being installed by - <filename>make install</filename>, you should do this - using a <filename>do_install_append</filename> function - using the install command as described in - the "Manual" bulleted item later in this list. - </para></listitem> - <listitem><para><emphasis>Other (using - <filename>make install</filename>):</emphasis> - You need to define a - <filename>do_install</filename> function in your - recipe. - The function should call - <filename>oe_runmake install</filename> and will likely - need to pass in the destination directory as well. - How you pass that path is dependent on how the - <filename>Makefile</filename> being run is written - (e.g. <filename>DESTDIR=${D}</filename>, - <filename>PREFIX=${D}</filename>, - <filename>INSTALLROOT=${D}</filename>, and so forth). - </para> - <para>For an example recipe using - <filename>make install</filename>, see the - "<link linkend='new-recipe-makefile-based-package'>Makefile-Based Package</link>" - section.</para></listitem> - <listitem><para><emphasis>Manual:</emphasis> - You need to define a - <filename>do_install</filename> function in your - recipe. - The function must first use - <filename>install -d</filename> to create the - directories under - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename>. - Once the directories exist, your function can use - <filename>install</filename> to manually install the - built software into the directories.</para> - <para>You can find more information on - <filename>install</filename> at - <ulink url='http://www.gnu.org/software/coreutils/manual/html_node/install-invocation.html'></ulink>. - </para></listitem> - </itemizedlist> - </para> - - <para> - For the scenarios that do not use Autotools or - CMake, you need to track the installation - and diagnose and fix any issues until everything installs - correctly. - You need to look in the default location of - <filename>${D}</filename>, which is - <filename>${WORKDIR}/image</filename>, to be sure your - files have been installed correctly. - </para> - - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - During the installation process, you might need to - modify some of the installed files to suit the target - layout. - For example, you might need to replace hard-coded paths - in an initscript with values of variables provided by - the build system, such as replacing - <filename>/usr/bin/</filename> with - <filename>${bindir}</filename>. - If you do perform such modifications during - <filename>do_install</filename>, be sure to modify the - destination file after copying rather than before - copying. - Modifying after copying ensures that the build system - can re-execute <filename>do_install</filename> if - needed. - </para></listitem> - <listitem><para> - <filename>oe_runmake install</filename>, which can be - run directly or can be run indirectly by the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-cmake'><filename>cmake</filename></ulink> - classes, runs <filename>make install</filename> in - parallel. - Sometimes, a Makefile can have missing dependencies - between targets that can result in race conditions. - If you experience intermittent failures during - <filename>do_install</filename>, you might be able to - work around them by disabling parallel Makefile - installs by adding the following to the recipe: - <literallayout class='monospaced'> - PARALLEL_MAKEINST = "" - </literallayout> - See - <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename></ulink> - for additional information. - </para></listitem> - <listitem><para> - If you need to install one or more custom CMake - toolchain files that are supplied by the - application you are building, install the files to - <filename>${D}${datadir}/cmake/</filename> Modules - during - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink>. - </para></listitem> - </itemizedlist> - </note> - </section> - - <section id='new-recipe-enabling-system-services'> - <title>Enabling System Services</title> - - <para> - If you want to install a service, which is a process that - usually starts on boot and runs in the background, then - you must include some additional definitions in your recipe. - </para> - - <para> - If you are adding services and the service initialization - script or the service file itself is not installed, you must - provide for that installation in your recipe using a - <filename>do_install_append</filename> function. - If your recipe already has a <filename>do_install</filename> - function, update the function near its end rather than - adding an additional <filename>do_install_append</filename> - function. - </para> - - <para> - When you create the installation for your services, you need - to accomplish what is normally done by - <filename>make install</filename>. - In other words, make sure your installation arranges the output - similar to how it is arranged on the target system. - </para> - - <para> - The OpenEmbedded build system provides support for starting - services two different ways: - <itemizedlist> - <listitem><para><emphasis>SysVinit:</emphasis> - SysVinit is a system and service manager that - manages the init system used to control the very basic - functions of your system. - The init program is the first program - started by the Linux kernel when the system boots. - Init then controls the startup, running and shutdown - of all other programs.</para> - <para>To enable a service using SysVinit, your recipe - needs to inherit the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-update-rc.d'><filename>update-rc.d</filename></ulink> - class. - The class helps facilitate safely installing the - package on the target.</para> - <para>You will need to set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PACKAGES'><filename>INITSCRIPT_PACKAGES</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_NAME'><filename>INITSCRIPT_NAME</filename></ulink>, - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-INITSCRIPT_PARAMS'><filename>INITSCRIPT_PARAMS</filename></ulink> - variables within your recipe.</para></listitem> - <listitem><para><emphasis>systemd:</emphasis> - System Management Daemon (systemd) was designed to - replace SysVinit and to provide - enhanced management of services. - For more information on systemd, see the systemd - homepage at - <ulink url='http://freedesktop.org/wiki/Software/systemd/'></ulink>. - </para> - <para>To enable a service using systemd, your recipe - needs to inherit the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-systemd'><filename>systemd</filename></ulink> - class. - See the <filename>systemd.bbclass</filename> file - located in your - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. - section for more information. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='new-recipe-packaging'> - <title>Packaging</title> - - <para> - Successful packaging is a combination of automated processes - performed by the OpenEmbedded build system and some - specific steps you need to take. - The following list describes the process: - <itemizedlist> - <listitem><para><emphasis>Splitting Files</emphasis>: - The <filename>do_package</filename> task splits the - files produced by the recipe into logical components. - Even software that produces a single binary might - still have debug symbols, documentation, and other - logical components that should be split out. - The <filename>do_package</filename> task ensures - that files are split up and packaged correctly. - </para></listitem> - <listitem><para><emphasis>Running QA Checks</emphasis>: - The - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink> - class adds a step to - the package generation process so that output quality - assurance checks are generated by the OpenEmbedded - build system. - This step performs a range of checks to be sure the - build's output is free of common problems that show - up during runtime. - For information on these checks, see the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-insane'><filename>insane</filename></ulink> - class and the - "<ulink url='&YOCTO_DOCS_REF_URL;#ref-qa-checks'>QA Error and Warning Messages</ulink>" - chapter in the Yocto Project Reference Manual. - </para></listitem> - <listitem><para><emphasis>Hand-Checking Your Packages</emphasis>: - After you build your software, you need to be sure - your packages are correct. - Examine the - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/packages-split</filename> - directory and make sure files are where you expect - them to be. - If you discover problems, you can set - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>, - <filename>do_install(_append)</filename>, and so forth as - needed. - </para></listitem> - <listitem><para><emphasis>Splitting an Application into Multiple Packages</emphasis>: - If you need to split an application into several - packages, see the - "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application into Multiple Packages</link>" - section for an example. - </para></listitem> - <listitem><para><emphasis>Installing a Post-Installation Script</emphasis>: - For an example showing how to install a - post-installation script, see the - "<link linkend='new-recipe-post-installation-scripts'>Post-Installation Scripts</link>" - section. - </para></listitem> - <listitem><para><emphasis>Marking Package Architecture</emphasis>: - Depending on what your recipe is building and how it - is configured, it might be important to mark the - packages produced as being specific to a particular - machine, or to mark them as not being specific to - a particular machine or architecture at all.</para> - <para>By default, packages apply to any machine with the - same architecture as the target machine. - When a recipe produces packages that are - machine-specific (e.g. the - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> - value is passed into the configure script or a patch - is applied only for a particular machine), you should - mark them as such by adding the following to the - recipe: - <literallayout class='monospaced'> - PACKAGE_ARCH = "${MACHINE_ARCH}" - </literallayout></para> - <para>On the other hand, if the recipe produces packages - that do not contain anything specific to the target - machine or architecture at all (e.g. recipes - that simply package script files or configuration - files), you should use the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink> - class to do this for you by adding this to your - recipe: - <literallayout class='monospaced'> - inherit allarch - </literallayout> - Ensuring that the package architecture is correct is - not critical while you are doing the first few builds - of your recipe. - However, it is important in order - to ensure that your recipe rebuilds (or does not - rebuild) appropriately in response to changes in - configuration, and to ensure that you get the - appropriate packages installed on the target machine, - particularly if you run separate builds for more - than one target machine. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='new-sharing-files-between-recipes'> - <title>Sharing Files Between Recipes</title> - - <para> - Recipes often need to use files provided by other recipes on - the build host. - For example, an application linking to a common library needs - access to the library itself and its associated headers. - The way this access is accomplished is by populating a sysroot - with files. - Each recipe has two sysroots in its work directory, one for - target files - (<filename>recipe-sysroot</filename>) and one for files that - are native to the build host - (<filename>recipe-sysroot-native</filename>). - <note> - You could find the term "staging" used within the Yocto - project regarding files populating sysroots (e.g. the - <ulink url='&YOCTO_DOCS_REF_URL;#var-STAGING_DIR'><filename>STAGING_DIR</filename></ulink> - variable). - </note> - </para> - - <para> - Recipes should never populate the sysroot directly (i.e. write - files into sysroot). - Instead, files should be installed into standard locations - during the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink> - task within the - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink><filename>}</filename> - directory. - The reason for this limitation is that almost all files that - populate the sysroot are cataloged in manifests in order to - ensure the files can be removed later when a recipe is either - modified or removed. - Thus, the sysroot is able to remain free from stale files. - </para> - - <para> - A subset of the files installed by the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink> - task are used by the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink> - task as defined by the the - <ulink url='&YOCTO_DOCS_REF_URL;#var-SYSROOT_DIRS'><filename>SYSROOT_DIRS</filename></ulink> - variable to automatically populate the sysroot. - It is possible to modify the list of directories that populate - the sysroot. - The following example shows how you could add the - <filename>/opt</filename> directory to the list of - directories within a recipe: - <literallayout class='monospaced'> - SYSROOT_DIRS += "/opt" - </literallayout> - </para> - - <para> - For a more complete description of the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink> - task and its associated functions, see the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-staging'><filename>staging</filename></ulink> - class. - </para> - </section> - - <section id='metadata-virtual-providers'> - <title>Using Virtual Providers</title> - - <para> - Prior to a build, if you know that several different recipes - provide the same functionality, you can use a virtual provider - (i.e. <filename>virtual/*</filename>) as a placeholder for the - actual provider. - The actual provider is determined at build-time. - </para> - - <para> - A common scenario where a virtual provider is used would be - for the kernel recipe. - Suppose you have three kernel recipes whose - <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> - values map to <filename>kernel-big</filename>, - <filename>kernel-mid</filename>, and - <filename>kernel-small</filename>. - Furthermore, each of these recipes in some way uses a - <ulink url='&YOCTO_DOCS_REF_URL;#var-PROVIDES'><filename>PROVIDES</filename></ulink> - statement that essentially identifies itself as being able - to provide <filename>virtual/kernel</filename>. - Here is one way through the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-kernel'><filename>kernel</filename></ulink> - class: - <literallayout class='monospaced'> - PROVIDES += "${@ "virtual/kernel" if (d.getVar("KERNEL_PACKAGE_NAME") == "kernel") else "" }" - </literallayout> - Any recipe that inherits the <filename>kernel</filename> class - is going to utilize a <filename>PROVIDES</filename> statement - that identifies that recipe as being able to provide the - <filename>virtual/kernel</filename> item. - </para> - - <para> - Now comes the time to actually build an image and you need a - kernel recipe, but which one? - You can configure your build to call out the kernel recipe - you want by using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER'><filename>PREFERRED_PROVIDER</filename></ulink> - variable. - As an example, consider the - <ulink url='https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/conf/machine/include/x86-base.inc'><filename>x86-base.inc</filename></ulink> - include file, which is a machine - (i.e. <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>) - configuration file. - This include file is the reason all x86-based machines use the - <filename>linux-yocto</filename> kernel. - Here are the relevant lines from the include file: - <literallayout class='monospaced'> - PREFERRED_PROVIDER_virtual/kernel ??= "linux-yocto" - PREFERRED_VERSION_linux-yocto ??= "4.15%" - </literallayout> - </para> - - <para> - When you use a virtual provider, you do not have to - "hard code" a recipe name as a build dependency. - You can use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> - variable to state the build is dependent on - <filename>virtual/kernel</filename> for example: - <literallayout class='monospaced'> - DEPENDS = "virtual/kernel" - </literallayout> - During the build, the OpenEmbedded build system picks - the correct recipe needed for the - <filename>virtual/kernel</filename> dependency based on the - <filename>PREFERRED_PROVIDER</filename> variable. - If you want to use the small kernel mentioned at the beginning - of this section, configure your build as follows: - <literallayout class='monospaced'> - PREFERRED_PROVIDER_virtual/kernel ??= "kernel-small" - </literallayout> - <note> - Any recipe that - <ulink url='&YOCTO_DOCS_REF_URL;#var-PROVIDES'><filename>PROVIDES</filename></ulink> - a <filename>virtual/*</filename> item that is ultimately - not selected through - <filename>PREFERRED_PROVIDER</filename> does not get built. - Preventing these recipes from building is usually the - desired behavior since this mechanism's purpose is to - select between mutually exclusive alternative providers. - </note> - </para> - - <para> - The following lists specific examples of virtual providers: - <itemizedlist> - <listitem><para> - <filename>virtual/kernel</filename>: - Provides the name of the kernel recipe to use when - building a kernel image. - </para></listitem> - <listitem><para> - <filename>virtual/bootloader</filename>: - Provides the name of the bootloader to use when - building an image. - </para></listitem> - <listitem><para> - <filename>virtual/mesa</filename>: - Provides <filename>gbm.pc</filename>. - </para></listitem> - <listitem><para> - <filename>virtual/egl</filename>: - Provides <filename>egl.pc</filename> and possibly - <filename>wayland-egl.pc</filename>. - </para></listitem> - <listitem><para> - <filename>virtual/libgl</filename>: - Provides <filename>gl.pc</filename> (i.e. libGL). - </para></listitem> - <listitem><para> - <filename>virtual/libgles1</filename>: - Provides <filename>glesv1_cm.pc</filename> - (i.e. libGLESv1_CM). - </para></listitem> - <listitem><para> - <filename>virtual/libgles2</filename>: - Provides <filename>glesv2.pc</filename> - (i.e. libGLESv2). - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='properly-versioning-pre-release-recipes'> - <title>Properly Versioning Pre-Release Recipes</title> - - <para> - Sometimes the name of a recipe can lead to versioning - problems when the recipe is upgraded to a final release. - For example, consider the - <filename>irssi_0.8.16-rc1.bb</filename> recipe file in - the list of example recipes in the - "<link linkend='new-recipe-storing-and-naming-the-recipe'>Storing and Naming the Recipe</link>" - section. - This recipe is at a release candidate stage (i.e. - "rc1"). - When the recipe is released, the recipe filename becomes - <filename>irssi_0.8.16.bb</filename>. - The version change from <filename>0.8.16-rc1</filename> - to <filename>0.8.16</filename> is seen as a decrease by the - build system and package managers, so the resulting packages - will not correctly trigger an upgrade. - </para> - - <para> - In order to ensure the versions compare properly, the - recommended convention is to set - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> - within the recipe to - "<replaceable>previous_version</replaceable>+<replaceable>current_version</replaceable>". - You can use an additional variable so that you can use the - current version elsewhere. - Here is an example: - <literallayout class='monospaced'> - REALPV = "0.8.16-rc1" - PV = "0.8.15+${REALPV}" - </literallayout> - </para> - </section> - - <section id='new-recipe-post-installation-scripts'> - <title>Post-Installation Scripts</title> - - <para> - Post-installation scripts run immediately after installing - a package on the target or during image creation when a - package is included in an image. - To add a post-installation script to a package, add a - <filename>pkg_postinst_</filename><replaceable>PACKAGENAME</replaceable><filename>()</filename> function to - the recipe file (<filename>.bb</filename>) and replace - <replaceable>PACKAGENAME</replaceable> with the name of the package - you want to attach to the <filename>postinst</filename> - script. - To apply the post-installation script to the main package - for the recipe, which is usually what is required, specify - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename> - in place of <replaceable>PACKAGENAME</replaceable>. - </para> - - <para> - A post-installation function has the following structure: - <literallayout class='monospaced'> - pkg_postinst_<replaceable>PACKAGENAME</replaceable>() { - # Commands to carry out - } - </literallayout> - </para> - - <para> - The script defined in the post-installation function is - called when the root filesystem is created. - If the script succeeds, the package is marked as installed. - <note> - Any RPM post-installation script that runs on the target - should return a 0 exit code. - RPM does not allow non-zero exit codes for these scripts, - and the RPM package manager will cause the package to fail - installation on the target. - </note> - </para> - - <para> - Sometimes it is necessary for the execution of a - post-installation script to be delayed until the first boot. - For example, the script might need to be executed on the - device itself. - To delay script execution until boot time, you must explicitly - mark post installs to defer to the target. - You can use <filename>pkg_postinst_ontarget()</filename> or - call - <filename>postinst_intercept delay_to_first_boot</filename> - from <filename>pkg_postinst()</filename>. - Any failure of a <filename>pkg_postinst()</filename> script - (including exit 1) triggers an error during the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs'><filename>do_rootfs</filename></ulink> - task. - </para> - - <para> - If you have recipes that use - <filename>pkg_postinst</filename> function - and they require the use of non-standard native - tools that have dependencies during rootfs construction, you - need to use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_WRITE_DEPS'><filename>PACKAGE_WRITE_DEPS</filename></ulink> - variable in your recipe to list these tools. - If you do not use this variable, the tools might be missing and - execution of the post-installation script is deferred until - first boot. - Deferring the script to first boot is undesirable and for - read-only rootfs impossible. - </para> - - <note> - Equivalent support for pre-install, pre-uninstall, and - post-uninstall scripts exist by way of - <filename>pkg_preinst</filename>, - <filename>pkg_prerm</filename>, and - <filename>pkg_postrm</filename>, respectively. - These scrips work in exactly the same way as does - <filename>pkg_postinst</filename> with the exception - that they run at different times. - Also, because of when they run, they are not applicable to - being run at image creation time like - <filename>pkg_postinst</filename>. - </note> - </section> - - <section id='new-recipe-testing'> - <title>Testing</title> - - <para> - The final step for completing your recipe is to be sure that - the software you built runs correctly. - To accomplish runtime testing, add the build's output - packages to your image and test them on the target. - </para> - - <para> - For information on how to customize your image by adding - specific packages, see the - "<link linkend='usingpoky-extend-customimage'>Customizing Images</link>" - section. - </para> - </section> - - <section id='new-recipe-testing-examples'> - <title>Examples</title> - - <para> - To help summarize how to write a recipe, this section provides - some examples given various scenarios: - <itemizedlist> - <listitem><para>Recipes that use local files</para></listitem> - <listitem><para>Using an Autotooled package</para></listitem> - <listitem><para>Using a Makefile-based package</para></listitem> - <listitem><para>Splitting an application into multiple packages</para></listitem> - <listitem><para>Adding binaries to an image</para></listitem> - </itemizedlist> - </para> - - <section id='new-recipe-single-c-file-package-hello-world'> - <title>Single .c File Package (Hello World!)</title> - - <para> - Building an application from a single file that is stored - locally (e.g. under <filename>files</filename>) requires - a recipe that has the file listed in the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> - variable. - Additionally, you need to manually write the - <filename>do_compile</filename> and - <filename>do_install</filename> tasks. - The <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename> - variable defines the directory containing the source code, - which is set to - <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink> - in this case - the directory BitBake uses for the build. - <literallayout class='monospaced'> - SUMMARY = "Simple helloworld application" - SECTION = "examples" - LICENSE = "MIT" - LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302" - - SRC_URI = "file://helloworld.c" - - S = "${WORKDIR}" - - do_compile() { - ${CC} helloworld.c -o helloworld - } - - do_install() { - install -d ${D}${bindir} - install -m 0755 helloworld ${D}${bindir} - } - </literallayout> - </para> - - <para> - By default, the <filename>helloworld</filename>, - <filename>helloworld-dbg</filename>, and - <filename>helloworld-dev</filename> packages are built. - For information on how to customize the packaging process, - see the - "<link linkend='splitting-an-application-into-multiple-packages'>Splitting an Application into Multiple Packages</link>" - section. - </para> - </section> - - <section id='new-recipe-autotooled-package'> - <title>Autotooled Package</title> - <para> - Applications that use Autotools such as <filename>autoconf</filename> and - <filename>automake</filename> require a recipe that has a source archive listed in - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> and - also inherit the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-autotools'><filename>autotools</filename></ulink> - class, which contains the definitions of all the steps - needed to build an Autotool-based application. - The result of the build is automatically packaged. - And, if the application uses NLS for localization, packages with local information are - generated (one package per language). - Following is one example: (<filename>hello_2.3.bb</filename>) - <literallayout class='monospaced'> - SUMMARY = "GNU Helloworld application" - SECTION = "examples" - LICENSE = "GPLv2+" - LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe" - - SRC_URI = "${GNU_MIRROR}/hello/hello-${PV}.tar.gz" - - inherit autotools gettext - </literallayout> - </para> - - <para> - The variable - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'>LIC_FILES_CHKSUM</ulink></filename> - is used to track source license changes as described in the - "<link linkend='usingpoky-configuring-LIC_FILES_CHKSUM'>Tracking License Changes</link>" - section in the Yocto Project Overview and Concepts Manual. - You can quickly create Autotool-based recipes in a manner - similar to the previous example. - </para> - </section> - - <section id='new-recipe-makefile-based-package'> - <title>Makefile-Based Package</title> - - <para> - Applications that use GNU <filename>make</filename> also require a recipe that has - the source archive listed in - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>. - You do not need to add a <filename>do_compile</filename> step since by default BitBake - starts the <filename>make</filename> command to compile the application. - If you need additional <filename>make</filename> options, you should store them in the - <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_OEMAKE'><filename>EXTRA_OEMAKE</filename></ulink> - or - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGECONFIG_CONFARGS'><filename>PACKAGECONFIG_CONFARGS</filename></ulink> - variables. - BitBake passes these options into the GNU <filename>make</filename> invocation. - Note that a <filename>do_install</filename> task is still required. - Otherwise, BitBake runs an empty <filename>do_install</filename> task by default. - </para> - - <para> - Some applications might require extra parameters to be passed to the compiler. - For example, the application might need an additional header path. - You can accomplish this by adding to the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-CFLAGS'>CFLAGS</ulink></filename> variable. - The following example shows this: - <literallayout class='monospaced'> - CFLAGS_prepend = "-I ${S}/include " - </literallayout> - </para> - - <para> - In the following example, <filename>mtd-utils</filename> is a makefile-based package: - <literallayout class='monospaced'> - SUMMARY = "Tools for managing memory technology devices" - SECTION = "base" - DEPENDS = "zlib lzo e2fsprogs util-linux" - HOMEPAGE = "http://www.linux-mtd.infradead.org/" - LICENSE = "GPLv2+" - LIC_FILES_CHKSUM = "file://COPYING;md5=0636e73ff0215e8d672dc4c32c317bb3 \ - file://include/common.h;beginline=1;endline=17;md5=ba05b07912a44ea2bf81ce409380049c" - - # Use the latest version at 26 Oct, 2013 - SRCREV = "9f107132a6a073cce37434ca9cda6917dd8d866b" - SRC_URI = "git://git.infradead.org/mtd-utils.git \ - file://add-exclusion-to-mkfs-jffs2-git-2.patch \ - " - - PV = "1.5.1+git${SRCPV}" - - S = "${WORKDIR}/git" - - EXTRA_OEMAKE = "'CC=${CC}' 'RANLIB=${RANLIB}' 'AR=${AR}' 'CFLAGS=${CFLAGS} -I${S}/include -DWITHOUT_XATTR' 'BUILDDIR=${S}'" - - do_install () { - oe_runmake install DESTDIR=${D} SBINDIR=${sbindir} MANDIR=${mandir} INCLUDEDIR=${includedir} - } - - PACKAGES =+ "mtd-utils-jffs2 mtd-utils-ubifs mtd-utils-misc" - - FILES_mtd-utils-jffs2 = "${sbindir}/mkfs.jffs2 ${sbindir}/jffs2dump ${sbindir}/jffs2reader ${sbindir}/sumtool" - FILES_mtd-utils-ubifs = "${sbindir}/mkfs.ubifs ${sbindir}/ubi*" - FILES_mtd-utils-misc = "${sbindir}/nftl* ${sbindir}/ftl* ${sbindir}/rfd* ${sbindir}/doc* ${sbindir}/serve_image ${sbindir}/recv_image" - - PARALLEL_MAKE = "" - - BBCLASSEXTEND = "native" - </literallayout> - </para> - </section> - - <section id='splitting-an-application-into-multiple-packages'> - <title>Splitting an Application into Multiple Packages</title> - - <para> - You can use the variables - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'>PACKAGES</ulink></filename> and - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'>FILES</ulink></filename> - to split an application into multiple packages. - </para> - - <para> - Following is an example that uses the <filename>libxpm</filename> recipe. - By default, this recipe generates a single package that contains the library along - with a few binaries. - You can modify the recipe to split the binaries into separate packages: - <literallayout class='monospaced'> - require xorg-lib-common.inc - - SUMMARY = "Xpm: X Pixmap extension library" - LICENSE = "BSD" - LIC_FILES_CHKSUM = "file://COPYING;md5=51f4270b012ecd4ab1a164f5f4ed6cf7" - DEPENDS += "libxext libsm libxt" - PE = "1" - - XORG_PN = "libXpm" - - PACKAGES =+ "sxpm cxpm" - FILES_cxpm = "${bindir}/cxpm" - FILES_sxpm = "${bindir}/sxpm" - </literallayout> - </para> - - <para> - In the previous example, we want to ship the <filename>sxpm</filename> - and <filename>cxpm</filename> binaries in separate packages. - Since <filename>bindir</filename> would be packaged into the main - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'>PN</ulink></filename> - package by default, we prepend the <filename>PACKAGES</filename> - variable so additional package names are added to the start of list. - This results in the extra <filename>FILES_*</filename> - variables then containing information that define which files and - directories go into which packages. - Files included by earlier packages are skipped by latter packages. - Thus, the main <filename>PN</filename> package - does not include the above listed files. - </para> - </section> - - <section id='packaging-externally-produced-binaries'> - <title>Packaging Externally Produced Binaries</title> - - <para> - Sometimes, you need to add pre-compiled binaries to an - image. - For example, suppose that binaries for proprietary code - exist, which are created by a particular division of a - company. - Your part of the company needs to use those binaries as - part of an image that you are building using the - OpenEmbedded build system. - Since you only have the binaries and not the source code, - you cannot use a typical recipe that expects to fetch the - source specified in - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - and then compile it. - </para> - - <para> - One method is to package the binaries and then install them - as part of the image. - Generally, it is not a good idea to package binaries - since, among other things, it can hinder the ability to - reproduce builds and could lead to compatibility problems - with ABI in the future. - However, sometimes you have no choice. - </para> - - <para> - The easiest solution is to create a recipe that uses - the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-bin-package'><filename>bin_package</filename></ulink> - class and to be sure that you are using default locations - for build artifacts. - In most cases, the <filename>bin_package</filename> class - handles "skipping" the configure and compile steps as well - as sets things up to grab packages from the appropriate - area. - In particular, this class sets <filename>noexec</filename> - on both the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink> - tasks, sets - <filename>FILES_${PN}</filename> to "/" so that it picks - up all files, and sets up a - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink> - task, which effectively copies all files from - <filename>${S}</filename> to <filename>${D}</filename>. - The <filename>bin_package</filename> class works well when - the files extracted into <filename>${S}</filename> are - already laid out in the way they should be laid out - on the target. - For more information on these variables, see the - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>, - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink> - variables in the Yocto Project Reference Manual's variable - glossary. - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - Using - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> - is a good idea even for components distributed - in binary form, and is often necessary for - shared libraries. - For a shared library, listing the library - dependencies in - <filename>DEPENDS</filename> makes sure that - the libraries are available in the staging - sysroot when other recipes link against the - library, which might be necessary for - successful linking. - </para></listitem> - <listitem><para> - Using <filename>DEPENDS</filename> also - allows runtime dependencies between packages - to be added automatically. - See the - "<ulink url='&YOCTO_DOCS_OM_URL;#automatically-added-runtime-dependencies'>Automatically Added Runtime Dependencies</ulink>" - section in the Yocto Project Overview and - Concepts Manual for more information. - </para></listitem> - </itemizedlist> - </note> - </para> - - <para> - If you cannot use the <filename>bin_package</filename> - class, you need to be sure you are doing the following: - <itemizedlist> - <listitem><para> - Create a recipe where the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink> - tasks do nothing: - It is usually sufficient to just not define these - tasks in the recipe, because the default - implementations do nothing unless a Makefile is - found in - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>. - </para> - - <para>If - <filename>${S}</filename> might contain a Makefile, - or if you inherit some class that replaces - <filename>do_configure</filename> and - <filename>do_compile</filename> with custom - versions, then you can use the - <filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>noexec</filename></ulink><filename>]</filename> - flag to turn the tasks into no-ops, as follows: - <literallayout class='monospaced'> - do_configure[noexec] = "1" - do_compile[noexec] = "1" - </literallayout> - Unlike - <ulink url='&YOCTO_DOCS_BB_URL;#deleting-a-task'><filename>deleting the tasks</filename></ulink>, - using the flag preserves the dependency chain from - the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink>, <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-unpack'><filename>do_unpack</filename></ulink>, - and - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> - tasks to the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink> - task. - </para></listitem> - <listitem><para>Make sure your - <filename>do_install</filename> task installs the - binaries appropriately. - </para></listitem> - <listitem><para>Ensure that you set up - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink> - (usually - <filename>FILES_${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink><filename>}</filename>) - to point to the files you have installed, which of - course depends on where you have installed them - and whether those files are in different locations - than the defaults. - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id="following-recipe-style-guidelines"> - <title>Following Recipe Style Guidelines</title> - - <para> - When writing recipes, it is good to conform to existing - style guidelines. - The - <ulink url='http://www.openembedded.org/wiki/Styleguide'>OpenEmbedded Styleguide</ulink> - wiki page provides rough guidelines for preferred recipe style. - </para> - - <para> - It is common for existing recipes to deviate a bit from this - style. - However, aiming for at least a consistent style is a good idea. - Some practices, such as omitting spaces around - <filename>=</filename> operators in assignments or ordering - recipe components in an erratic way, are widely seen as poor - style. - </para> - </section> - - <section id='recipe-syntax'> - <title>Recipe Syntax</title> - - <para> - Understanding recipe file syntax is important for writing - recipes. - The following list overviews the basic items that make up a - BitBake recipe file. - For more complete BitBake syntax descriptions, see the - "<ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink>" - chapter of the BitBake User Manual. - <itemizedlist> - <listitem><para> - <emphasis>Variable Assignments and Manipulations:</emphasis> - Variable assignments allow a value to be assigned to a - variable. - The assignment can be static text or might include - the contents of other variables. - In addition to the assignment, appending and prepending - operations are also supported.</para> - - <para>The following example shows some of the ways - you can use variables in recipes: - <literallayout class='monospaced'> - S = "${WORKDIR}/postfix-${PV}" - CFLAGS += "-DNO_ASM" - SRC_URI_append = " file://fixup.patch" - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Functions:</emphasis> - Functions provide a series of actions to be performed. - You usually use functions to override the default - implementation of a task function or to complement - a default function (i.e. append or prepend to an - existing function). - Standard functions use <filename>sh</filename> shell - syntax, although access to OpenEmbedded variables and - internal methods are also available.</para> - - <para>The following is an example function from the - <filename>sed</filename> recipe: - <literallayout class='monospaced'> - do_install () { - autotools_do_install - install -d ${D}${base_bindir} - mv ${D}${bindir}/sed ${D}${base_bindir}/sed - rmdir ${D}${bindir}/ - } - </literallayout> - It is also possible to implement new functions that - are called between existing tasks as long as the - new functions are not replacing or complementing the - default functions. - You can implement functions in Python - instead of shell. - Both of these options are not seen in the majority of - recipes. - </para></listitem> - <listitem><para><emphasis>Keywords:</emphasis> - BitBake recipes use only a few keywords. - You use keywords to include common - functions (<filename>inherit</filename>), load parts - of a recipe from other files - (<filename>include</filename> and - <filename>require</filename>) and export variables - to the environment (<filename>export</filename>). - </para> - - <para>The following example shows the use of some of - these keywords: - <literallayout class='monospaced'> - export POSTCONF = "${STAGING_BINDIR}/postconf" - inherit autoconf - require otherfile.inc - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Comments (#):</emphasis> - Any lines that begin with the hash character - (<filename>#</filename>) are treated as comment lines - and are ignored: - <literallayout class='monospaced'> - # This is a comment - </literallayout> - </para></listitem> - </itemizedlist> - </para> - - <para> - This next list summarizes the most important and most commonly - used parts of the recipe syntax. - For more information on these parts of the syntax, you can - reference the - <ulink url='&YOCTO_DOCS_BB_URL;#bitbake-user-manual-metadata'>Syntax and Operators</ulink> - chapter in the BitBake User Manual. - <itemizedlist> - <listitem><para> - <emphasis>Line Continuation (\):</emphasis> - Use the backward slash (<filename>\</filename>) - character to split a statement over multiple lines. - Place the slash character at the end of the line that - is to be continued on the next line: - <literallayout class='monospaced'> - VAR = "A really long \ - line" - </literallayout> - <note> - You cannot have any characters including spaces - or tabs after the slash character. - </note> - </para></listitem> - <listitem><para> - <emphasis>Using Variables (${<replaceable>VARNAME</replaceable>}):</emphasis> - Use the <filename>${<replaceable>VARNAME</replaceable>}</filename> - syntax to access the contents of a variable: - <literallayout class='monospaced'> - SRC_URI = "${SOURCEFORGE_MIRROR}/libpng/zlib-${PV}.tar.gz" - </literallayout> - <note> - It is important to understand that the value of a - variable expressed in this form does not get - substituted automatically. - The expansion of these expressions happens - on-demand later (e.g. usually when a function that - makes reference to the variable executes). - This behavior ensures that the values are most - appropriate for the context in which they are - finally used. - On the rare occasion that you do need the variable - expression to be expanded immediately, you can use - the <filename>:=</filename> operator instead of - <filename>=</filename> when you make the - assignment, but this is not generally needed. - </note> - </para></listitem> - <listitem><para> - <emphasis>Quote All Assignments ("<replaceable>value</replaceable>"):</emphasis> - Use double quotes around values in all variable - assignments (e.g. - <filename>"<replaceable>value</replaceable>"</filename>). - Following is an example: - <literallayout class='monospaced'> - VAR1 = "${OTHERVAR}" - VAR2 = "The version is ${PV}" - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Conditional Assignment (?=):</emphasis> - Conditional assignment is used to assign a - value to a variable, but only when the variable is - currently unset. - Use the question mark followed by the equal sign - (<filename>?=</filename>) to make a "soft" assignment - used for conditional assignment. - Typically, "soft" assignments are used in the - <filename>local.conf</filename> file for variables - that are allowed to come through from the external - environment. - </para> - - <para>Here is an example where - <filename>VAR1</filename> is set to "New value" if - it is currently empty. - However, if <filename>VAR1</filename> has already been - set, it remains unchanged: - <literallayout class='monospaced'> - VAR1 ?= "New value" - </literallayout> - In this next example, <filename>VAR1</filename> - is left with the value "Original value": - <literallayout class='monospaced'> - VAR1 = "Original value" - VAR1 ?= "New value" - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Appending (+=):</emphasis> - Use the plus character followed by the equals sign - (<filename>+=</filename>) to append values to existing - variables. - <note> - This operator adds a space between the existing - content of the variable and the new content. - </note></para> - - <para>Here is an example: - <literallayout class='monospaced'> - SRC_URI += "file://fix-makefile.patch" - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Prepending (=+):</emphasis> - Use the equals sign followed by the plus character - (<filename>=+</filename>) to prepend values to existing - variables. - <note> - This operator adds a space between the new content - and the existing content of the variable. - </note></para> - - <para>Here is an example: - <literallayout class='monospaced'> - VAR =+ "Starts" - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Appending (_append):</emphasis> - Use the <filename>_append</filename> operator to - append values to existing variables. - This operator does not add any additional space. - Also, the operator is applied after all the - <filename>+=</filename>, and - <filename>=+</filename> operators have been applied and - after all <filename>=</filename> assignments have - occurred. - </para> - - <para>The following example shows the space being - explicitly added to the start to ensure the appended - value is not merged with the existing value: - <literallayout class='monospaced'> - SRC_URI_append = " file://fix-makefile.patch" - </literallayout> - You can also use the <filename>_append</filename> - operator with overrides, which results in the actions - only being performed for the specified target or - machine: - <literallayout class='monospaced'> - SRC_URI_append_sh4 = " file://fix-makefile.patch" - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Prepending (_prepend):</emphasis> - Use the <filename>_prepend</filename> operator to - prepend values to existing variables. - This operator does not add any additional space. - Also, the operator is applied after all the - <filename>+=</filename>, and - <filename>=+</filename> operators have been applied and - after all <filename>=</filename> assignments have - occurred. - </para> - - <para>The following example shows the space being - explicitly added to the end to ensure the prepended - value is not merged with the existing value: - <literallayout class='monospaced'> - CFLAGS_prepend = "-I${S}/myincludes " - </literallayout> - You can also use the <filename>_prepend</filename> - operator with overrides, which results in the actions - only being performed for the specified target or - machine: - <literallayout class='monospaced'> - CFLAGS_prepend_sh4 = "-I${S}/myincludes " - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Overrides:</emphasis> - You can use overrides to set a value conditionally, - typically based on how the recipe is being built. - For example, to set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-KBRANCH'><filename>KBRANCH</filename></ulink> - variable's value to "standard/base" for any target - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>, - except for qemuarm where it should be set to - "standard/arm-versatile-926ejs", you would do the - following: - <literallayout class='monospaced'> - KBRANCH = "standard/base" - KBRANCH_qemuarm = "standard/arm-versatile-926ejs" - </literallayout> - Overrides are also used to separate alternate values - of a variable in other situations. - For example, when setting variables such as - <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink> - that are specific to individual packages produced by - a recipe, you should always use an override that - specifies the name of the package. - </para></listitem> - <listitem><para> - <emphasis>Indentation:</emphasis> - Use spaces for indentation rather than than tabs. - For shell functions, both currently work. - However, it is a policy decision of the Yocto Project - to use tabs in shell functions. - Realize that some layers have a policy to use spaces - for all indentation. - </para></listitem> - <listitem><para> - <emphasis>Using Python for Complex Operations:</emphasis> - For more advanced processing, it is possible to use - Python code during variable assignments (e.g. - search and replacement on a variable).</para> - - <para>You indicate Python code using the - <filename>${@<replaceable>python_code</replaceable>}</filename> - syntax for the variable assignment: - <literallayout class='monospaced'> - SRC_URI = "ftp://ftp.info-zip.org/pub/infozip/src/zip${@d.getVar('PV',1).replace('.', '')}.tgz - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Shell Function Syntax:</emphasis> - Write shell functions as if you were writing a shell - script when you describe a list of actions to take. - You should ensure that your script works with a generic - <filename>sh</filename> and that it does not require - any <filename>bash</filename> or other shell-specific - functionality. - The same considerations apply to various system - utilities (e.g. <filename>sed</filename>, - <filename>grep</filename>, <filename>awk</filename>, - and so forth) that you might wish to use. - If in doubt, you should check with multiple - implementations - including those from BusyBox. - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id="platdev-newmachine"> - <title>Adding a New Machine</title> - - <para> - Adding a new machine to the Yocto Project is a straightforward - process. - This section describes how to add machines that are similar - to those that the Yocto Project already supports. - <note> - Although well within the capabilities of the Yocto Project, - adding a totally new architecture might require - changes to <filename>gcc/glibc</filename> and to the site - information, which is beyond the scope of this manual. - </note> - </para> - - <para> - For a complete example that shows how to add a new machine, - see the - "<ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script'>Creating a New BSP Layer Using the <filename>bitbake-layers</filename> Script</ulink>" - section in the Yocto Project Board Support Package (BSP) - Developer's Guide. - </para> - - <section id="platdev-newmachine-conffile"> - <title>Adding the Machine Configuration File</title> - - <para> - To add a new machine, you need to add a new machine - configuration file to the layer's - <filename>conf/machine</filename> directory. - This configuration file provides details about the device - you are adding. - </para> - - <para> - The OpenEmbedded build system uses the root name of the - machine configuration file to reference the new machine. - For example, given a machine configuration file named - <filename>crownbay.conf</filename>, the build system - recognizes the machine as "crownbay". - </para> - - <para> - The most important variables you must set in your machine - configuration file or include from a lower-level configuration - file are as follows: - <itemizedlist> - <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_ARCH'>TARGET_ARCH</ulink></filename> - (e.g. "arm")</para></listitem> - <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</ulink>_virtual/kernel</filename> - </para></listitem> - <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'>MACHINE_FEATURES</ulink></filename> - (e.g. "apm screen wifi")</para></listitem> - </itemizedlist> - </para> - - <para> - You might also need these variables: - <itemizedlist> - <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES'>SERIAL_CONSOLES</ulink></filename> - (e.g. "115200;ttyS0 115200;ttyS1")</para></listitem> - <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-KERNEL_IMAGETYPE'>KERNEL_IMAGETYPE</ulink></filename> - (e.g. "zImage")</para></listitem> - <listitem><para><filename><ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'>IMAGE_FSTYPES</ulink></filename> - (e.g. "tar.gz jffs2")</para></listitem> - </itemizedlist> - </para> - - <para> - You can find full details on these variables in the reference - section. - You can leverage existing machine <filename>.conf</filename> - files from <filename>meta-yocto-bsp/conf/machine/</filename>. - </para> - </section> - - <section id="platdev-newmachine-kernel"> - <title>Adding a Kernel for the Machine</title> - - <para> - The OpenEmbedded build system needs to be able to build a kernel - for the machine. - You need to either create a new kernel recipe for this machine, - or extend an existing kernel recipe. - You can find several kernel recipe examples in the - Source Directory at - <filename>meta/recipes-kernel/linux</filename> - that you can use as references. - </para> - - <para> - If you are creating a new kernel recipe, normal recipe-writing - rules apply for setting up a - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename>. - Thus, you need to specify any necessary patches and set - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'>S</ulink></filename> - to point at the source code. - You need to create a <filename>do_configure</filename> task that - configures the unpacked kernel with a - <filename>defconfig</filename> file. - You can do this by using a <filename>make defconfig</filename> - command or, more commonly, by copying in a suitable - <filename>defconfig</filename> file and then running - <filename>make oldconfig</filename>. - By making use of <filename>inherit kernel</filename> and - potentially some of the <filename>linux-*.inc</filename> files, - most other functionality is centralized and the defaults of the - class normally work well. - </para> - - <para> - If you are extending an existing kernel recipe, it is usually - a matter of adding a suitable <filename>defconfig</filename> - file. - The file needs to be added into a location similar to - <filename>defconfig</filename> files used for other machines - in a given kernel recipe. - A possible way to do this is by listing the file in the - <filename>SRC_URI</filename> and adding the machine to the - expression in - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-COMPATIBLE_MACHINE'>COMPATIBLE_MACHINE</ulink></filename>: - <literallayout class='monospaced'> - COMPATIBLE_MACHINE = '(qemux86|qemumips)' - </literallayout> - For more information on <filename>defconfig</filename> files, - see the - "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#changing-the-configuration'>Changing the Configuration</ulink>" - section in the Yocto Project Linux Kernel Development Manual. - </para> - </section> - - <section id="platdev-newmachine-formfactor"> - <title>Adding a Formfactor Configuration File</title> - - <para> - A formfactor configuration file provides information about the - target hardware for which the image is being built and information that - the build system cannot obtain from other sources such as the kernel. - Some examples of information contained in a formfactor configuration file include - framebuffer orientation, whether or not the system has a keyboard, - the positioning of the keyboard in relation to the screen, and - the screen resolution. - </para> - - <para> - The build system uses reasonable defaults in most cases. - However, if customization is - necessary, you need to create a <filename>machconfig</filename> file - in the <filename>meta/recipes-bsp/formfactor/files</filename> - directory. - This directory contains directories for specific machines such as - <filename>qemuarm</filename> and <filename>qemux86</filename>. - For information about the settings available and the defaults, see the - <filename>meta/recipes-bsp/formfactor/files/config</filename> file found in the - same area. - </para> - - <para> - Following is an example for "qemuarm" machine: - <literallayout class='monospaced'> - HAVE_TOUCHSCREEN=1 - HAVE_KEYBOARD=1 - - DISPLAY_CAN_ROTATE=0 - DISPLAY_ORIENTATION=0 - #DISPLAY_WIDTH_PIXELS=640 - #DISPLAY_HEIGHT_PIXELS=480 - #DISPLAY_BPP=16 - DISPLAY_DPI=150 - DISPLAY_SUBPIXEL_ORDER=vrgb - </literallayout> - </para> - </section> - </section> - - <section id='gs-upgrading-recipes'> - <title>Upgrading Recipes</title> - - <para> - Over time, upstream developers publish new versions for software - built by layer recipes. - It is recommended to keep recipes up-to-date with upstream - version releases. - </para> - - <para> - While several methods exist that allow you upgrade a recipe, - you might consider checking on the upgrade status of a recipe - first. - You can do so using the - <filename>devtool check-upgrade-status</filename> command. - See the - "<ulink url='&YOCTO_DOCS_REF_URL;#devtool-checking-on-the-upgrade-status-of-a-recipe'>Checking on the Upgrade Status of a Recipe</ulink>" - section in the Yocto Project Reference Manual for more information. - </para> - - <para> - The remainder of this section describes three ways you can - upgrade a recipe. - You can use the Automated Upgrade Helper (AUH) to set up - automatic version upgrades. - Alternatively, you can use <filename>devtool upgrade</filename> - to set up semi-automatic version upgrades. - Finally, you can manually upgrade a recipe by editing the - recipe itself. - </para> - - <section id='gs-using-the-auto-upgrade-helper'> - <title>Using the Auto Upgrade Helper (AUH)</title> - - <para> - The AUH utility works in conjunction with the - OpenEmbedded build system in order to automatically generate - upgrades for recipes based on new versions being - published upstream. - Use AUH when you want to create a service that performs the - upgrades automatically and optionally sends you an email with - the results. - </para> - - <para> - AUH allows you to update several recipes with a single use. - You can also optionally perform build and integration tests - using images with the results saved to your hard drive and - emails of results optionally sent to recipe maintainers. - Finally, AUH creates Git commits with appropriate commit - messages in the layer's tree for the changes made to recipes. - <note> - Conditions do exist when you should not use AUH to upgrade - recipes and you should instead use either - <filename>devtool upgrade</filename> or upgrade your - recipes manually: - <itemizedlist> - <listitem><para> - When AUH cannot complete the upgrade sequence. - This situation usually results because custom - patches carried by the recipe cannot be - automatically rebased to the new version. - In this case, <filename>devtool upgrade</filename> - allows you to manually resolve conflicts. - </para></listitem> - <listitem><para> - When for any reason you want fuller control over - the upgrade process. - For example, when you want special arrangements - for testing. - </para></listitem> - </itemizedlist> - </note> - </para> - - <para> - The following steps describe how to set up the AUH utility: - <orderedlist> - <listitem><para> - <emphasis>Be Sure the Development Host is Set Up:</emphasis> - You need to be sure that your development host is - set up to use the Yocto Project. - For information on how to set up your host, see the - "<link linkend='dev-preparing-the-build-host'>Preparing the Build Host</link>" - section. - </para></listitem> - <listitem><para> - <emphasis>Make Sure Git is Configured:</emphasis> - The AUH utility requires Git to be configured because - AUH uses Git to save upgrades. - Thus, you must have Git user and email configured. - The following command shows your configurations: - <literallayout class='monospaced'> - $ git config --list - </literallayout> - If you do not have the user and email configured, you - can use the following commands to do so: - <literallayout class='monospaced'> - $ git config --global user.name <replaceable>some_name</replaceable> - $ git config --global user.email <replaceable>username</replaceable>@<replaceable>domain</replaceable>.com - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Clone the AUH Repository:</emphasis> - To use AUH, you must clone the repository onto your - development host. - The following command uses Git to create a local - copy of the repository on your system: - <literallayout class='monospaced'> - $ git clone git://git.yoctoproject.org/auto-upgrade-helper - Cloning into 'auto-upgrade-helper'... - remote: Counting objects: 768, done. - remote: Compressing objects: 100% (300/300), done. - remote: Total 768 (delta 499), reused 703 (delta 434) - Receiving objects: 100% (768/768), 191.47 KiB | 98.00 KiB/s, done. - Resolving deltas: 100% (499/499), done. - Checking connectivity... done. - </literallayout> - AUH is not part of the - <ulink url='&YOCTO_DOCS_REF_URL;#oe-core'>OpenEmbedded-Core (OE-Core)</ulink> - or - <ulink url='&YOCTO_DOCS_REF_URL;#poky'>Poky</ulink> - repositories. - </para></listitem> - <listitem><para> - <emphasis>Create a Dedicated Build Directory:</emphasis> - Run the - <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>oe-init-build-env</filename></ulink> - script to create a fresh build directory that you - use exclusively for running the AUH utility: - <literallayout class='monospaced'> - $ cd ~/poky - $ source oe-init-build-env <replaceable>your_AUH_build_directory</replaceable> - </literallayout> - Re-using an existing build directory and its - configurations is not recommended as existing settings - could cause AUH to fail or behave undesirably. - </para></listitem> - <listitem><para> - <emphasis>Make Configurations in Your Local Configuration File:</emphasis> - Several settings need to exist in the - <filename>local.conf</filename> file in the build - directory you just created for AUH. - Make these following configurations: - <itemizedlist> - <listitem><para> - If you want to enable - <ulink url='&YOCTO_DOCS_DEV_URL;#maintaining-build-output-quality'>Build History</ulink>, - which is optional, you need the following - lines in the - <filename>conf/local.conf</filename> file: - <literallayout class='monospaced'> - INHERIT =+ "buildhistory" - BUILDHISTORY_COMMIT = "1" - </literallayout> - With this configuration and a successful - upgrade, a build history "diff" file appears in - the - <filename>upgrade-helper/work/recipe/buildhistory-diff.txt</filename> - file found in your build directory. - </para></listitem> - <listitem><para> - If you want to enable testing through the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink> - class, which is optional, you need to have the - following set in your - <filename>conf/local.conf</filename> file: - <literallayout class='monospaced'> - INHERIT += "testimage" - </literallayout> - <note> - If your distro does not enable by default - ptest, which Poky does, you need the - following in your - <filename>local.conf</filename> file: - <literallayout class='monospaced'> - DISTRO_FEATURES_append = " ptest" - </literallayout> - </note> - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <emphasis>Optionally Start a vncserver:</emphasis> - If you are running in a server without an X11 session, - you need to start a vncserver: - <literallayout class='monospaced'> - $ vncserver :1 - $ export DISPLAY=:1 - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Create and Edit an AUH Configuration File:</emphasis> - You need to have the - <filename>upgrade-helper/upgrade-helper.conf</filename> - configuration file in your build directory. - You can find a sample configuration file in the - <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/'>AUH source repository</ulink>. - </para> - - <para>Read through the sample file and make - configurations as needed. - For example, if you enabled build history in your - <filename>local.conf</filename> as described earlier, - you must enable it in - <filename>upgrade-helper.conf</filename>.</para> - - <para>Also, if you are using the default - <filename>maintainers.inc</filename> file supplied - with Poky and located in - <filename>meta-yocto</filename> and you do not set a - "maintainers_whitelist" or "global_maintainer_override" - in the <filename>upgrade-helper.conf</filename> - configuration, and you specify "-e all" on the - AUH command-line, the utility automatically sends out - emails to all the default maintainers. - Please avoid this. - </para></listitem> - </orderedlist> - </para> - - <para> - This next set of examples describes how to use the AUH: - <itemizedlist> - <listitem><para> - <emphasis>Upgrading a Specific Recipe:</emphasis> - To upgrade a specific recipe, use the following - form: - <literallayout class='monospaced'> - $ upgrade-helper.py <replaceable>recipe_name</replaceable> - </literallayout> - For example, this command upgrades the - <filename>xmodmap</filename> recipe: - <literallayout class='monospaced'> - $ upgrade-helper.py xmodmap - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Upgrading a Specific Recipe to a Particular Version:</emphasis> - To upgrade a specific recipe to a particular version, - use the following form: - <literallayout class='monospaced'> - $ upgrade-helper.py <replaceable>recipe_name</replaceable> -t <replaceable>version</replaceable> - </literallayout> - For example, this command upgrades the - <filename>xmodmap</filename> recipe to version - 1.2.3: - <literallayout class='monospaced'> - $ upgrade-helper.py xmodmap -t 1.2.3 - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Upgrading all Recipes to the Latest Versions and Suppressing Email Notifications:</emphasis> - To upgrade all recipes to their most recent versions - and suppress the email notifications, use the following - command: - <literallayout class='monospaced'> - $ upgrade-helper.py all - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Upgrading all Recipes to the Latest Versions and Send Email Notifications:</emphasis> - To upgrade all recipes to their most recent versions - and send email messages to maintainers for each - attempted recipe as well as a status email, use the - following command: - <literallayout class='monospaced'> - $ upgrade-helper.py -e all - </literallayout> - </para></listitem> - </itemizedlist> - </para> - - <para> - Once you have run the AUH utility, you can find the results - in the AUH build directory: - <literallayout class='monospaced'> - ${BUILDDIR}/upgrade-helper/<replaceable>timestamp</replaceable> - </literallayout> - The AUH utility also creates recipe update commits from - successful upgrade attempts in the layer tree. - </para> - - <para> - You can easily set up to run the AUH utility on a regular - basis by using a cron job. - See the - <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/auto-upgrade-helper/tree/weeklyjob.sh'><filename>weeklyjob.sh</filename></ulink> - file distributed with the utility for an example. - </para> - </section> - - <section id='gs-using-devtool-upgrade'> - <title>Using <filename>devtool upgrade</filename></title> - - <para> - As mentioned earlier, an alternative method for upgrading - recipes to newer versions is to use - <ulink url='&YOCTO_DOCS_REF_URL;#ref-devtool-reference'><filename>devtool upgrade</filename></ulink>. - You can read about <filename>devtool upgrade</filename> in - general in the - "<ulink url='&YOCTO_DOCS_SDK_URL;#sdk-devtool-use-devtool-upgrade-to-create-a-version-of-the-recipe-that-supports-a-newer-version-of-the-software'>Use <filename>devtool upgrade</filename> to Create a Version of the Recipe that Supports a Newer Version of the Software</ulink>" - section in the Yocto Project Application Development and the - Extensible Software Development Kit (eSDK) Manual. - </para> - - <para> - To see all the command-line options available with - <filename>devtool upgrade</filename>, use the following help - command: - <literallayout class='monospaced'> - $ devtool upgrade -h - </literallayout> - </para> - - <para> - If you want to find out what version a recipe is currently at - upstream without any attempt to upgrade your local version of - the recipe, you can use the following command: - <literallayout class='monospaced'> - $ devtool latest-version <replaceable>recipe_name</replaceable> - </literallayout> - </para> - - <para> - As mentioned in the previous section describing AUH, - <filename>devtool upgrade</filename> works in a - less-automated manner than AUH. - Specifically, <filename>devtool upgrade</filename> only - works on a single recipe that you name on the command line, - cannot perform build and integration testing using images, - and does not automatically generate commits for changes in - the source tree. - Despite all these "limitations", - <filename>devtool upgrade</filename> updates the recipe file - to the new upstream version and attempts to rebase custom - patches contained by the recipe as needed. - <note> - AUH uses much of <filename>devtool upgrade</filename> - behind the scenes making AUH somewhat of a "wrapper" - application for <filename>devtool upgrade</filename>. - </note> - </para> - - <para> - A typical scenario involves having used Git to clone an - upstream repository that you use during build operations. - Because you are (or have) built the recipe in the past, the - layer is likely added to your configuration already. - If for some reason, the layer is not added, you could add - it easily using the - <ulink url='&YOCTO_DOCS_BSP_URL;#creating-a-new-bsp-layer-using-the-bitbake-layers-script'><filename>bitbake-layers</filename></ulink> - script. - For example, suppose you use the <filename>nano.bb</filename> - recipe from the <filename>meta-oe</filename> layer in the - <filename>meta-openembedded</filename> repository. - For this example, assume that the layer has been cloned into - following area: - <literallayout class='monospaced'> - /home/scottrif/meta-openembedded - </literallayout> - The following command from your - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - adds the layer to your build configuration (i.e. - <filename>${BUILDDIR}/conf/bblayers.conf</filename>): - <literallayout class='monospaced'> - $ bitbake-layers add-layer /home/scottrif/meta-openembedded/meta-oe - NOTE: Starting bitbake server... - Parsing recipes: 100% |##########################################| Time: 0:00:55 - Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. - Removing 12 recipes from the x86_64 sysroot: 100% |##############| Time: 0:00:00 - Removing 1 recipes from the x86_64_i586 sysroot: 100% |##########| Time: 0:00:00 - Removing 5 recipes from the i586 sysroot: 100% |#################| Time: 0:00:00 - Removing 5 recipes from the qemux86 sysroot: 100% |##############| Time: 0:00:00 - </literallayout> - For this example, assume that the <filename>nano.bb</filename> - recipe that is upstream has a 2.9.3 version number. - However, the version in the local repository is 2.7.4. - The following command from your build directory automatically - upgrades the recipe for you: - <note> - Using the <filename>-V</filename> option is not necessary. - Omitting the version number causes - <filename>devtool upgrade</filename> to upgrade the recipe - to the most recent version. - </note> - <literallayout class='monospaced'> - $ devtool upgrade nano -V 2.9.3 - NOTE: Starting bitbake server... - NOTE: Creating workspace layer in /home/scottrif/poky/build/workspace - Parsing recipes: 100% |##########################################| Time: 0:00:46 - Parsing of 1431 .bb files complete (0 cached, 1431 parsed). 2040 targets, 56 skipped, 0 masked, 0 errors. - NOTE: Extracting current version source... - NOTE: Resolving any missing task queue dependencies - . - . - . - NOTE: Executing SetScene Tasks - NOTE: Executing RunQueue Tasks - NOTE: Tasks Summary: Attempted 74 tasks of which 72 didn't need to be rerun and all succeeded. - Adding changed files: 100% |#####################################| Time: 0:00:00 - NOTE: Upgraded source extracted to /home/scottrif/poky/build/workspace/sources/nano - NOTE: New recipe is /home/scottrif/poky/build/workspace/recipes/nano/nano_2.9.3.bb - </literallayout> - Continuing with this example, you can use - <filename>devtool build</filename> to build the newly upgraded - recipe: - <literallayout class='monospaced'> - $ devtool build nano - NOTE: Starting bitbake server... - Loading cache: 100% |################################################################################################| Time: 0:00:01 - Loaded 2040 entries from dependency cache. - Parsing recipes: 100% |##############################################################################################| Time: 0:00:00 - Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. - NOTE: Resolving any missing task queue dependencies - . - . - . - NOTE: Executing SetScene Tasks - NOTE: Executing RunQueue Tasks - NOTE: nano: compiling from external source tree /home/scottrif/poky/build/workspace/sources/nano - NOTE: Tasks Summary: Attempted 520 tasks of which 304 didn't need to be rerun and all succeeded. - </literallayout> - Within the <filename>devtool upgrade</filename> workflow, - opportunity exists to deploy and test your rebuilt software. - For this example, however, running - <filename>devtool finish</filename> cleans up the workspace - once the source in your workspace is clean. - This usually means using Git to stage and submit commits - for the changes generated by the upgrade process. - </para> - - <para> - Once the tree is clean, you can clean things up in this - example with the following command from the - <filename>${BUILDDIR}/workspace/sources/nano</filename> - directory: - <literallayout class='monospaced'> - $ devtool finish nano meta-oe - NOTE: Starting bitbake server... - Loading cache: 100% |################################################################################################| Time: 0:00:00 - Loaded 2040 entries from dependency cache. - Parsing recipes: 100% |##############################################################################################| Time: 0:00:01 - Parsing of 1432 .bb files complete (1431 cached, 1 parsed). 2041 targets, 56 skipped, 0 masked, 0 errors. - NOTE: Adding new patch 0001-nano.bb-Stuff-I-changed-when-upgrading-nano.bb.patch - NOTE: Updating recipe nano_2.9.3.bb - NOTE: Removing file /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano/nano_2.7.4.bb - NOTE: Moving recipe file to /home/scottrif/meta-openembedded/meta-oe/recipes-support/nano - NOTE: Leaving source tree /home/scottrif/poky/build/workspace/sources/nano as-is; if you no longer need it then please delete it manually - </literallayout> - Using the <filename>devtool finish</filename> command cleans - up the workspace and creates a patch file based on your - commits. - The tool puts all patch files back into the source directory - in a sub-directory named <filename>nano</filename> in this - case. - </para> - </section> - - <section id='dev-manually-upgrading-a-recipe'> - <title>Manually Upgrading a Recipe</title> - - <para> - If for some reason you choose not to upgrade recipes using the - <link linkend='gs-using-the-auto-upgrade-helper'>Auto Upgrade Helper (AUH)</link> - or by using - <link linkend='gs-using-devtool-upgrade'><filename>devtool upgrade</filename></link>, - you can manually edit the recipe files to upgrade the versions. - <note><title>Caution</title> - Manually updating multiple recipes scales poorly and - involves many steps. - The recommendation to upgrade recipe versions is through - AUH or <filename>devtool upgrade</filename>, both of which - automate some steps and provide guidance for others needed - for the manual process. - </note> - </para> - - <para> - To manually upgrade recipe versions, follow these general steps: - <orderedlist> - <listitem><para> - <emphasis>Change the Version:</emphasis> - Rename the recipe such that the version (i.e. the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> - part of the recipe name) changes appropriately. - If the version is not part of the recipe name, change - the value as it is set for <filename>PV</filename> - within the recipe itself. - </para></listitem> - <listitem><para> - <emphasis>Update <filename>SRCREV</filename> if Needed:</emphasis> - If the source code your recipe builds is fetched from - Git or some other version control system, update - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink> - to point to the commit hash that matches the new - version. - </para></listitem> - <listitem><para> - <emphasis>Build the Software:</emphasis> - Try to build the recipe using BitBake. - Typical build failures include the following: - <itemizedlist> - <listitem><para> - License statements were updated for the new - version. - For this case, you need to review any changes - to the license and update the values of - <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink> - as needed. - <note> - License changes are often inconsequential. - For example, the license text's copyright - year might have changed. - </note> - </para></listitem> - <listitem><para> - Custom patches carried by the older version of - the recipe might fail to apply to the new - version. - For these cases, you need to review the - failures. - Patches might not be necessary for the new - version of the software if the upgraded version - has fixed those issues. - If a patch is necessary and failing, you need - to rebase it into the new version. - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <emphasis>Optionally Attempt to Build for Several Architectures:</emphasis> - Once you successfully build the new software for a - given architecture, you could test the build for - other architectures by changing the - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> - variable and rebuilding the software. - This optional step is especially important if the - recipe is to be released publicly. - </para></listitem> - <listitem><para> - <emphasis>Check the Upstream Change Log or Release Notes:</emphasis> - Checking both these reveals if new features exist that - could break backwards-compatibility. - If so, you need to take steps to mitigate or eliminate - that situation. - </para></listitem> - <listitem><para> - <emphasis>Optionally Create a Bootable Image and Test:</emphasis> - If you want, you can test the new software by booting - it onto actual hardware. - </para></listitem> - <listitem><para> - <emphasis>Create a Commit with the Change in the Layer Repository:</emphasis> - After all builds work and any testing is successful, - you can create commits for any changes in the layer - holding your upgraded recipe. - </para></listitem> - </orderedlist> - </para> - </section> - </section> - - <section id='finding-the-temporary-source-code'> - <title>Finding Temporary Source Code</title> - - <para> - You might find it helpful during development to modify the - temporary source code used by recipes to build packages. - For example, suppose you are developing a patch and you need to - experiment a bit to figure out your solution. - After you have initially built the package, you can iteratively - tweak the source code, which is located in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>, - and then you can force a re-compile and quickly test your altered - code. - Once you settle on a solution, you can then preserve your changes - in the form of patches. - </para> - - <para> - During a build, the unpacked temporary source code used by recipes - to build packages is available in the Build Directory as - defined by the - <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> - variable. - Below is the default value for the <filename>S</filename> variable - as defined in the - <filename>meta/conf/bitbake.conf</filename> configuration file - in the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>: - <literallayout class='monospaced'> - S = "${WORKDIR}/${BP}" - </literallayout> - You should be aware that many recipes override the - <filename>S</filename> variable. - For example, recipes that fetch their source from Git usually set - <filename>S</filename> to <filename>${WORKDIR}/git</filename>. - <note> - The - <ulink url='&YOCTO_DOCS_REF_URL;#var-BP'><filename>BP</filename></ulink> - represents the base recipe name, which consists of the name - and version: - <literallayout class='monospaced'> - BP = "${BPN}-${PV}" - </literallayout> - </note> - </para> - - <para> - The path to the work directory for the recipe - (<ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>) - is defined as follows: - <literallayout class='monospaced'> - ${TMPDIR}/work/${MULTIMACH_TARGET_SYS}/${PN}/${EXTENDPE}${PV}-${PR} - </literallayout> - The actual directory depends on several things: - <itemizedlist> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>: - The top-level build output directory. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-MULTIMACH_TARGET_SYS'><filename>MULTIMACH_TARGET_SYS</filename></ulink>: - The target system identifier. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink>: - The recipe name. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTENDPE'><filename>EXTENDPE</filename></ulink>: - The epoch - (if - <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink> - is not specified, which is usually the case for most - recipes, then <filename>EXTENDPE</filename> is blank). - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>: - The recipe version. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>: - The recipe revision. - </para></listitem> - </itemizedlist> - </para> - - <para> - As an example, assume a Source Directory top-level folder - named <filename>poky</filename>, a default Build Directory at - <filename>poky/build</filename>, and a - <filename>qemux86-poky-linux</filename> machine target - system. - Furthermore, suppose your recipe is named - <filename>foo_1.3.0.bb</filename>. - In this case, the work directory the build system uses to - build the package would be as follows: - <literallayout class='monospaced'> - poky/build/tmp/work/qemux86-poky-linux/foo/1.3.0-r0 - </literallayout> - </para> - </section> - - <section id="using-a-quilt-workflow"> - <title>Using Quilt in Your Workflow</title> - - <para> - <ulink url='http://savannah.nongnu.org/projects/quilt'>Quilt</ulink> - is a powerful tool that allows you to capture source code changes - without having a clean source tree. - This section outlines the typical workflow you can use to modify - source code, test changes, and then preserve the changes in the - form of a patch all using Quilt. - <note><title>Tip</title> - With regard to preserving changes to source files, if you - clean a recipe or have <filename>rm_work</filename> enabled, - the - <ulink url='&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow'><filename>devtool</filename> workflow</ulink> - as described in the Yocto Project Application Development - and the Extensible Software Development Kit (eSDK) manual - is a safer development flow than the flow that uses Quilt. - </note> - </para> - - <para> - Follow these general steps: - <orderedlist> - <listitem><para> - <emphasis>Find the Source Code:</emphasis> - Temporary source code used by the OpenEmbedded build system - is kept in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - See the - "<link linkend='finding-the-temporary-source-code'>Finding Temporary Source Code</link>" - section to learn how to locate the directory that has the - temporary source code for a particular package. - </para></listitem> - <listitem><para> - <emphasis>Change Your Working Directory:</emphasis> - You need to be in the directory that has the temporary - source code. - That directory is defined by the - <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> - variable.</para></listitem> - <listitem><para> - <emphasis>Create a New Patch:</emphasis> - Before modifying source code, you need to create a new - patch. - To create a new patch file, use - <filename>quilt new</filename> as below: - <literallayout class='monospaced'> - $ quilt new my_changes.patch - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Notify Quilt and Add Files:</emphasis> - After creating the patch, you need to notify Quilt about - the files you plan to edit. - You notify Quilt by adding the files to the patch you - just created: - <literallayout class='monospaced'> - $ quilt add file1.c file2.c file3.c - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Edit the Files:</emphasis> - Make your changes in the source code to the files you added - to the patch. - </para></listitem> - <listitem><para> - <emphasis>Test Your Changes:</emphasis> - Once you have modified the source code, the easiest way to - test your changes is by calling the - <filename>do_compile</filename> task as shown in the - following example: - <literallayout class='monospaced'> - $ bitbake -c compile -f <replaceable>package</replaceable> - </literallayout> - The <filename>-f</filename> or <filename>--force</filename> - option forces the specified task to execute. - If you find problems with your code, you can just keep - editing and re-testing iteratively until things work - as expected. - <note> - All the modifications you make to the temporary - source code disappear once you run the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-clean'><filename>do_clean</filename></ulink> - or - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-cleanall'><filename>do_cleanall</filename></ulink> - tasks using BitBake (i.e. - <filename>bitbake -c clean <replaceable>package</replaceable></filename> - and - <filename>bitbake -c cleanall <replaceable>package</replaceable></filename>). - Modifications will also disappear if you use the - <filename>rm_work</filename> feature as described - in the - "<ulink url='&YOCTO_DOCS_DEV_URL;#dev-saving-memory-during-a-build'>Conserving Disk Space During Builds</ulink>" - section. - </note> - </para></listitem> - <listitem><para> - <emphasis>Generate the Patch:</emphasis> - Once your changes work as expected, you need to use Quilt - to generate the final patch that contains all your - modifications. - <literallayout class='monospaced'> - $ quilt refresh - </literallayout> - At this point, the <filename>my_changes.patch</filename> - file has all your edits made to the - <filename>file1.c</filename>, <filename>file2.c</filename>, - and <filename>file3.c</filename> files.</para> - - <para>You can find the resulting patch file in the - <filename>patches/</filename> subdirectory of the source - (<filename>S</filename>) directory. - </para></listitem> - <listitem><para> - <emphasis>Copy the Patch File:</emphasis> - For simplicity, copy the patch file into a directory - named <filename>files</filename>, which you can create - in the same directory that holds the recipe - (<filename>.bb</filename>) file or the append - (<filename>.bbappend</filename>) file. - Placing the patch here guarantees that the OpenEmbedded - build system will find the patch. - Next, add the patch into the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'>SRC_URI</ulink></filename> - of the recipe. - Here is an example: - <literallayout class='monospaced'> - SRC_URI += "file://my_changes.patch" - </literallayout> - </para></listitem> - </orderedlist> - </para> - </section> - - <section id="platdev-appdev-devshell"> - <title>Using a Development Shell</title> - - <para> - When debugging certain commands or even when just editing packages, - <filename>devshell</filename> can be a useful tool. - When you invoke <filename>devshell</filename>, all tasks up to and - including - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> - are run for the specified target. - Then, a new terminal is opened and you are placed in - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink><filename>}</filename>, - the source directory. - In the new terminal, all the OpenEmbedded build-related environment variables are - still defined so you can use commands such as <filename>configure</filename> and - <filename>make</filename>. - The commands execute just as if the OpenEmbedded build system were executing them. - Consequently, working this way can be helpful when debugging a build or preparing - software to be used with the OpenEmbedded build system. - </para> - - <para> - Following is an example that uses <filename>devshell</filename> on a target named - <filename>matchbox-desktop</filename>: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -c devshell - </literallayout> - </para> - - <para> - This command spawns a terminal with a shell prompt within the OpenEmbedded build environment. - The <ulink url='&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL'><filename>OE_TERMINAL</filename></ulink> - variable controls what type of shell is opened. - </para> - - <para> - For spawned terminals, the following occurs: - <itemizedlist> - <listitem><para>The <filename>PATH</filename> variable includes the - cross-toolchain.</para></listitem> - <listitem><para>The <filename>pkgconfig</filename> variables find the correct - <filename>.pc</filename> files.</para></listitem> - <listitem><para>The <filename>configure</filename> command finds the - Yocto Project site files as well as any other necessary files.</para></listitem> - </itemizedlist> - </para> - - <para> - Within this environment, you can run configure or compile - commands as if they were being run by - the OpenEmbedded build system itself. - As noted earlier, the working directory also automatically changes to the - Source Directory (<ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink>). - </para> - - <para> - To manually run a specific task using <filename>devshell</filename>, - run the corresponding <filename>run.*</filename> script in - the - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/temp</filename> - directory (e.g., - <filename>run.do_configure.</filename><replaceable>pid</replaceable>). - If a task's script does not exist, which would be the case if the task was - skipped by way of the sstate cache, you can create the task by first running - it outside of the <filename>devshell</filename>: - <literallayout class='monospaced'> - $ bitbake -c <replaceable>task</replaceable> - </literallayout> - <note><title>Notes</title> - <itemizedlist> - <listitem><para>Execution of a task's <filename>run.*</filename> - script and BitBake's execution of a task are identical. - In other words, running the script re-runs the task - just as it would be run using the - <filename>bitbake -c</filename> command. - </para></listitem> - <listitem><para>Any <filename>run.*</filename> file that does not - have a <filename>.pid</filename> extension is a - symbolic link (symlink) to the most recent version of that - file. - </para></listitem> - </itemizedlist> - </note> - </para> - - <para> - Remember, that the <filename>devshell</filename> is a mechanism that allows - you to get into the BitBake task execution environment. - And as such, all commands must be called just as BitBake would call them. - That means you need to provide the appropriate options for - cross-compilation and so forth as applicable. - </para> - - <para> - When you are finished using <filename>devshell</filename>, exit the shell - or close the terminal window. - </para> - - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - It is worth remembering that when using <filename>devshell</filename> - you need to use the full compiler name such as <filename>arm-poky-linux-gnueabi-gcc</filename> - instead of just using <filename>gcc</filename>. - The same applies to other applications such as <filename>binutils</filename>, - <filename>libtool</filename> and so forth. - BitBake sets up environment variables such as <filename>CC</filename> - to assist applications, such as <filename>make</filename> to find the correct tools. - </para></listitem> - <listitem><para> - It is also worth noting that <filename>devshell</filename> still works over - X11 forwarding and similar situations. - </para></listitem> - </itemizedlist> - </note> - </section> - - <section id="platdev-appdev-devpyshell"> - <title>Using a Development Python Shell</title> - - <para> - Similar to working within a development shell as described in - the previous section, you can also spawn and work within an - interactive Python development shell. - When debugging certain commands or even when just editing packages, - <filename>devpyshell</filename> can be a useful tool. - When you invoke <filename>devpyshell</filename>, all tasks up to and - including - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-patch'><filename>do_patch</filename></ulink> - are run for the specified target. - Then a new terminal is opened. - Additionally, key Python objects and code are available in the same - way they are to BitBake tasks, in particular, the data store 'd'. - So, commands such as the following are useful when exploring the data - store and running functions: - <literallayout class='monospaced'> - pydevshell> d.getVar("STAGING_DIR") - '/media/build1/poky/build/tmp/sysroots' - pydevshell> d.getVar("STAGING_DIR") - '${TMPDIR}/sysroots' - pydevshell> d.setVar("FOO", "bar") - pydevshell> d.getVar("FOO") - 'bar' - pydevshell> d.delVar("FOO") - pydevshell> d.getVar("FOO") - pydevshell> bb.build.exec_func("do_unpack", d) - pydevshell> - </literallayout> - The commands execute just as if the OpenEmbedded build system were executing them. - Consequently, working this way can be helpful when debugging a build or preparing - software to be used with the OpenEmbedded build system. - </para> - - <para> - Following is an example that uses <filename>devpyshell</filename> on a target named - <filename>matchbox-desktop</filename>: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -c devpyshell - </literallayout> - </para> - - <para> - This command spawns a terminal and places you in an interactive - Python interpreter within the OpenEmbedded build environment. - The <ulink url='&YOCTO_DOCS_REF_URL;#var-OE_TERMINAL'><filename>OE_TERMINAL</filename></ulink> - variable controls what type of shell is opened. - </para> - - <para> - When you are finished using <filename>devpyshell</filename>, you - can exit the shell either by using Ctrl+d or closing the terminal - window. - </para> - </section> - - <section id='dev-building'> - <title>Building</title> - - <para> - This section describes various build procedures. - For example, the steps needed for a simple build, a target that - uses multiple configurations, building an image for more than - one machine, and so forth. - </para> - - <section id='dev-building-a-simple-image'> - <title>Building a Simple Image</title> - - <para> - In the development environment, you need to build an image - whenever you change hardware support, add or change system - libraries, or add or change services that have dependencies. - Several methods exist that allow you to build an image within - the Yocto Project. - This section presents the basic steps you need to build a - simple image using BitBake from a build host running Linux. - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - For information on how to build an image using - <ulink url='&YOCTO_DOCS_REF_URL;#toaster-term'>Toaster</ulink>, - see the - <ulink url='&YOCTO_DOCS_TOAST_URL;'>Toaster User Manual</ulink>. - </para></listitem> - <listitem><para> - For information on how to use - <filename>devtool</filename> to build images, see - the - "<ulink url='&YOCTO_DOCS_SDK_URL;#using-devtool-in-your-sdk-workflow'>Using <filename>devtool</filename> in Your SDK Workflow</ulink>" - section in the Yocto Project Application - Development and the Extensible Software Development - Kit (eSDK) manual. - </para></listitem> - <listitem><para> - For a quick example on how to build an image using - the OpenEmbedded build system, see the - <ulink url='&YOCTO_DOCS_BRIEF_URL;'>Yocto Project Quick Build</ulink> - document. - </para></listitem> - </itemizedlist> - </note> - </para> - - <para> - The build process creates an entire Linux distribution from - source and places it in your - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - under <filename>tmp/deploy/images</filename>. - For detailed information on the build process using BitBake, - see the - "<ulink url='&YOCTO_DOCS_OM_URL;#images-dev-environment'>Images</ulink>" - section in the Yocto Project Overview and Concepts Manual. - </para> - - <para> - The following figure and list overviews the build process: - <imagedata fileref="figures/bitbake-build-flow.png" width="7in" depth="4in" align="center" scalefit="1" /> - <orderedlist> - <listitem><para> - <emphasis>Set up Your Host Development System to Support - Development Using the Yocto Project</emphasis>: - See the - "<link linkend='dev-manual-start'>Setting Up to Use the Yocto Project</link>" - section for options on how to get a build host ready to - use the Yocto Project. - </para></listitem> - <listitem><para> - <emphasis>Initialize the Build Environment:</emphasis> - Initialize the build environment by sourcing the build - environment script (i.e. - <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>): - <literallayout class='monospaced'> - $ source &OE_INIT_FILE; [<replaceable>build_dir</replaceable>] - </literallayout></para> - - <para>When you use the initialization script, the - OpenEmbedded build system uses - <filename>build</filename> as the default Build - Directory in your current work directory. - You can use a <replaceable>build_dir</replaceable> - argument with the script to specify a different build - directory. - <note><title>Tip</title> - A common practice is to use a different Build - Directory for different targets. - For example, <filename>~/build/x86</filename> for a - <filename>qemux86</filename> target, and - <filename>~/build/arm</filename> for a - <filename>qemuarm</filename> target. - </note> - </para></listitem> - <listitem><para> - <emphasis>Make Sure Your <filename>local.conf</filename> - File is Correct:</emphasis> - Ensure the <filename>conf/local.conf</filename> - configuration file, which is found in the Build - Directory, is set up how you want it. - This file defines many aspects of the build environment - including the target machine architecture through the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'>MACHINE</ulink></filename> variable, - the packaging format used during the build - (<ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink>), - and a centralized tarball download directory through the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> variable. - </para></listitem> - <listitem><para> - <emphasis>Build the Image:</emphasis> - Build the image using the <filename>bitbake</filename> - command: - <literallayout class='monospaced'> - $ bitbake <replaceable>target</replaceable> - </literallayout> - <note> - For information on BitBake, see the - <ulink url='&YOCTO_DOCS_BB_URL;'>BitBake User Manual</ulink>. - </note> - The <replaceable>target</replaceable> is the name of the - recipe you want to build. - Common targets are the images in - <filename>meta/recipes-core/images</filename>, - <filename>meta/recipes-sato/images</filename>, and so - forth all found in the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. - Or, the target can be the name of a recipe for a - specific piece of software such as BusyBox. - For more details about the images the OpenEmbedded build - system supports, see the - "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" - chapter in the Yocto Project Reference Manual.</para> - - <para>As an example, the following command builds the - <filename>core-image-minimal</filename> image: - <literallayout class='monospaced'> - $ bitbake core-image-minimal - </literallayout> - Once an image has been built, it often needs to be - installed. - The images and kernels built by the OpenEmbedded - build system are placed in the Build Directory in - <filename class="directory">tmp/deploy/images</filename>. - For information on how to run pre-built images such as - <filename>qemux86</filename> and <filename>qemuarm</filename>, - see the - <ulink url='&YOCTO_DOCS_SDK_URL;'>Yocto Project Application Development and the Extensible Software Development Kit (eSDK)</ulink> - manual. - For information about how to install these images, - see the documentation for your particular board or - machine. - </para></listitem> - </orderedlist> - </para> - </section> - - <section id='dev-building-images-for-multiple-targets-using-multiple-configurations'> - <title>Building Images for Multiple Targets Using Multiple Configurations</title> - - <para> - You can use a single <filename>bitbake</filename> command - to build multiple images or packages for different targets - where each image or package requires a different configuration - (multiple configuration builds). - The builds, in this scenario, are sometimes referred to as - "multiconfigs", and this section uses that term throughout. - </para> - - <para> - This section describes how to set up for multiple - configuration builds and how to account for cross-build - dependencies between the multiconfigs. - </para> - - <section id='dev-setting-up-and-running-a-multiple-configuration-build'> - <title>Setting Up and Running a Multiple Configuration Build</title> - - <para> - To accomplish a multiple configuration build, you must - define each target's configuration separately using - a parallel configuration file in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>, - and you must follow a required file hierarchy. - Additionally, you must enable the multiple configuration - builds in your <filename>local.conf</filename> file. - </para> - - <para> - Follow these steps to set up and execute multiple - configuration builds: - <itemizedlist> - <listitem><para> - <emphasis>Create Separate Configuration Files</emphasis>: - You need to create a single configuration file for - each build target (each multiconfig). - Minimally, each configuration file must define the - machine and the temporary directory BitBake uses - for the build. - Suggested practice dictates that you do not - overlap the temporary directories - used during the builds. - However, it is possible that you can share the - temporary directory - (<ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>). - For example, consider a scenario with two - different multiconfigs for the same - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>: "qemux86" built for - two distributions such as "poky" and "poky-lsb". - In this case, you might want to use the same - <filename>TMPDIR</filename>.</para> - - <para>Here is an example showing the minimal - statements needed in a configuration file for - a "qemux86" target whose temporary build directory - is <filename>tmpmultix86</filename>: - <literallayout class='monospaced'> - MACHINE="qemux86" - TMPDIR="${TOPDIR}/tmpmultix86" - </literallayout></para> - - <para>The location for these multiconfig - configuration files is specific. - They must reside in the current build directory in - a sub-directory of <filename>conf</filename> named - <filename>multiconfig</filename>. - Following is an example that defines two - configuration files for the "x86" and "arm" - multiconfigs: - <imagedata fileref="figures/multiconfig_files.png" align="center" width="4in" depth="3in" /> - </para> - - <para>The reason for this required file hierarchy - is because the <filename>BBPATH</filename> variable - is not constructed until the layers are parsed. - Consequently, using the configuration file as a - pre-configuration file is not possible unless it is - located in the current working directory. - </para></listitem> - <listitem><para> - <emphasis>Add the BitBake Multi-configuration Variable to the Local Configuration File</emphasis>: - Use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBMULTICONFIG'><filename>BBMULTICONFIG</filename></ulink> - variable in your - <filename>conf/local.conf</filename> configuration - file to specify each multiconfig. - Continuing with the example from the previous - figure, the <filename>BBMULTICONFIG</filename> - variable needs to enable two multiconfigs: "x86" - and "arm" by specifying each configuration file: - <literallayout class='monospaced'> - BBMULTICONFIG = "x86 arm" - </literallayout> - <note> - A "default" configuration already exists by - definition. - This configuration is named: "" (i.e. empty - string) and is defined by the variables coming - from your <filename>local.conf</filename> file. - Consequently, the previous example actually - adds two additional configurations to your - build: "arm" and "x86" along with "". - </note> - </para></listitem> - <listitem><para> - <emphasis>Launch BitBake</emphasis>: - Use the following BitBake command form to launch the - multiple configuration build: - <literallayout class='monospaced'> - $ bitbake [mc:<replaceable>multiconfigname</replaceable>:]<replaceable>target</replaceable> [[[mc:<replaceable>multiconfigname</replaceable>:]<replaceable>target</replaceable>] ... ] - </literallayout> - For the example in this section, the following - command applies: - <literallayout class='monospaced'> - $ bitbake mc:x86:core-image-minimal mc:arm:core-image-sato mc::core-image-base - </literallayout> - The previous BitBake command builds a - <filename>core-image-minimal</filename> image that - is configured through the - <filename>x86.conf</filename> configuration file, - a <filename>core-image-sato</filename> - image that is configured through the - <filename>arm.conf</filename> configuration file - and a <filename>core-image-base</filename> that is - configured through your - <filename>local.conf</filename> configuration file. - </para></listitem> - </itemizedlist> - <note> - Support for multiple configuration builds in the - Yocto Project &DISTRO; (&DISTRO_NAME;) Release does - not include Shared State (sstate) optimizations. - Consequently, if a build uses the same object twice - in, for example, two different - <filename>TMPDIR</filename> directories, the build - either loads from an existing sstate cache for that - build at the start or builds the object fresh. - </note> - </para> - </section> - - <section id='dev-enabling-multiple-configuration-build-dependencies'> - <title>Enabling Multiple Configuration Build Dependencies</title> - - <para> - Sometimes dependencies can exist between targets - (multiconfigs) in a multiple configuration build. - For example, suppose that in order to build a - <filename>core-image-sato</filename> image for an "x86" - multiconfig, the root filesystem of an "arm" - multiconfig must exist. - This dependency is essentially that the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-image'><filename>do_image</filename></ulink> - task in the <filename>core-image-sato</filename> recipe - depends on the completion of the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-rootfs'><filename>do_rootfs</filename></ulink> - task of the <filename>core-image-minimal</filename> - recipe. - </para> - - <para> - To enable dependencies in a multiple configuration - build, you must declare the dependencies in the recipe - using the following statement form: - <literallayout class='monospaced'> - <replaceable>task_or_package</replaceable>[mcdepends] = "mc:<replaceable>from_multiconfig</replaceable>:<replaceable>to_multiconfig</replaceable>:<replaceable>recipe_name</replaceable>:<replaceable>task_on_which_to_depend</replaceable>" - </literallayout> - To better show how to use this statement, consider the - example scenario from the first paragraph of this section. - The following statement needs to be added to the recipe - that builds the <filename>core-image-sato</filename> - image: - <literallayout class='monospaced'> - do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_rootfs" - </literallayout> - In this example, the - <replaceable>from_multiconfig</replaceable> is "x86". - The <replaceable>to_multiconfig</replaceable> is "arm". - The task on which the <filename>do_image</filename> task - in the recipe depends is the <filename>do_rootfs</filename> - task from the <filename>core-image-minimal</filename> - recipe associated with the "arm" multiconfig. - </para> - - <para> - Once you set up this dependency, you can build the - "x86" multiconfig using a BitBake command as follows: - <literallayout class='monospaced'> - $ bitbake mc:x86:core-image-sato - </literallayout> - This command executes all the tasks needed to create - the <filename>core-image-sato</filename> image for the - "x86" multiconfig. - Because of the dependency, BitBake also executes through - the <filename>do_rootfs</filename> task for the "arm" - multiconfig build. - </para> - - <para> - Having a recipe depend on the root filesystem of another - build might not seem that useful. - Consider this change to the statement in the - <filename>core-image-sato</filename> recipe: - <literallayout class='monospaced'> - do_image[mcdepends] = "mc:x86:arm:core-image-minimal:do_image" - </literallayout> - In this case, BitBake must create the - <filename>core-image-minimal</filename> image for the - "arm" build since the "x86" build depends on it. - </para> - - <para> - Because "x86" and "arm" are enabled for multiple - configuration builds and have separate configuration - files, BitBake places the artifacts for each build in the - respective temporary build directories (i.e. - <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink>). - </para> - </section> - </section> - - <section id='building-an-initramfs-image'> - <title>Building an Initial RAM Filesystem (initramfs) Image</title> - - <para> - An initial RAM filesystem (initramfs) image provides a temporary - root filesystem used for early system initialization (e.g. - loading of modules needed to locate and mount the "real" root - filesystem). - <note> - The initramfs image is the successor of initial RAM disk - (initrd). - It is a "copy in and out" (cpio) archive of the initial - filesystem that gets loaded into memory during the Linux - startup process. - Because Linux uses the contents of the archive during - initialization, the initramfs image needs to contain all of the - device drivers and tools needed to mount the final root - filesystem. - </note> - </para> - - <para> - Follow these steps to create an initramfs image: - <orderedlist> - <listitem><para> - <emphasis>Create the initramfs Image Recipe:</emphasis> - You can reference the - <filename>core-image-minimal-initramfs.bb</filename> - recipe found in the <filename>meta/recipes-core</filename> - directory of the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - as an example from which to work. - </para></listitem> - <listitem><para> - <emphasis>Decide if You Need to Bundle the initramfs Image - Into the Kernel Image:</emphasis> - If you want the initramfs image that is built to be - bundled in with the kernel image, set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE'><filename>INITRAMFS_IMAGE_BUNDLE</filename></ulink> - variable to "1" in your <filename>local.conf</filename> - configuration file and set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE'><filename>INITRAMFS_IMAGE</filename></ulink> - variable in the recipe that builds the kernel image. - <note><title>Tip</title> - It is recommended that you do bundle the initramfs - image with the kernel image to avoid circular - dependencies between the kernel recipe and the - initramfs recipe should the initramfs image - include kernel modules. - </note> - Setting the <filename>INITRAMFS_IMAGE_BUNDLE</filename> - flag causes the initramfs image to be unpacked - into the <filename>${B}/usr/</filename> directory. - The unpacked initramfs image is then passed to the kernel's - <filename>Makefile</filename> using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-CONFIG_INITRAMFS_SOURCE'><filename>CONFIG_INITRAMFS_SOURCE</filename></ulink> - variable, allowing the initramfs image to be built into - the kernel normally. - <note> - If you choose to not bundle the initramfs image with - the kernel image, you are essentially using an - <ulink url='https://en.wikipedia.org/wiki/Initrd'>Initial RAM Disk (initrd)</ulink>. - Creating an initrd is handled primarily through the - <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRD_IMAGE'><filename>INITRD_IMAGE</filename></ulink>, - <filename>INITRD_LIVE</filename>, and - <filename>INITRD_IMAGE_LIVE</filename> variables. - For more information, see the - <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/meta/classes/image-live.bbclass'><filename>image-live.bbclass</filename></ulink> - file. - </note> - </para></listitem> - <listitem><para> - <emphasis>Optionally Add Items to the initramfs Image - Through the initramfs Image Recipe:</emphasis> - If you add items to the initramfs image by way of its - recipe, you should use - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_INSTALL'><filename>PACKAGE_INSTALL</filename></ulink> - rather than - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>. - <filename>PACKAGE_INSTALL</filename> gives more direct - control of what is added to the image as compared to - the defaults you might not necessarily want that are - set by the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-image'><filename>image</filename></ulink> - or - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-core-image'><filename>core-image</filename></ulink> - classes. - </para></listitem> - <listitem><para> - <emphasis>Build the Kernel Image and the initramfs - Image:</emphasis> - Build your kernel image using BitBake. - Because the initramfs image recipe is a dependency of the - kernel image, the initramfs image is built as well and - bundled with the kernel image if you used the - <ulink url='&YOCTO_DOCS_REF_URL;#var-INITRAMFS_IMAGE_BUNDLE'><filename>INITRAMFS_IMAGE_BUNDLE</filename></ulink> - variable described earlier. - </para></listitem> - </orderedlist> - </para> - </section> - - <section id='building-a-tiny-system'> - <title>Building a Tiny System</title> - - <para> - Very small distributions have some significant advantages such - as requiring less on-die or in-package memory (cheaper), better - performance through efficient cache usage, lower power requirements - due to less memory, faster boot times, and reduced development - overhead. - Some real-world examples where a very small distribution gives - you distinct advantages are digital cameras, medical devices, - and small headless systems. - </para> - - <para> - This section presents information that shows you how you can - trim your distribution to even smaller sizes than the - <filename>poky-tiny</filename> distribution, which is around - 5 Mbytes, that can be built out-of-the-box using the Yocto Project. - </para> - - <section id='tiny-system-overview'> - <title>Overview</title> - - <para> - The following list presents the overall steps you need to - consider and perform to create distributions with smaller - root filesystems, achieve faster boot times, maintain your critical - functionality, and avoid initial RAM disks: - <itemizedlist> - <listitem><para> - <link linkend='goals-and-guiding-principles'>Determine your goals and guiding principles.</link> - </para></listitem> - <listitem><para> - <link linkend='understand-what-gives-your-image-size'>Understand what contributes to your image size.</link> - </para></listitem> - <listitem><para> - <link linkend='trim-the-root-filesystem'>Reduce the size of the root filesystem.</link> - </para></listitem> - <listitem><para> - <link linkend='trim-the-kernel'>Reduce the size of the kernel.</link> - </para></listitem> - <listitem><para> - <link linkend='remove-package-management-requirements'>Eliminate packaging requirements.</link> - </para></listitem> - <listitem><para> - <link linkend='look-for-other-ways-to-minimize-size'>Look for other ways to minimize size.</link> - </para></listitem> - <listitem><para> - <link linkend='iterate-on-the-process'>Iterate on the process.</link> - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='goals-and-guiding-principles'> - <title>Goals and Guiding Principles</title> - - <para> - Before you can reach your destination, you need to know - where you are going. - Here is an example list that you can use as a guide when - creating very small distributions: - <itemizedlist> - <listitem><para>Determine how much space you need - (e.g. a kernel that is 1 Mbyte or less and - a root filesystem that is 3 Mbytes or less). - </para></listitem> - <listitem><para>Find the areas that are currently - taking 90% of the space and concentrate on reducing - those areas. - </para></listitem> - <listitem><para>Do not create any difficult "hacks" - to achieve your goals.</para></listitem> - <listitem><para>Leverage the device-specific - options.</para></listitem> - <listitem><para>Work in a separate layer so that you - keep changes isolated. - For information on how to create layers, see - the "<link linkend='understanding-and-creating-layers'>Understanding and Creating Layers</link>" section. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='understand-what-gives-your-image-size'> - <title>Understand What Contributes to Your Image Size</title> - - <para> - It is easiest to have something to start with when creating - your own distribution. - You can use the Yocto Project out-of-the-box to create the - <filename>poky-tiny</filename> distribution. - Ultimately, you will want to make changes in your own - distribution that are likely modeled after - <filename>poky-tiny</filename>. - <note> - To use <filename>poky-tiny</filename> in your build, - set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> - variable in your - <filename>local.conf</filename> file to "poky-tiny" - as described in the - "<link linkend='creating-your-own-distribution'>Creating Your Own Distribution</link>" - section. - </note> - </para> - - <para> - Understanding some memory concepts will help you reduce the - system size. - Memory consists of static, dynamic, and temporary memory. - Static memory is the TEXT (code), DATA (initialized data - in the code), and BSS (uninitialized data) sections. - Dynamic memory represents memory that is allocated at runtime: - stacks, hash tables, and so forth. - Temporary memory is recovered after the boot process. - This memory consists of memory used for decompressing - the kernel and for the <filename>__init__</filename> - functions. - </para> - - <para> - To help you see where you currently are with kernel and root - filesystem sizes, you can use two tools found in the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> in - the <filename>scripts/tiny/</filename> directory: - <itemizedlist> - <listitem><para><filename>ksize.py</filename>: Reports - component sizes for the kernel build objects. - </para></listitem> - <listitem><para><filename>dirsize.py</filename>: Reports - component sizes for the root filesystem.</para></listitem> - </itemizedlist> - This next tool and command help you organize configuration - fragments and view file dependencies in a human-readable form: - <itemizedlist> - <listitem><para><filename>merge_config.sh</filename>: - Helps you manage configuration files and fragments - within the kernel. - With this tool, you can merge individual configuration - fragments together. - The tool allows you to make overrides and warns you - of any missing configuration options. - The tool is ideal for allowing you to iterate on - configurations, create minimal configurations, and - create configuration files for different machines - without having to duplicate your process.</para> - <para>The <filename>merge_config.sh</filename> script is - part of the Linux Yocto kernel Git repositories - (i.e. <filename>linux-yocto-3.14</filename>, - <filename>linux-yocto-3.10</filename>, - <filename>linux-yocto-3.8</filename>, and so forth) - in the - <filename>scripts/kconfig</filename> directory.</para> - <para>For more information on configuration fragments, - see the - "<ulink url='&YOCTO_DOCS_KERNEL_DEV_URL;#creating-config-fragments'>Creating Configuration Fragments</ulink>" - section in the Yocto Project Linux Kernel Development - Manual. - </para></listitem> - <listitem><para><filename>bitbake -u taskexp -g <replaceable>bitbake_target</replaceable></filename>: - Using the BitBake command with these options brings up - a Dependency Explorer from which you can view file - dependencies. - Understanding these dependencies allows you to make - informed decisions when cutting out various pieces of the - kernel and root filesystem.</para></listitem> - </itemizedlist> - </para> - </section> - - <section id='trim-the-root-filesystem'> - <title>Trim the Root Filesystem</title> - - <para> - The root filesystem is made up of packages for booting, - libraries, and applications. - To change things, you can configure how the packaging happens, - which changes the way you build them. - You can also modify the filesystem itself or select a different - filesystem. - </para> - - <para> - First, find out what is hogging your root filesystem by running the - <filename>dirsize.py</filename> script from your root directory: - <literallayout class='monospaced'> - $ cd <replaceable>root-directory-of-image</replaceable> - $ dirsize.py 100000 > dirsize-100k.log - $ cat dirsize-100k.log - </literallayout> - You can apply a filter to the script to ignore files under - a certain size. - The previous example filters out any files below 100 Kbytes. - The sizes reported by the tool are uncompressed, and thus - will be smaller by a relatively constant factor in a - compressed root filesystem. - When you examine your log file, you can focus on areas of the - root filesystem that take up large amounts of memory. - </para> - - <para> - You need to be sure that what you eliminate does not cripple - the functionality you need. - One way to see how packages relate to each other is by using - the Dependency Explorer UI with the BitBake command: - <literallayout class='monospaced'> - $ cd <replaceable>image-directory</replaceable> - $ bitbake -u taskexp -g <replaceable>image</replaceable> - </literallayout> - Use the interface to select potential packages you wish to - eliminate and see their dependency relationships. - </para> - - <para> - When deciding how to reduce the size, get rid of packages that - result in minimal impact on the feature set. - For example, you might not need a VGA display. - Or, you might be able to get by with <filename>devtmpfs</filename> - and <filename>mdev</filename> instead of - <filename>udev</filename>. - </para> - - <para> - Use your <filename>local.conf</filename> file to make changes. - For example, to eliminate <filename>udev</filename> and - <filename>glib</filename>, set the following in the - local configuration file: - <literallayout class='monospaced'> - VIRTUAL-RUNTIME_dev_manager = "" - </literallayout> - </para> - - <para> - Finally, you should consider exactly the type of root - filesystem you need to meet your needs while also reducing - its size. - For example, consider <filename>cramfs</filename>, - <filename>squashfs</filename>, <filename>ubifs</filename>, - <filename>ext2</filename>, or an <filename>initramfs</filename> - using <filename>initramfs</filename>. - Be aware that <filename>ext3</filename> requires a 1 Mbyte - journal. - If you are okay with running read-only, you do not need this - journal. - </para> - - <note> - After each round of elimination, you need to rebuild your - system and then use the tools to see the effects of your - reductions. - </note> - </section> - - <section id='trim-the-kernel'> - <title>Trim the Kernel</title> - - <para> - The kernel is built by including policies for hardware-independent - aspects. - What subsystems do you enable? - For what architecture are you building? - Which drivers do you build by default? - <note>You can modify the kernel source if you want to help - with boot time. - </note> - </para> - - <para> - Run the <filename>ksize.py</filename> script from the top-level - Linux build directory to get an idea of what is making up - the kernel: - <literallayout class='monospaced'> - $ cd <replaceable>top-level-linux-build-directory</replaceable> - $ ksize.py > ksize.log - $ cat ksize.log - </literallayout> - When you examine the log, you will see how much space is - taken up with the built-in <filename>.o</filename> files for - drivers, networking, core kernel files, filesystem, sound, - and so forth. - The sizes reported by the tool are uncompressed, and thus - will be smaller by a relatively constant factor in a compressed - kernel image. - Look to reduce the areas that are large and taking up around - the "90% rule." - </para> - - <para> - To examine, or drill down, into any particular area, use the - <filename>-d</filename> option with the script: - <literallayout class='monospaced'> - $ ksize.py -d > ksize.log - </literallayout> - Using this option breaks out the individual file information - for each area of the kernel (e.g. drivers, networking, and - so forth). - </para> - - <para> - Use your log file to see what you can eliminate from the kernel - based on features you can let go. - For example, if you are not going to need sound, you do not - need any drivers that support sound. - </para> - - <para> - After figuring out what to eliminate, you need to reconfigure - the kernel to reflect those changes during the next build. - You could run <filename>menuconfig</filename> and make all your - changes at once. - However, that makes it difficult to see the effects of your - individual eliminations and also makes it difficult to replicate - the changes for perhaps another target device. - A better method is to start with no configurations using - <filename>allnoconfig</filename>, create configuration - fragments for individual changes, and then manage the - fragments into a single configuration file using - <filename>merge_config.sh</filename>. - The tool makes it easy for you to iterate using the - configuration change and build cycle. - </para> - - <para> - Each time you make configuration changes, you need to rebuild - the kernel and check to see what impact your changes had on - the overall size. - </para> - </section> - - <section id='remove-package-management-requirements'> - <title>Remove Package Management Requirements</title> - - <para> - Packaging requirements add size to the image. - One way to reduce the size of the image is to remove all the - packaging requirements from the image. - This reduction includes both removing the package manager - and its unique dependencies as well as removing the package - management data itself. - </para> - - <para> - To eliminate all the packaging requirements for an image, - be sure that "package-management" is not part of your - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> - statement for the image. - When you remove this feature, you are removing the package - manager as well as its dependencies from the root filesystem. - </para> - </section> - - <section id='look-for-other-ways-to-minimize-size'> - <title>Look for Other Ways to Minimize Size</title> - - <para> - Depending on your particular circumstances, other areas that you - can trim likely exist. - The key to finding these areas is through tools and methods - described here combined with experimentation and iteration. - Here are a couple of areas to experiment with: - <itemizedlist> - <listitem><para><filename>glibc</filename>: - In general, follow this process: - <orderedlist> - <listitem><para>Remove <filename>glibc</filename> - features from - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> - that you think you do not need.</para></listitem> - <listitem><para>Build your distribution. - </para></listitem> - <listitem><para>If the build fails due to missing - symbols in a package, determine if you can - reconfigure the package to not need those - features. - For example, change the configuration to not - support wide character support as is done for - <filename>ncurses</filename>. - Or, if support for those characters is needed, - determine what <filename>glibc</filename> - features provide the support and restore the - configuration. - </para></listitem> - <listitem><para>Rebuild and repeat the process. - </para></listitem> - </orderedlist></para></listitem> - <listitem><para><filename>busybox</filename>: - For BusyBox, use a process similar as described for - <filename>glibc</filename>. - A difference is you will need to boot the resulting - system to see if you are able to do everything you - expect from the running system. - You need to be sure to integrate configuration fragments - into Busybox because BusyBox handles its own core - features and then allows you to add configuration - fragments on top. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='iterate-on-the-process'> - <title>Iterate on the Process</title> - - <para> - If you have not reached your goals on system size, you need - to iterate on the process. - The process is the same. - Use the tools and see just what is taking up 90% of the root - filesystem and the kernel. - Decide what you can eliminate without limiting your device - beyond what you need. - </para> - - <para> - Depending on your system, a good place to look might be - Busybox, which provides a stripped down - version of Unix tools in a single, executable file. - You might be able to drop virtual terminal services or perhaps - ipv6. - </para> - </section> - </section> - - <section id='building-images-for-more-than-one-machine'> - <title>Building Images for More than One Machine</title> - - <para> - A common scenario developers face is creating images for several - different machines that use the same software environment. - In this situation, it is tempting to set the - tunings and optimization flags for each build specifically for - the targeted hardware (i.e. "maxing out" the tunings). - Doing so can considerably add to build times and package feed - maintenance collectively for the machines. - For example, selecting tunes that are extremely specific to a - CPU core used in a system might enable some micro optimizations - in GCC for that particular system but would otherwise not gain - you much of a performance difference across the other systems - as compared to using a more general tuning across all the builds - (e.g. setting - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEFAULTTUNE'><filename>DEFAULTTUNE</filename></ulink> - specifically for each machine's build). - Rather than "max out" each build's tunings, you can take steps that - cause the OpenEmbedded build system to reuse software across the - various machines where it makes sense. - </para> - - <para> - If build speed and package feed maintenance are considerations, - you should consider the points in this section that can help you - optimize your tunings to best consider build times and package - feed maintenance. - <itemizedlist> - <listitem><para> - <emphasis>Share the Build Directory:</emphasis> - If at all possible, share the - <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink> - across builds. - The Yocto Project supports switching between different - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> - values in the same <filename>TMPDIR</filename>. - This practice is well supported and regularly used by - developers when building for multiple machines. - When you use the same <filename>TMPDIR</filename> for - multiple machine builds, the OpenEmbedded build system can - reuse the existing native and often cross-recipes for - multiple machines. - Thus, build time decreases. - <note> - If - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> - settings change or fundamental configuration settings - such as the filesystem layout, you need to work with - a clean <filename>TMPDIR</filename>. - Sharing <filename>TMPDIR</filename> under these - circumstances might work but since it is not - guaranteed, you should use a clean - <filename>TMPDIR</filename>. - </note> - </para></listitem> - <listitem><para> - <emphasis>Enable the Appropriate Package Architecture:</emphasis> - By default, the OpenEmbedded build system enables three - levels of package architectures: "all", "tune" or "package", - and "machine". - Any given recipe usually selects one of these package - architectures (types) for its output. - Depending for what a given recipe creates packages, making - sure you enable the appropriate package architecture can - directly impact the build time.</para> - - <para>A recipe that just generates scripts can enable - "all" architecture because there are no binaries to build. - To specifically enable "all" architecture, be sure your - recipe inherits the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-allarch'><filename>allarch</filename></ulink> - class. - This class is useful for "all" architectures because it - configures many variables so packages can be used across - multiple architectures.</para> - - <para>If your recipe needs to generate packages that are - machine-specific or when one of the build or runtime - dependencies is already machine-architecture dependent, - which makes your recipe also machine-architecture dependent, - make sure your recipe enables the "machine" package - architecture through the - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></ulink> - variable: - <literallayout class='monospaced'> - PACKAGE_ARCH = "${MACHINE_ARCH}" - </literallayout> - When you do not specifically enable a package - architecture through the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink>, - The OpenEmbedded build system defaults to the - <ulink url='&YOCTO_DOCS_REF_URL;#var-TUNE_PKGARCH'><filename>TUNE_PKGARCH</filename></ulink> - setting: - <literallayout class='monospaced'> - PACKAGE_ARCH = "${TUNE_PKGARCH}" - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Choose a Generic Tuning File if Possible:</emphasis> - Some tunes are more generic and can run on multiple targets - (e.g. an <filename>armv5</filename> set of packages could - run on <filename>armv6</filename> and - <filename>armv7</filename> processors in most cases). - Similarly, <filename>i486</filename> binaries could work - on <filename>i586</filename> and higher processors. - You should realize, however, that advances on newer - processor versions would not be used.</para> - - <para>If you select the same tune for several different - machines, the OpenEmbedded build system reuses software - previously built, thus speeding up the overall build time. - Realize that even though a new sysroot for each machine is - generated, the software is not recompiled and only one - package feed exists. - </para></listitem> - <listitem><para> - <emphasis>Manage Granular Level Packaging:</emphasis> - Sometimes cases exist where injecting another level of - package architecture beyond the three higher levels noted - earlier can be useful. - For example, consider how NXP (formerly Freescale) allows - for the easy reuse of binary packages in their layer - <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/'><filename>meta-freescale</filename></ulink>. - In this example, the - <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/meta-freescale/tree/classes/fsl-dynamic-packagearch.bbclass'><filename>fsl-dynamic-packagearch</filename></ulink> - class shares GPU packages for i.MX53 boards because - all boards share the AMD GPU. - The i.MX6-based boards can do the same because all boards - share the Vivante GPU. - This class inspects the BitBake datastore to identify if - the package provides or depends on one of the - sub-architecture values. - If so, the class sets the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink> - value based on the <filename>MACHINE_SUBARCH</filename> - value. - If the package does not provide or depend on one of the - sub-architecture values but it matches a value in the - machine-specific filter, it sets - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_ARCH'><filename>MACHINE_ARCH</filename></ulink>. - This behavior reduces the number of packages built and - saves build time by reusing binaries. - </para></listitem> - <listitem><para> - <emphasis>Use Tools to Debug Issues:</emphasis> - Sometimes you can run into situations where software is - being rebuilt when you think it should not be. - For example, the OpenEmbedded build system might not be - using shared state between machines when you think it - should be. - These types of situations are usually due to references - to machine-specific variables such as - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-SERIAL_CONSOLES'><filename>SERIAL_CONSOLES</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-XSERVER'><filename>XSERVER</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES'><filename>MACHINE_FEATURES</filename></ulink>, - and so forth in code that is supposed to only be - tune-specific or when the recipe depends - (<ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-RSUGGESTS'><filename>RSUGGESTS</filename></ulink>, - and so forth) on some other recipe that already has - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ARCH'><filename>PACKAGE_ARCH</filename></ulink> - defined as "${MACHINE_ARCH}". - <note> - Patches to fix any issues identified are most welcome - as these issues occasionally do occur. - </note></para> - - <para>For such cases, you can use some tools to help you - sort out the situation: - <itemizedlist> - <listitem><para> - <emphasis><filename>sstate-diff-machines.sh</filename>:</emphasis> - You can find this tool in the - <filename>scripts</filename> directory of the - Source Repositories. - See the comments in the script for information on - how to use the tool. - </para></listitem> - <listitem><para> - <emphasis>BitBake's "-S printdiff" Option:</emphasis> - Using this option causes BitBake to try to - establish the closest signature match it can - (e.g. in the shared state cache) and then run - <filename>bitbake-diffsigs</filename> over the - matches to determine the stamps and delta where - these two stamp trees diverge. - </para></listitem> - </itemizedlist> - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id="building-software-from-an-external-source"> - <title>Building Software from an External Source</title> - - <para> - By default, the OpenEmbedded build system uses the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - when building source code. - The build process involves fetching the source files, unpacking - them, and then patching them if necessary before the build takes - place. - </para> - - <para> - Situations exist where you might want to build software from source - files that are external to and thus outside of the - OpenEmbedded build system. - For example, suppose you have a project that includes a new BSP with - a heavily customized kernel. - And, you want to minimize exposing the build system to the - development team so that they can focus on their project and - maintain everyone's workflow as much as possible. - In this case, you want a kernel source directory on the development - machine where the development occurs. - You want the recipe's - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - variable to point to the external directory and use it as is, not - copy it. - </para> - - <para> - To build from software that comes from an external source, all you - need to do is inherit the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink> - class and then set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink> - variable to point to your external source code. - Here are the statements to put in your - <filename>local.conf</filename> file: - <literallayout class='monospaced'> - INHERIT += "externalsrc" - EXTERNALSRC_pn-<replaceable>myrecipe</replaceable> = "<replaceable>path-to-your-source-tree</replaceable>" - </literallayout> - </para> - - <para> - This next example shows how to accomplish the same thing by setting - <filename>EXTERNALSRC</filename> in the recipe itself or in the - recipe's append file: - <literallayout class='monospaced'> - EXTERNALSRC = "<replaceable>path</replaceable>" - EXTERNALSRC_BUILD = "<replaceable>path</replaceable>" - </literallayout> - <note> - In order for these settings to take effect, you must globally - or locally inherit the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-externalsrc'><filename>externalsrc</filename></ulink> - class. - </note> - </para> - - <para> - By default, <filename>externalsrc.bbclass</filename> builds - the source code in a directory separate from the external source - directory as specified by - <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC'><filename>EXTERNALSRC</filename></ulink>. - If you need to have the source built in the same directory in - which it resides, or some other nominated directory, you can set - <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTERNALSRC_BUILD'><filename>EXTERNALSRC_BUILD</filename></ulink> - to point to that directory: - <literallayout class='monospaced'> - EXTERNALSRC_BUILD_pn-<replaceable>myrecipe</replaceable> = "<replaceable>path-to-your-source-tree</replaceable>" - </literallayout> - </para> - </section> - - <section id="replicating-a-build-offline"> - <title>Replicating a Build Offline</title> - - <para> - It can be useful to take a "snapshot" of upstream sources - used in a build and then use that "snapshot" later to - replicate the build offline. - To do so, you need to first prepare and populate your downloads - directory your "snapshot" of files. - Once your downloads directory is ready, you can use it at - any time and from any machine to replicate your build. - </para> - - <para> - Follow these steps to populate your Downloads directory: - <orderedlist> - <listitem><para> - <emphasis>Create a Clean Downloads Directory:</emphasis> - Start with an empty downloads directory - (<ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>). - You start with an empty downloads directory by either - removing the files in the existing directory or by - setting - <filename>DL_DIR</filename> to point to either an - empty location or one that does not yet exist. - </para></listitem> - <listitem><para> - <emphasis>Generate Tarballs of the Source Git Repositories:</emphasis> - Edit your <filename>local.conf</filename> configuration - file as follows: - <literallayout class='monospaced'> - DL_DIR = "/home/<replaceable>your-download-dir</replaceable>/" - BB_GENERATE_MIRROR_TARBALLS = "1" - </literallayout> - During the fetch process in the next step, BitBake - gathers the source files and creates tarballs in - the directory pointed to by <filename>DL_DIR</filename>. - See the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink> - variable for more information. - </para></listitem> - <listitem><para> - <emphasis>Populate Your Downloads Directory Without Building:</emphasis> - Use BitBake to fetch your sources but inhibit the - build: - <literallayout class='monospaced'> - $ bitbake <replaceable>target</replaceable> --runonly=fetch - </literallayout> - The downloads directory (i.e. - <filename>${DL_DIR}</filename>) now has a "snapshot" of - the source files in the form of tarballs, which can - be used for the build. - </para></listitem> - <listitem><para> - <emphasis>Optionally Remove Any Git or other SCM Subdirectories From the Downloads Directory:</emphasis> - If you want, you can clean up your downloads directory - by removing any Git or other Source Control Management - (SCM) subdirectories such as - <filename>${DL_DIR}/git2/*</filename>. - The tarballs already contain these subdirectories. - </para></listitem> - </orderedlist> - </para> - - <para> - Once your downloads directory has everything it needs regarding - source files, you can create your "own-mirror" and build - your target. - Understand that you can use the files to build the target - offline from any machine and at any time. - </para> - - <para> - Follow these steps to build your target using the files in the - downloads directory: - <orderedlist> - <listitem><para> - <emphasis>Using Local Files Only:</emphasis> - Inside your <filename>local.conf</filename> file, add - the - <ulink url='&YOCTO_DOCS_REF_URL;#var-SOURCE_MIRROR_URL'><filename>SOURCE_MIRROR_URL</filename></ulink> - variable, - inherit the <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-own-mirrors'><filename>own-mirrors</filename></ulink> - class, and use the - <ulink url='&YOCTO_DOCS_BB_URL;#var-bb-BB_NO_NETWORK'><filename>BB_NO_NETWORK</filename></ulink> - variable to your <filename>local.conf</filename>. - <literallayout class='monospaced'> - SOURCE_MIRROR_URL ?= "file:///home/<replaceable>your-download-dir</replaceable>/" - INHERIT += "own-mirrors" - BB_NO_NETWORK = "1" - </literallayout> - The <filename>SOURCE_MIRROR_URL</filename> and - <filename>own-mirror</filename> class set up the system - to use the downloads directory as your "own mirror". - Using the <filename>BB_NO_NETWORK</filename> - variable makes sure that BitBake's fetching process - in step 3 stays local, which means files from - your "own-mirror" are used. - </para></listitem> - <listitem><para> - <emphasis>Start With a Clean Build:</emphasis> - You can start with a clean build by removing the - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink><filename>}</filename> - directory or using a new - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - </para></listitem> - <listitem><para> - <emphasis>Build Your Target:</emphasis> - Use BitBake to build your target: - <literallayout class='monospaced'> - $ bitbake <replaceable>target</replaceable> - </literallayout> - The build completes using the known local "snapshot" of - source files from your mirror. - The resulting tarballs for your "snapshot" of source - files are in the downloads directory. - <note> - <para>The offline build does not work if recipes - attempt to find the latest version of software - by setting - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink> - to - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-AUTOREV'><filename>AUTOREV</filename></ulink><filename>}</filename>: - <literallayout class='monospaced'> - SRCREV = "${AUTOREV}" - </literallayout> - When a recipe sets - <filename>SRCREV</filename> to - <filename>${AUTOREV}</filename>, the build system - accesses the network in an attempt to determine the - latest version of software from the SCM. - Typically, recipes that use - <filename>AUTOREV</filename> are custom or - modified recipes. - Recipes that reside in public repositories - usually do not use <filename>AUTOREV</filename>. - </para> - - <para>If you do have recipes that use - <filename>AUTOREV</filename>, you can take steps to - still use the recipes in an offline build. - Do the following: - <orderedlist> - <listitem><para> - Use a configuration generated by - enabling - <link linkend='maintaining-build-output-quality'>build history</link>. - </para></listitem> - <listitem><para> - Use the - <filename>buildhistory-collect-srcrevs</filename> - command to collect the stored - <filename>SRCREV</filename> values from - the build's history. - For more information on collecting these - values, see the - "<link linkend='build-history-package-information'>Build History Package Information</link>" - section. - </para></listitem> - <listitem><para> - Once you have the correct source - revisions, you can modify those recipes - to to set <filename>SRCREV</filename> - to specific versions of the software. - </para></listitem> - </orderedlist> - </para> - </note> - </para></listitem> - </orderedlist> - </para> - </section> - </section> - - <section id='speeding-up-a-build'> - <title>Speeding Up a Build</title> - - <para> - Build time can be an issue. - By default, the build system uses simple controls to try and maximize - build efficiency. - In general, the default settings for all the following variables - result in the most efficient build times when dealing with single - socket systems (i.e. a single CPU). - If you have multiple CPUs, you might try increasing the default - values to gain more speed. - See the descriptions in the glossary for each variable for more - information: - <itemizedlist> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_NUMBER_THREADS'><filename>BB_NUMBER_THREADS</filename>:</ulink> - The maximum number of threads BitBake simultaneously executes. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_NUMBER_PARSE_THREADS'><filename>BB_NUMBER_PARSE_THREADS</filename>:</ulink> - The number of threads BitBake uses during parsing. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename>:</ulink> - Extra options passed to the <filename>make</filename> command - during the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink> - task in order to specify parallel compilation on the - local build host. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename>:</ulink> - Extra options passed to the <filename>make</filename> command - during the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-install'><filename>do_install</filename></ulink> - task in order to specify parallel installation on the - local build host. - </para></listitem> - </itemizedlist> - As mentioned, these variables all scale to the number of processor - cores available on the build system. - For single socket systems, this auto-scaling ensures that the build - system fundamentally takes advantage of potential parallel operations - during the build based on the build machine's capabilities. - </para> - - <para> - Following are additional factors that can affect build speed: - <itemizedlist> - <listitem><para> - File system type: - The file system type that the build is being performed on can - also influence performance. - Using <filename>ext4</filename> is recommended as compared - to <filename>ext2</filename> and <filename>ext3</filename> - due to <filename>ext4</filename> improved features - such as extents. - </para></listitem> - <listitem><para> - Disabling the updating of access time using - <filename>noatime</filename>: - The <filename>noatime</filename> mount option prevents the - build system from updating file and directory access times. - </para></listitem> - <listitem><para> - Setting a longer commit: - Using the "commit=" mount option increases the interval - in seconds between disk cache writes. - Changing this interval from the five second default to - something longer increases the risk of data loss but decreases - the need to write to the disk, thus increasing the build - performance. - </para></listitem> - <listitem><para> - Choosing the packaging backend: - Of the available packaging backends, IPK is the fastest. - Additionally, selecting a singular packaging backend also - helps. - </para></listitem> - <listitem><para> - Using <filename>tmpfs</filename> for - <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink> - as a temporary file system: - While this can help speed up the build, the benefits are - limited due to the compiler using - <filename>-pipe</filename>. - The build system goes to some lengths to avoid - <filename>sync()</filename> calls into the - file system on the principle that if there was a significant - failure, the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - contents could easily be rebuilt. - </para></listitem> - <listitem><para> - Inheriting the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-rm-work'><filename>rm_work</filename></ulink> - class: - Inheriting this class has shown to speed up builds due to - significantly lower amounts of data stored in the data - cache as well as on disk. - Inheriting this class also makes cleanup of - <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink> - faster, at the expense of being easily able to dive into the - source code. - File system maintainers have recommended that the fastest way - to clean up large numbers of files is to reformat partitions - rather than delete files due to the linear nature of - partitions. - This, of course, assumes you structure the disk partitions and - file systems in a way that this is practical. - </para></listitem> - </itemizedlist> - Aside from the previous list, you should keep some trade offs in - mind that can help you speed up the build: - <itemizedlist> - <listitem><para> - Remove items from - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> - that you might not need. - </para></listitem> - <listitem><para> - Exclude debug symbols and other debug information: - If you do not need these symbols and other debug information, - disabling the <filename>*-dbg</filename> package generation - can speed up the build. - You can disable this generation by setting the - <ulink url='&YOCTO_DOCS_REF_URL;#var-INHIBIT_PACKAGE_DEBUG_SPLIT'><filename>INHIBIT_PACKAGE_DEBUG_SPLIT</filename></ulink> - variable to "1". - </para></listitem> - <listitem><para> - Disable static library generation for recipes derived from - <filename>autoconf</filename> or <filename>libtool</filename>: - Following is an example showing how to disable static - libraries and still provide an override to handle exceptions: - <literallayout class='monospaced'> - STATICLIBCONF = "--disable-static" - STATICLIBCONF_sqlite3-native = "" - EXTRA_OECONF += "${STATICLIBCONF}" - </literallayout> - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - Some recipes need static libraries in order to work - correctly (e.g. <filename>pseudo-native</filename> - needs <filename>sqlite3-native</filename>). - Overrides, as in the previous example, account for - these kinds of exceptions. - </para></listitem> - <listitem><para> - Some packages have packaging code that assumes the - presence of the static libraries. - If so, you might need to exclude them as well. - </para></listitem> - </itemizedlist> - </note> - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id="platdev-working-with-libraries"> - <title>Working With Libraries</title> - - <para> - Libraries are an integral part of your system. - This section describes some common practices you might find - helpful when working with libraries to build your system: - <itemizedlist> - <listitem><para><link linkend='including-static-library-files'>How to include static library files</link> - </para></listitem> - <listitem><para><link linkend='combining-multiple-versions-library-files-into-one-image'>How to use the Multilib feature to combine multiple versions of library files into a single image</link> - </para></listitem> - <listitem><para><link linkend='installing-multiple-versions-of-the-same-library'>How to install multiple versions of the same library in parallel on the same system</link> - </para></listitem> - </itemizedlist> - </para> - - <section id='including-static-library-files'> - <title>Including Static Library Files</title> - - <para> - If you are building a library and the library offers static linking, you can control - which static library files (<filename>*.a</filename> files) get included in the - built library. - </para> - - <para> - The <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink> - and <ulink url='&YOCTO_DOCS_REF_URL;#var-FILES'><filename>FILES_*</filename></ulink> - variables in the - <filename>meta/conf/bitbake.conf</filename> configuration file define how files installed - by the <filename>do_install</filename> task are packaged. - By default, the <filename>PACKAGES</filename> variable includes - <filename>${PN}-staticdev</filename>, which represents all static library files. - <note> - Some previously released versions of the Yocto Project - defined the static library files through - <filename>${PN}-dev</filename>. - </note> - Following is part of the BitBake configuration file, where - you can see how the static library files are defined: - <literallayout class='monospaced'> - PACKAGE_BEFORE_PN ?= "" - PACKAGES = "${PN}-dbg ${PN}-staticdev ${PN}-dev ${PN}-doc ${PN}-locale ${PACKAGE_BEFORE_PN} ${PN}" - PACKAGES_DYNAMIC = "^${PN}-locale-.*" - FILES = "" - - FILES_${PN} = "${bindir}/* ${sbindir}/* ${libexecdir}/* ${libdir}/lib*${SOLIBS} \ - ${sysconfdir} ${sharedstatedir} ${localstatedir} \ - ${base_bindir}/* ${base_sbindir}/* \ - ${base_libdir}/*${SOLIBS} \ - ${base_prefix}/lib/udev/rules.d ${prefix}/lib/udev/rules.d \ - ${datadir}/${BPN} ${libdir}/${BPN}/* \ - ${datadir}/pixmaps ${datadir}/applications \ - ${datadir}/idl ${datadir}/omf ${datadir}/sounds \ - ${libdir}/bonobo/servers" - - FILES_${PN}-bin = "${bindir}/* ${sbindir}/*" - - FILES_${PN}-doc = "${docdir} ${mandir} ${infodir} ${datadir}/gtk-doc \ - ${datadir}/gnome/help" - SECTION_${PN}-doc = "doc" - - FILES_SOLIBSDEV ?= "${base_libdir}/lib*${SOLIBSDEV} ${libdir}/lib*${SOLIBSDEV}" - FILES_${PN}-dev = "${includedir} ${FILES_SOLIBSDEV} ${libdir}/*.la \ - ${libdir}/*.o ${libdir}/pkgconfig ${datadir}/pkgconfig \ - ${datadir}/aclocal ${base_libdir}/*.o \ - ${libdir}/${BPN}/*.la ${base_libdir}/*.la" - SECTION_${PN}-dev = "devel" - ALLOW_EMPTY_${PN}-dev = "1" - RDEPENDS_${PN}-dev = "${PN} (= ${EXTENDPKGV})" - - FILES_${PN}-staticdev = "${libdir}/*.a ${base_libdir}/*.a ${libdir}/${BPN}/*.a" - SECTION_${PN}-staticdev = "devel" - RDEPENDS_${PN}-staticdev = "${PN}-dev (= ${EXTENDPKGV})" - </literallayout> - </para> - </section> - - <section id="combining-multiple-versions-library-files-into-one-image"> - <title>Combining Multiple Versions of Library Files into One Image</title> - - <para> - The build system offers the ability to build libraries with different - target optimizations or architecture formats and combine these together - into one system image. - You can link different binaries in the image - against the different libraries as needed for specific use cases. - This feature is called "Multilib." - </para> - - <para> - An example would be where you have most of a system compiled in 32-bit - mode using 32-bit libraries, but you have something large, like a database - engine, that needs to be a 64-bit application and uses 64-bit libraries. - Multilib allows you to get the best of both 32-bit and 64-bit libraries. - </para> - - <para> - While the Multilib feature is most commonly used for 32 and 64-bit differences, - the approach the build system uses facilitates different target optimizations. - You could compile some binaries to use one set of libraries and other binaries - to use a different set of libraries. - The libraries could differ in architecture, compiler options, or other - optimizations. - </para> - - <para> - Several examples exist in the - <filename>meta-skeleton</filename> layer found in the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>: - <itemizedlist> - <listitem><para><filename>conf/multilib-example.conf</filename> - configuration file</para></listitem> - <listitem><para><filename>conf/multilib-example2.conf</filename> - configuration file</para></listitem> - <listitem><para><filename>recipes-multilib/images/core-image-multilib-example.bb</filename> - recipe</para></listitem> - </itemizedlist> - </para> - - <section id='preparing-to-use-multilib'> - <title>Preparing to Use Multilib</title> - - <para> - User-specific requirements drive the Multilib feature. - Consequently, there is no one "out-of-the-box" configuration that likely - exists to meet your needs. - </para> - - <para> - In order to enable Multilib, you first need to ensure your recipe is - extended to support multiple libraries. - Many standard recipes are already extended and support multiple libraries. - You can check in the <filename>meta/conf/multilib.conf</filename> - configuration file in the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> to see how this is - done using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBCLASSEXTEND'><filename>BBCLASSEXTEND</filename></ulink> - variable. - Eventually, all recipes will be covered and this list will - not be needed. - </para> - - <para> - For the most part, the Multilib class extension works automatically to - extend the package name from <filename>${PN}</filename> to - <filename>${MLPREFIX}${PN}</filename>, where <filename>MLPREFIX</filename> - is the particular multilib (e.g. "lib32-" or "lib64-"). - Standard variables such as - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-RPROVIDES'><filename>RPROVIDES</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink>, and - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink> - are automatically extended by the system. - If you are extending any manual code in the recipe, you can use the - <filename>${MLPREFIX}</filename> variable to ensure those names are extended - correctly. - This automatic extension code resides in <filename>multilib.bbclass</filename>. - </para> - </section> - - <section id='using-multilib'> - <title>Using Multilib</title> - - <para> - After you have set up the recipes, you need to define the actual - combination of multiple libraries you want to build. - You accomplish this through your <filename>local.conf</filename> - configuration file in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - An example configuration would be as follows: - <literallayout class='monospaced'> - MACHINE = "qemux86-64" - require conf/multilib.conf - MULTILIBS = "multilib:lib32" - DEFAULTTUNE_virtclass-multilib-lib32 = "x86" - IMAGE_INSTALL_append = " lib32-glib-2.0" - </literallayout> - This example enables an - additional library named <filename>lib32</filename> alongside the - normal target packages. - When combining these "lib32" alternatives, the example uses "x86" for tuning. - For information on this particular tuning, see - <filename>meta/conf/machine/include/ia32/arch-ia32.inc</filename>. - </para> - - <para> - The example then includes <filename>lib32-glib-2.0</filename> - in all the images, which illustrates one method of including a - multiple library dependency. - You can use a normal image build to include this dependency, - for example: - <literallayout class='monospaced'> - $ bitbake core-image-sato - </literallayout> - You can also build Multilib packages specifically with a command like this: - <literallayout class='monospaced'> - $ bitbake lib32-glib-2.0 - </literallayout> - </para> - </section> - - <section id='additional-implementation-details'> - <title>Additional Implementation Details</title> - - <para> - Generic implementation details as well as details that are - specific to package management systems exist. - Following are implementation details that exist regardless - of the package management system: - <itemizedlist> - <listitem><para>The typical convention used for the - class extension code as used by - Multilib assumes that all package names specified - in - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink> - that contain <filename>${PN}</filename> have - <filename>${PN}</filename> at the start of the name. - When that convention is not followed and - <filename>${PN}</filename> appears at - the middle or the end of a name, problems occur. - </para></listitem> - <listitem><para>The - <ulink url='&YOCTO_DOCS_REF_URL;#var-TARGET_VENDOR'><filename>TARGET_VENDOR</filename></ulink> - value under Multilib will be extended to - "-<replaceable>vendor</replaceable>ml<replaceable>multilib</replaceable>" - (e.g. "-pokymllib32" for a "lib32" Multilib with - Poky). - The reason for this slightly unwieldy contraction - is that any "-" characters in the vendor - string presently break Autoconf's - <filename>config.sub</filename>, and - other separators are problematic for different - reasons. - </para></listitem> - </itemizedlist> - </para> - - <para> - For the RPM Package Management System, the following implementation details - exist: - <itemizedlist> - <listitem><para>A unique architecture is defined for the Multilib packages, - along with creating a unique deploy folder under - <filename>tmp/deploy/rpm</filename> in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - For example, consider <filename>lib32</filename> in a - <filename>qemux86-64</filename> image. - The possible architectures in the system are "all", "qemux86_64", - "lib32_qemux86_64", and "lib32_x86".</para></listitem> - <listitem><para>The <filename>${MLPREFIX}</filename> variable is stripped from - <filename>${PN}</filename> during RPM packaging. - The naming for a normal RPM package and a Multilib RPM package in a - <filename>qemux86-64</filename> system resolves to something similar to - <filename>bash-4.1-r2.x86_64.rpm</filename> and - <filename>bash-4.1.r2.lib32_x86.rpm</filename>, respectively. - </para></listitem> - <listitem><para>When installing a Multilib image, the RPM backend first - installs the base image and then installs the Multilib libraries. - </para></listitem> - <listitem><para>The build system relies on RPM to resolve the identical files in the - two (or more) Multilib packages.</para></listitem> - </itemizedlist> - </para> - - <para> - For the IPK Package Management System, the following implementation details exist: - <itemizedlist> - <listitem><para>The <filename>${MLPREFIX}</filename> is not stripped from - <filename>${PN}</filename> during IPK packaging. - The naming for a normal RPM package and a Multilib IPK package in a - <filename>qemux86-64</filename> system resolves to something like - <filename>bash_4.1-r2.x86_64.ipk</filename> and - <filename>lib32-bash_4.1-rw_x86.ipk</filename>, respectively. - </para></listitem> - <listitem><para>The IPK deploy folder is not modified with - <filename>${MLPREFIX}</filename> because packages with and without - the Multilib feature can exist in the same folder due to the - <filename>${PN}</filename> differences.</para></listitem> - <listitem><para>IPK defines a sanity check for Multilib installation - using certain rules for file comparison, overridden, etc. - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id='installing-multiple-versions-of-the-same-library'> - <title>Installing Multiple Versions of the Same Library</title> - - <para> - Situations can exist where you need to install and use - multiple versions of the same library on the same system - at the same time. - These situations almost always exist when a library API - changes and you have multiple pieces of software that - depend on the separate versions of the library. - To accommodate these situations, you can install multiple - versions of the same library in parallel on the same system. - </para> - - <para> - The process is straightforward as long as the libraries use - proper versioning. - With properly versioned libraries, all you need to do to - individually specify the libraries is create separate, - appropriately named recipes where the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> part of the - name includes a portion that differentiates each library version - (e.g.the major part of the version number). - Thus, instead of having a single recipe that loads one version - of a library (e.g. <filename>clutter</filename>), you provide - multiple recipes that result in different versions - of the libraries you want. - As an example, the following two recipes would allow the - two separate versions of the <filename>clutter</filename> - library to co-exist on the same system: - <literallayout class='monospaced'> - clutter-1.6_1.6.20.bb - clutter-1.8_1.8.4.bb - </literallayout> - Additionally, if you have other recipes that depend on a given - library, you need to use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> - variable to create the dependency. - Continuing with the same example, if you want to have a recipe - depend on the 1.8 version of the <filename>clutter</filename> - library, use the following in your recipe: - <literallayout class='monospaced'> - DEPENDS = "clutter-1.8" - </literallayout> - </para> - </section> - </section> - - <section id='using-x32-psabi'> - <title>Using x32 psABI</title> - - <para> - x32 processor-specific Application Binary Interface - (<ulink url='https://software.intel.com/en-us/node/628948'>x32 psABI</ulink>) - is a native 32-bit processor-specific ABI for - <trademark class='registered'>Intel</trademark> 64 (x86-64) - architectures. - An ABI defines the calling conventions between functions in a - processing environment. - The interface determines what registers are used and what the - sizes are for various C data types. - </para> - - <para> - Some processing environments prefer using 32-bit applications even - when running on Intel 64-bit platforms. - Consider the i386 psABI, which is a very old 32-bit ABI for Intel - 64-bit platforms. - The i386 psABI does not provide efficient use and access of the - Intel 64-bit processor resources, leaving the system underutilized. - Now consider the x86_64 psABI. - This ABI is newer and uses 64-bits for data sizes and program - pointers. - The extra bits increase the footprint size of the programs, - libraries, and also increases the memory and file system size - requirements. - Executing under the x32 psABI enables user programs to utilize CPU - and system resources more efficiently while keeping the memory - footprint of the applications low. - Extra bits are used for registers but not for addressing mechanisms. - </para> - - <para> - The Yocto Project supports the final specifications of x32 psABI - as follows: - <itemizedlist> - <listitem><para> - You can create packages and images in x32 psABI format on - x86_64 architecture targets. - </para></listitem> - <listitem><para> - You can successfully build recipes with the x32 toolchain. - </para></listitem> - <listitem><para> - You can create and boot - <filename>core-image-minimal</filename> and - <filename>core-image-sato</filename> images. - </para></listitem> - <listitem><para> - RPM Package Manager (RPM) support exists for x32 binaries. - </para></listitem> - <listitem><para> - Support for large images exists. - </para></listitem> - </itemizedlist> - </para> - - <para> - To use the x32 psABI, you need to edit your - <filename>conf/local.conf</filename> configuration file as - follows: - <literallayout class='monospaced'> - MACHINE = "qemux86-64" - DEFAULTTUNE = "x86-64-x32" - baselib = "${@d.getVar('BASE_LIB_tune-' + (d.getVar('DEFAULTTUNE') \ - or 'INVALID')) or 'lib'}" - </literallayout> - Once you have set up your configuration file, use BitBake to - build an image that supports the x32 psABI. - Here is an example: - <literallayout class='monospaced'> - $ bitbake core-image-sato - </literallayout> - </para> - </section> - - <section id='enabling-gobject-introspection-support'> - <title>Enabling GObject Introspection Support</title> - - <para> - <ulink url='https://wiki.gnome.org/Projects/GObjectIntrospection'>GObject introspection</ulink> - is the standard mechanism for accessing GObject-based software - from runtime environments. - GObject is a feature of the GLib library that provides an object - framework for the GNOME desktop and related software. - GObject Introspection adds information to GObject that allows - objects created within it to be represented across different - programming languages. - If you want to construct GStreamer pipelines using Python, or - control UPnP infrastructure using Javascript and GUPnP, - GObject introspection is the only way to do it. - </para> - - <para> - This section describes the Yocto Project support for generating - and packaging GObject introspection data. - GObject introspection data is a description of the - API provided by libraries built on top of GLib framework, - and, in particular, that framework's GObject mechanism. - GObject Introspection Repository (GIR) files go to - <filename>-dev</filename> packages, - <filename>typelib</filename> files go to main packages as they - are packaged together with libraries that are introspected. - </para> - - <para> - The data is generated when building such a library, by linking - the library with a small executable binary that asks the library - to describe itself, and then executing the binary and - processing its output. - </para> - - <para> - Generating this data in a cross-compilation environment - is difficult because the library is produced for the target - architecture, but its code needs to be executed on the build host. - This problem is solved with the OpenEmbedded build system by - running the code through QEMU, which allows precisely that. - Unfortunately, QEMU does not always work perfectly as mentioned - in the - "<link linkend='known-issues'>Known Issues</link>" section. - </para> - - <section id='enabling-the-generation-of-introspection-data'> - <title>Enabling the Generation of Introspection Data</title> - - <para> - Enabling the generation of introspection data (GIR files) - in your library package involves the following: - <orderedlist> - <listitem><para> - Inherit the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-gobject-introspection'><filename>gobject-introspection</filename></ulink> - class. - </para></listitem> - <listitem><para> - Make sure introspection is not disabled anywhere in - the recipe or from anything the recipe includes. - Also, make sure that "gobject-introspection-data" is - not in - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink> - and that "qemu-usermode" is not in - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED'><filename>MACHINE_FEATURES_BACKFILL_CONSIDERED</filename></ulink>. - If either of these conditions exist, nothing will - happen. - </para></listitem> - <listitem><para> - Try to build the recipe. - If you encounter build errors that look like - something is unable to find - <filename>.so</filename> libraries, check where these - libraries are located in the source tree and add - the following to the recipe: - <literallayout class='monospaced'> - GIR_EXTRA_LIBS_PATH = "${B}/<replaceable>something</replaceable>/.libs" - </literallayout> - <note> - See recipes in the <filename>oe-core</filename> - repository that use that - <filename>GIR_EXTRA_LIBS_PATH</filename> variable - as an example. - </note> - </para></listitem> - <listitem><para> - Look for any other errors, which probably mean that - introspection support in a package is not entirely - standard, and thus breaks down in a cross-compilation - environment. - For such cases, custom-made fixes are needed. - A good place to ask and receive help in these cases - is the - <ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Yocto Project mailing lists</ulink>. - </para></listitem> - </orderedlist> - <note> - Using a library that no longer builds against the latest - Yocto Project release and prints introspection related - errors is a good candidate for the previous procedure. - </note> - </para> - </section> - - <section id='disabling-the-generation-of-introspection-data'> - <title>Disabling the Generation of Introspection Data</title> - - <para> - You might find that you do not want to generate - introspection data. - Or, perhaps QEMU does not work on your build host and - target architecture combination. - If so, you can use either of the following methods to - disable GIR file generations: - <itemizedlist> - <listitem><para> - Add the following to your distro configuration: - <literallayout class='monospaced'> - DISTRO_FEATURES_BACKFILL_CONSIDERED = "gobject-introspection-data" - </literallayout> - Adding this statement disables generating - introspection data using QEMU but will still enable - building introspection tools and libraries - (i.e. building them does not require the use of QEMU). - </para></listitem> - <listitem><para> - Add the following to your machine configuration: - <literallayout class='monospaced'> - MACHINE_FEATURES_BACKFILL_CONSIDERED = "qemu-usermode" - </literallayout> - Adding this statement disables the use of QEMU - when building packages for your machine. - Currently, this feature is used only by introspection - recipes and has the same effect as the previously - described option. - <note> - Future releases of the Yocto Project might have - other features affected by this option. - </note> - </para></listitem> - </itemizedlist> - If you disable introspection data, you can still - obtain it through other means such as copying the data - from a suitable sysroot, or by generating it on the - target hardware. - The OpenEmbedded build system does not currently - provide specific support for these techniques. - </para> - </section> - - <section id='testing-that-introspection-works-in-an-image'> - <title>Testing that Introspection Works in an Image</title> - - <para> - Use the following procedure to test if generating - introspection data is working in an image: - <orderedlist> - <listitem><para> - Make sure that "gobject-introspection-data" is not in - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink> - and that "qemu-usermode" is not in - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE_FEATURES_BACKFILL_CONSIDERED'><filename>MACHINE_FEATURES_BACKFILL_CONSIDERED</filename></ulink>. - </para></listitem> - <listitem><para> - Build <filename>core-image-sato</filename>. - </para></listitem> - <listitem><para> - Launch a Terminal and then start Python in the - terminal. - </para></listitem> - <listitem><para> - Enter the following in the terminal: - <literallayout class='monospaced'> - >>> from gi.repository import GLib - >>> GLib.get_host_name() - </literallayout> - </para></listitem> - <listitem><para> - For something a little more advanced, enter the - following: - <literallayout class='monospaced'> - http://python-gtk-3-tutorial.readthedocs.org/en/latest/introduction.html - </literallayout> - </para></listitem> - </orderedlist> - </para> - </section> - - <section id='known-issues'> - <title>Known Issues</title> - - <para> - The following know issues exist for - GObject Introspection Support: - <itemizedlist> - <listitem><para> - <filename>qemu-ppc64</filename> immediately crashes. - Consequently, you cannot build introspection data on - that architecture. - </para></listitem> - <listitem><para> - x32 is not supported by QEMU. - Consequently, introspection data is disabled. - </para></listitem> - <listitem><para> - musl causes transient GLib binaries to crash on - assertion failures. - Consequently, generating introspection data is - disabled. - </para></listitem> - <listitem><para> - Because QEMU is not able to run the binaries correctly, - introspection is disabled for some specific packages - under specific architectures (e.g. - <filename>gcr</filename>, - <filename>libsecret</filename>, and - <filename>webkit</filename>). - </para></listitem> - <listitem><para> - QEMU usermode might not work properly when running - 64-bit binaries under 32-bit host machines. - In particular, "qemumips64" is known to not work under - i686. - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id='dev-optionally-using-an-external-toolchain'> - <title>Optionally Using an External Toolchain</title> - - <para> - You might want to use an external toolchain as part of your - development. - If this is the case, the fundamental steps you need to accomplish - are as follows: - <itemizedlist> - <listitem><para> - Understand where the installed toolchain resides. - For cases where you need to build the external toolchain, - you would need to take separate steps to build and install - the toolchain. - </para></listitem> - <listitem><para> - Make sure you add the layer that contains the toolchain to - your <filename>bblayers.conf</filename> file through the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBLAYERS'><filename>BBLAYERS</filename></ulink> - variable. - </para></listitem> - <listitem><para> - Set the <filename>EXTERNAL_TOOLCHAIN</filename> - variable in your <filename>local.conf</filename> file - to the location in which you installed the toolchain. - </para></listitem> - </itemizedlist> - A good example of an external toolchain used with the Yocto Project - is <trademark class='registered'>Mentor Graphics</trademark> - Sourcery G++ Toolchain. - You can see information on how to use that particular layer in the - <filename>README</filename> file at - <ulink url='http://github.com/MentorEmbedded/meta-sourcery/'></ulink>. - You can find further information by reading about the - <ulink url='&YOCTO_DOCS_REF_URL;#var-TCMODE'><filename>TCMODE</filename></ulink> - variable in the Yocto Project Reference Manual's variable glossary. - </para> - </section> - - <section id='creating-partitioned-images-using-wic'> - <title>Creating Partitioned Images Using Wic</title> - - <para> - Creating an image for a particular hardware target using the - OpenEmbedded build system does not necessarily mean you can boot - that image as is on your device. - Physical devices accept and boot images in various ways depending - on the specifics of the device. - Usually, information about the hardware can tell you what image - format the device requires. - Should your device require multiple partitions on an SD card, flash, - or an HDD, you can use the OpenEmbedded Image Creator, - Wic, to create the properly partitioned image. - </para> - - <para> - The <filename>wic</filename> command generates partitioned - images from existing OpenEmbedded build artifacts. - Image generation is driven by partitioning commands - contained in an Openembedded kickstart file - (<filename>.wks</filename>) specified either directly on - the command line or as one of a selection of canned - kickstart files as shown with the - <filename>wic list images</filename> command in the - "<link linkend='using-a-provided-kickstart-file'>Using an Existing Kickstart File</link>" - section. - When you apply the command to a given set of build - artifacts, the result is an image or set of images that - can be directly written onto media and used on a particular - system. - <note> - For a kickstart file reference, see the - "<ulink url='&YOCTO_DOCS_REF_URL;#ref-kickstart'>OpenEmbedded Kickstart (<filename>.wks</filename>) Reference</ulink>" - Chapter in the Yocto Project Reference Manual. - </note> - </para> - - <para> - The <filename>wic</filename> command and the infrastructure - it is based on is by definition incomplete. - The purpose of the command is to allow the generation of - customized images, and as such, was designed to be - completely extensible through a plugin interface. - See the - "<link linkend='wic-using-the-wic-plugin-interface'>Using the Wic PlugIn Interface</link>" - section for information on these plugins. - </para> - - <para> - This section provides some background information on Wic, - describes what you need to have in - place to run the tool, provides instruction on how to use - the Wic utility, provides information on using the Wic plugins - interface, and provides several examples that show how to use - Wic. - </para> - - <section id='wic-background'> - <title>Background</title> - - <para> - This section provides some background on the Wic utility. - While none of this information is required to use - Wic, you might find it interesting. - <itemizedlist> - <listitem><para> - The name "Wic" is derived from OpenEmbedded - Image Creator (oeic). - The "oe" diphthong in "oeic" was promoted to the - letter "w", because "oeic" is both difficult to - remember and to pronounce. - </para></listitem> - <listitem><para> - Wic is loosely based on the - Meego Image Creator (<filename>mic</filename>) - framework. - The Wic implementation has been - heavily modified to make direct use of OpenEmbedded - build artifacts instead of package installation and - configuration, which are already incorporated within - the OpenEmbedded artifacts. - </para></listitem> - <listitem><para> - Wic is a completely independent - standalone utility that initially provides - easier-to-use and more flexible replacements for an - existing functionality in OE-Core's - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-image-live'><filename>image-live</filename></ulink> - class. - The difference between Wic and those examples is - that with Wic the functionality of those scripts is - implemented by a general-purpose partitioning language, - which is based on Redhat kickstart syntax. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='wic-requirements'> - <title>Requirements</title> - - <para> - In order to use the Wic utility with the OpenEmbedded Build - system, your system needs to meet the following - requirements: - <itemizedlist> - <listitem><para> - The Linux distribution on your development host must - support the Yocto Project. - See the - "<ulink url='&YOCTO_DOCS_REF_URL;#detailed-supported-distros'>Supported Linux Distributions</ulink>" - section in the Yocto Project Reference Manual for - the list of distributions that support the - Yocto Project. - </para></listitem> - <listitem><para> - The standard system utilities, such as - <filename>cp</filename>, must be installed on your - development host system. - </para></listitem> - <listitem><para> - You must have sourced the build environment - setup script (i.e. - <ulink url='&YOCTO_DOCS_REF_URL;#structure-core-script'><filename>&OE_INIT_FILE;</filename></ulink>) - found in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - </para></listitem> - <listitem><para> - You need to have the build artifacts already - available, which typically means that you must - have already created an image using the - Openembedded build system (e.g. - <filename>core-image-minimal</filename>). - While it might seem redundant to generate an image - in order to create an image using - Wic, the current version of - Wic requires the artifacts - in the form generated by the OpenEmbedded build - system. - </para></listitem> - <listitem><para> - You must build several native tools, which are - built to run on the build system: - <literallayout class='monospaced'> - $ bitbake parted-native dosfstools-native mtools-native - </literallayout> - </para></listitem> - <listitem><para> - Include "wic" as part of the - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink> - variable. - </para></listitem> - <listitem><para> - Include the name of the - <ulink url='&YOCTO_DOCS_REF_URL;#openembedded-kickstart-wks-reference'>wic kickstart file</ulink> - as part of the - <ulink url='&YOCTO_DOCS_REF_URL;#var-WKS_FILE'><filename>WKS_FILE</filename></ulink> - variable - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='wic-getting-help'> - <title>Getting Help</title> - - <para> - You can get general help for the <filename>wic</filename> - command by entering the <filename>wic</filename> command - by itself or by entering the command with a help argument - as follows: - <literallayout class='monospaced'> - $ wic -h - $ wic --help - $ wic help - </literallayout> - </para> - - <para> - Currently, Wic supports seven commands: - <filename>cp</filename>, <filename>create</filename>, - <filename>help</filename>, <filename>list</filename>, - <filename>ls</filename>, <filename>rm</filename>, and - <filename>write</filename>. - You can get help for all these commands except "help" by - using the following form: - <literallayout class='monospaced'> - $ wic help <replaceable>command</replaceable> - </literallayout> - For example, the following command returns help for the - <filename>write</filename> command: - <literallayout class='monospaced'> - $ wic help write - </literallayout> - </para> - - <para> - Wic supports help for three topics: - <filename>overview</filename>, - <filename>plugins</filename>, and - <filename>kickstart</filename>. - You can get help for any topic using the following form: - <literallayout class='monospaced'> - $ wic help <replaceable>topic</replaceable> - </literallayout> - For example, the following returns overview help for Wic: - <literallayout class='monospaced'> - $ wic help overview - </literallayout> - </para> - - <para> - One additional level of help exists for Wic. - You can get help on individual images through the - <filename>list</filename> command. - You can use the <filename>list</filename> command to return the - available Wic images as follows: - <literallayout class='monospaced'> - $ wic list images - genericx86 Create an EFI disk image for genericx86* - beaglebone-yocto Create SD card image for Beaglebone - edgerouter Create SD card image for Edgerouter - qemux86-directdisk Create a qemu machine 'pcbios' direct disk image - directdisk-gpt Create a 'pcbios' direct disk image - mkefidisk Create an EFI disk image - directdisk Create a 'pcbios' direct disk image - systemd-bootdisk Create an EFI disk image with systemd-boot - mkhybridiso Create a hybrid ISO image - sdimage-bootpart Create SD card image with a boot partition - directdisk-multi-rootfs Create multi rootfs image using rootfs plugin - directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config - </literallayout> - Once you know the list of available Wic images, you can use - <filename>help</filename> with the command to get help on a - particular image. - For example, the following command returns help on the - "beaglebone-yocto" image: - <literallayout class='monospaced'> - $ wic list beaglebone-yocto help - - - Creates a partitioned SD card image for Beaglebone. - Boot files are located in the first vfat partition. - </literallayout> - </para> - </section> - - <section id='operational-modes'> - <title>Operational Modes</title> - - <para> - You can use Wic in two different - modes, depending on how much control you need for - specifying the Openembedded build artifacts that are - used for creating the image: Raw and Cooked: - <itemizedlist> - <listitem><para> - <emphasis>Raw Mode:</emphasis> - You explicitly specify build artifacts through - Wic command-line arguments. - </para></listitem> - <listitem><para> - <emphasis>Cooked Mode:</emphasis> - The current - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> - setting and image name are used to automatically - locate and provide the build artifacts. - You just supply a kickstart file and the name - of the image from which to use artifacts. - </para></listitem> - </itemizedlist> - </para> - - <para> - Regardless of the mode you use, you need to have the build - artifacts ready and available. - </para> - - <section id='raw-mode'> - <title>Raw Mode</title> - - <para> - Running Wic in raw mode allows you to specify all the - partitions through the <filename>wic</filename> - command line. - The primary use for raw mode is if you have built - your kernel outside of the Yocto Project - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - In other words, you can point to arbitrary kernel, - root filesystem locations, and so forth. - Contrast this behavior with cooked mode where Wic - looks in the Build Directory (e.g. - <filename>tmp/deploy/images/</filename><replaceable>machine</replaceable>). - </para> - - <para> - The general form of the - <filename>wic</filename> command in raw mode is: - <literallayout class='monospaced'> - $ wic create <replaceable>wks_file</replaceable> <replaceable>options</replaceable> ... - - Where: - - <replaceable>wks_file</replaceable>: - An OpenEmbedded kickstart file. You can provide - your own custom file or use a file from a set of - existing files as described by further options. - - optional arguments: - -h, --help show this help message and exit - -o <replaceable>OUTDIR</replaceable>, --outdir <replaceable>OUTDIR</replaceable> - name of directory to create image in - -e <replaceable>IMAGE_NAME</replaceable>, --image-name <replaceable>IMAGE_NAME</replaceable> - name of the image to use the artifacts from e.g. core- - image-sato - -r <replaceable>ROOTFS_DIR</replaceable>, --rootfs-dir <replaceable>ROOTFS_DIR</replaceable> - path to the /rootfs dir to use as the .wks rootfs - source - -b <replaceable>BOOTIMG_DIR</replaceable>, --bootimg-dir <replaceable>BOOTIMG_DIR</replaceable> - path to the dir containing the boot artifacts (e.g. - /EFI or /syslinux dirs) to use as the .wks bootimg - source - -k <replaceable>KERNEL_DIR</replaceable>, --kernel-dir <replaceable>KERNEL_DIR</replaceable> - path to the dir containing the kernel to use in the - .wks bootimg - -n <replaceable>NATIVE_SYSROOT</replaceable>, --native-sysroot <replaceable>NATIVE_SYSROOT</replaceable> - path to the native sysroot containing the tools to use - to build the image - -s, --skip-build-check - skip the build check - -f, --build-rootfs build rootfs - -c {gzip,bzip2,xz}, --compress-with {gzip,bzip2,xz} - compress image with specified compressor - -m, --bmap generate .bmap - --no-fstab-update Do not change fstab file. - -v <replaceable>VARS_DIR</replaceable>, --vars <replaceable>VARS_DIR</replaceable> - directory with <image>.env files that store bitbake - variables - -D, --debug output debug information - </literallayout> - <note> - You do not need root privileges to run - Wic. - In fact, you should not run as root when using the - utility. - </note> - </para> - </section> - - <section id='cooked-mode'> - <title>Cooked Mode</title> - - <para> - Running Wic in cooked mode leverages off artifacts in - the Build Directory. - In other words, you do not have to specify kernel or - root filesystem locations as part of the command. - All you need to provide is a kickstart file and the - name of the image from which to use artifacts by using - the "-e" option. - Wic looks in the Build Directory (e.g. - <filename>tmp/deploy/images/</filename><replaceable>machine</replaceable>) - for artifacts. - </para> - - <para> - The general form of the <filename>wic</filename> - command using Cooked Mode is as follows: - <literallayout class='monospaced'> - $ wic create <replaceable>wks_file</replaceable> -e <replaceable>IMAGE_NAME</replaceable> - - Where: - - <replaceable>wks_file</replaceable>: - An OpenEmbedded kickstart file. You can provide - your own custom file or use a file from a set of - existing files provided with the Yocto Project - release. - - required argument: - -e <replaceable>IMAGE_NAME</replaceable>, --image-name <replaceable>IMAGE_NAME</replaceable> - name of the image to use the artifacts from e.g. core- - image-sato - </literallayout> - </para> - </section> - </section> - - <section id='using-a-provided-kickstart-file'> - <title>Using an Existing Kickstart File</title> - - <para> - If you do not want to create your own kickstart file, you - can use an existing file provided by the Wic installation. - As shipped, kickstart files can be found in the - Yocto Project - <ulink url='&YOCTO_DOCS_OM_URL;#source-repositories'>Source Repositories</ulink> - in the following two locations: - <literallayout class='monospaced'> - poky/meta-yocto-bsp/wic - poky/scripts/lib/wic/canned-wks - </literallayout> - Use the following command to list the available kickstart - files: - <literallayout class='monospaced'> - $ wic list images - genericx86 Create an EFI disk image for genericx86* - beaglebone-yocto Create SD card image for Beaglebone - edgerouter Create SD card image for Edgerouter - qemux86-directdisk Create a qemu machine 'pcbios' direct disk image - directdisk-gpt Create a 'pcbios' direct disk image - mkefidisk Create an EFI disk image - directdisk Create a 'pcbios' direct disk image - systemd-bootdisk Create an EFI disk image with systemd-boot - mkhybridiso Create a hybrid ISO image - sdimage-bootpart Create SD card image with a boot partition - directdisk-multi-rootfs Create multi rootfs image using rootfs plugin - directdisk-bootloader-config Create a 'pcbios' direct disk image with custom bootloader config - </literallayout> - When you use an existing file, you do not have to use the - <filename>.wks</filename> extension. - Here is an example in Raw Mode that uses the - <filename>directdisk</filename> file: - <literallayout class='monospaced'> - $ wic create directdisk -r <replaceable>rootfs_dir</replaceable> -b <replaceable>bootimg_dir</replaceable> \ - -k <replaceable>kernel_dir</replaceable> -n <replaceable>native_sysroot</replaceable> - </literallayout> - </para> - - <para> - Here are the actual partition language commands - used in the <filename>genericx86.wks</filename> file to - generate an image: - <literallayout class='monospaced'> - # short-description: Create an EFI disk image for genericx86* - # long-description: Creates a partitioned EFI disk image for genericx86* machines - part /boot --source bootimg-efi --sourceparams="loader=grub-efi" --ondisk sda --label msdos --active --align 1024 - part / --source rootfs --ondisk sda --fstype=ext4 --label platform --align 1024 --use-uuid - part swap --ondisk sda --size 44 --label swap1 --fstype=swap - - bootloader --ptable gpt --timeout=5 --append="rootfstype=ext4 console=ttyS0,115200 console=tty0" - </literallayout> - </para> - </section> - - <section id='wic-using-the-wic-plugin-interface'> - <title>Using the Wic Plugin Interface</title> - - <para> - You can extend and specialize Wic functionality by using - Wic plugins. - This section explains the Wic plugin interface. - <note> - Wic plugins consist of "source" and "imager" plugins. - Imager plugins are beyond the scope of this section. - </note> - </para> - - <para> - Source plugins provide a mechanism to customize partition - content during the Wic image generation process. - You can use source plugins to map values that you specify - using <filename>--source</filename> commands in kickstart - files (i.e. <filename>*.wks</filename>) to a plugin - implementation used to populate a given partition. - <note> - If you use plugins that have build-time dependencies - (e.g. native tools, bootloaders, and so forth) - when building a Wic image, you need to specify those - dependencies using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-WKS_FILE_DEPENDS'><filename>WKS_FILE_DEPENDS</filename></ulink> - variable. - </note> - </para> - - <para> - Source plugins are subclasses defined in plugin files. - As shipped, the Yocto Project provides several plugin - files. - You can see the source plugin files that ship with the - Yocto Project - <ulink url='&YOCTO_GIT_URL;/cgit/cgit.cgi/poky/tree/scripts/lib/wic/plugins/source'>here</ulink>. - Each of these plugin files contains source plugins that - are designed to populate a specific Wic image partition. - </para> - - <para> - Source plugins are subclasses of the - <filename>SourcePlugin</filename> class, which is - defined in the - <filename>poky/scripts/lib/wic/pluginbase.py</filename> - file. - For example, the <filename>BootimgEFIPlugin</filename> - source plugin found in the - <filename>bootimg-efi.py</filename> file is a subclass of - the <filename>SourcePlugin</filename> class, which is found - in the <filename>pluginbase.py</filename> file. - </para> - - <para> - You can also implement source plugins in a layer outside - of the Source Repositories (external layer). - To do so, be sure that your plugin files are located in - a directory whose path is - <filename>scripts/lib/wic/plugins/source/</filename> - within your external layer. - When the plugin files are located there, the source - plugins they contain are made available to Wic. - </para> - - <para> - When the Wic implementation needs to invoke a - partition-specific implementation, it looks for the plugin - with the same name as the <filename>--source</filename> - parameter used in the kickstart file given to that - partition. - For example, if the partition is set up using the following - command in a kickstart file: - <literallayout class='monospaced'> - part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 - </literallayout> - The methods defined as class members of the matching - source plugin (i.e. <filename>bootimg-pcbios</filename>) - in the <filename>bootimg-pcbios.py</filename> plugin file - are used. - </para> - - <para> - To be more concrete, here is the corresponding plugin - definition from the <filename>bootimg-pcbios.py</filename> - file for the previous command along with an example - method called by the Wic implementation when it needs to - prepare a partition using an implementation-specific - function: - <literallayout class='monospaced'> - . - . - . - class BootimgPcbiosPlugin(SourcePlugin): - """ - Create MBR boot partition and install syslinux on it. - """ - - name = 'bootimg-pcbios' - . - . - . - @classmethod - def do_prepare_partition(cls, part, source_params, creator, cr_workdir, - oe_builddir, bootimg_dir, kernel_dir, - rootfs_dir, native_sysroot): - """ - Called to do the actual content population for a partition i.e. it - 'prepares' the partition to be incorporated into the image. - In this case, prepare content for legacy bios boot partition. - """ - . - . - . - </literallayout> - If a subclass (plugin) itself does not implement a - particular function, Wic locates and uses the default - version in the superclass. - It is for this reason that all source plugins are derived - from the <filename>SourcePlugin</filename> class. - </para> - - <para> - The <filename>SourcePlugin</filename> class defined in - the <filename>pluginbase.py</filename> file defines - a set of methods that source plugins can implement or - override. - Any plugins (subclass of - <filename>SourcePlugin</filename>) that do not implement - a particular method inherit the implementation of the - method from the <filename>SourcePlugin</filename> class. - For more information, see the - <filename>SourcePlugin</filename> class in the - <filename>pluginbase.py</filename> file for details: - </para> - - <para> - The following list describes the methods implemented in the - <filename>SourcePlugin</filename> class: - <itemizedlist> - <listitem><para> - <emphasis><filename>do_prepare_partition()</filename>:</emphasis> - Called to populate a partition with actual content. - In other words, the method prepares the final - partition image that is incorporated into the - disk image. - </para></listitem> - <listitem><para> - <emphasis><filename>do_configure_partition()</filename>:</emphasis> - Called before - <filename>do_prepare_partition()</filename> to - create custom configuration files for a partition - (e.g. syslinux or grub configuration files). - </para></listitem> - <listitem><para> - <emphasis><filename>do_install_disk()</filename>:</emphasis> - Called after all partitions have been prepared and - assembled into a disk image. - This method provides a hook to allow finalization - of a disk image (e.g. writing an MBR). - </para></listitem> - <listitem><para> - <emphasis><filename>do_stage_partition()</filename>:</emphasis> - Special content-staging hook called before - <filename>do_prepare_partition()</filename>. - This method is normally empty.</para> - - <para>Typically, a partition just uses the passed-in - parameters (e.g. the unmodified value of - <filename>bootimg_dir</filename>). - However, in some cases, things might need to be - more tailored. - As an example, certain files might additionally - need to be taken from - <filename>bootimg_dir + /boot</filename>. - This hook allows those files to be staged in a - customized fashion. - <note> - <filename>get_bitbake_var()</filename> - allows you to access non-standard variables - that you might want to use for this - behavior. - </note> - </para></listitem> - </itemizedlist> - </para> - - <para> - You can extend the source plugin mechanism. - To add more hooks, create more source plugin methods - within <filename>SourcePlugin</filename> and the - corresponding derived subclasses. - The code that calls the plugin methods uses the - <filename>plugin.get_source_plugin_methods()</filename> - function to find the method or methods needed by the call. - Retrieval of those methods is accomplished by filling up - a dict with keys that contain the method names of interest. - On success, these will be filled in with the actual - methods. - See the Wic implementation for examples and details. - </para> - </section> - - <section id='wic-usage-examples'> - <title>Examples</title> - - <para> - This section provides several examples that show how to use - the Wic utility. - All the examples assume the list of requirements in the - "<link linkend='wic-requirements'>Requirements</link>" - section have been met. - The examples assume the previously generated image is - <filename>core-image-minimal</filename>. - </para> - - <section id='generate-an-image-using-a-provided-kickstart-file'> - <title>Generate an Image using an Existing Kickstart File</title> - - <para> - This example runs in Cooked Mode and uses the - <filename>mkefidisk</filename> kickstart file: - <literallayout class='monospaced'> - $ wic create mkefidisk -e core-image-minimal - INFO: Building wic-tools... - . - . - . - INFO: The new image(s) can be found here: - ./mkefidisk-201804191017-sda.direct - - The following build artifacts were used to create the image(s): - ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs - BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share - KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 - NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native - - INFO: The image(s) were created using OE kickstart file: - /home/stephano/build/master/openembedded-core/scripts/lib/wic/canned-wks/mkefidisk.wks - </literallayout> - The previous example shows the easiest way to create - an image by running in cooked mode and supplying - a kickstart file and the "-e" option to point to the - existing build artifacts. - Your <filename>local.conf</filename> file needs to have - the - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> - variable set to the machine you are using, which is - "qemux86" in this example. - </para> - - <para> - Once the image builds, the output provides image - location, artifact use, and kickstart file information. - <note> - You should always verify the details provided in the - output to make sure that the image was indeed - created exactly as expected. - </note> - </para> - - <para> - Continuing with the example, you can now write the - image from the Build Directory onto a USB stick, or - whatever media for which you built your image, and boot - from the media. - You can write the image by using - <filename>bmaptool</filename> or - <filename>dd</filename>: - <literallayout class='monospaced'> - $ oe-run-native bmaptool copy mkefidisk-201804191017-sda.direct /dev/sd<replaceable>X</replaceable> - </literallayout> - or - <literallayout class='monospaced'> - $ sudo dd if=mkefidisk-201804191017-sda.direct of=/dev/sd<replaceable>X</replaceable> - </literallayout> - <note> - For more information on how to use the - <filename>bmaptool</filename> to flash a device - with an image, see the - "<link linkend='flashing-images-using-bmaptool'>Flashing Images Using <filename>bmaptool</filename></link>" - section. - </note> - </para> - </section> - - <section id='using-a-modified-kickstart-file'> - <title>Using a Modified Kickstart File</title> - - <para> - Because partitioned image creation is driven by the - kickstart file, it is easy to affect image creation by - changing the parameters in the file. - This next example demonstrates that through modification - of the <filename>directdisk-gpt</filename> kickstart - file. - </para> - - <para> - As mentioned earlier, you can use the command - <filename>wic list images</filename> to show the list - of existing kickstart files. - The directory in which the - <filename>directdisk-gpt.wks</filename> file resides is - <filename>scripts/lib/image/canned-wks/</filename>, - which is located in the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - (e.g. <filename>poky</filename>). - Because available files reside in this directory, - you can create and add your own custom files to the - directory. - Subsequent use of the - <filename>wic list images</filename> command would then - include your kickstart files. - </para> - - <para> - In this example, the existing - <filename>directdisk-gpt</filename> file already does - most of what is needed. - However, for the hardware in this example, the image - will need to boot from <filename>sdb</filename> instead - of <filename>sda</filename>, which is what the - <filename>directdisk-gpt</filename> kickstart file - uses. - </para> - - <para> - The example begins by making a copy of the - <filename>directdisk-gpt.wks</filename> file in the - <filename>scripts/lib/image/canned-wks</filename> - directory and then by changing the lines that specify - the target disk from which to boot. - <literallayout class='monospaced'> - $ cp /home/stephano/poky/scripts/lib/wic/canned-wks/directdisk-gpt.wks \ - /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks - </literallayout> - Next, the example modifies the - <filename>directdisksdb-gpt.wks</filename> file and - changes all instances of - "<filename>--ondisk sda</filename>" to - "<filename>--ondisk sdb</filename>". - The example changes the following two lines and leaves - the remaining lines untouched: - <literallayout class='monospaced'> - part /boot --source bootimg-pcbios --ondisk sdb --label boot --active --align 1024 - part / --source rootfs --ondisk sdb --fstype=ext4 --label platform --align 1024 --use-uuid - </literallayout> - Once the lines are changed, the example generates the - <filename>directdisksdb-gpt</filename> image. - The command points the process at the - <filename>core-image-minimal</filename> artifacts for - the Next Unit of Computing (nuc) - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> - the <filename>local.conf</filename>. - <literallayout class='monospaced'> - $ wic create directdisksdb-gpt -e core-image-minimal - INFO: Building wic-tools... - . - . - . - Initialising tasks: 100% |#######################################| Time: 0:00:01 - NOTE: Executing SetScene Tasks - NOTE: Executing RunQueue Tasks - NOTE: Tasks Summary: Attempted 1161 tasks of which 1157 didn't need to be rerun and all succeeded. - INFO: Creating image(s)... - - INFO: The new image(s) can be found here: - ./directdisksdb-gpt-201710090938-sdb.direct - - The following build artifacts were used to create the image(s): - ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs - BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share - KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 - NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native - - INFO: The image(s) were created using OE kickstart file: - /home/stephano/poky/scripts/lib/wic/canned-wks/directdisksdb-gpt.wks - </literallayout> - Continuing with the example, you can now directly - <filename>dd</filename> the image to a USB stick, or - whatever media for which you built your image, - and boot the resulting media: - <literallayout class='monospaced'> - $ sudo dd if=directdisksdb-gpt-201710090938-sdb.direct of=/dev/sdb - 140966+0 records in - 140966+0 records out - 72174592 bytes (72 MB, 69 MiB) copied, 78.0282 s, 925 kB/s - $ sudo eject /dev/sdb - </literallayout> - </para> - </section> - - <section id='using-a-modified-kickstart-file-and-running-in-raw-mode'> - <title>Using a Modified Kickstart File and Running in Raw Mode</title> - - <para> - This next example manually specifies each build artifact - (runs in Raw Mode) and uses a modified kickstart file. - The example also uses the <filename>-o</filename> option - to cause Wic to create the output - somewhere other than the default output directory, - which is the current directory: - <literallayout class='monospaced'> - $ wic create /home/stephano/my_yocto/test.wks -o /home/stephano/testwic \ - --rootfs-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/rootfs \ - --bootimg-dir /home/stephano/build/master/build/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share \ - --kernel-dir /home/stephano/build/master/build/tmp/deploy/images/qemux86 \ - --native-sysroot /home/stephano/build/master/build/tmp/work/i586-poky-linux/wic-tools/1.0-r0/recipe-sysroot-native - - INFO: Creating image(s)... - - INFO: The new image(s) can be found here: - /home/stephano/testwic/test-201710091445-sdb.direct - - The following build artifacts were used to create the image(s): - ROOTFS_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs - BOOTIMG_DIR: /home/stephano/build/master/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/recipe-sysroot/usr/share - KERNEL_DIR: /home/stephano/build/master/build/tmp-glibc/deploy/images/qemux86 - NATIVE_SYSROOT: /home/stephano/build/master/build/tmp-glibc/work/i586-oe-linux/wic-tools/1.0-r0/recipe-sysroot-native - - INFO: The image(s) were created using OE kickstart file: - /home/stephano/my_yocto/test.wks - </literallayout> - For this example, - <ulink url='&YOCTO_DOCS_REF_URL;#var-MACHINE'><filename>MACHINE</filename></ulink> - did not have to be specified in the - <filename>local.conf</filename> file since the - artifact is manually specified. - </para> - </section> - - <section id='using-wic-to-manipulate-an-image'> - <title>Using Wic to Manipulate an Image</title> - - <para> - Wic image manipulation allows you to shorten turnaround - time during image development. - For example, you can use Wic to delete the kernel partition - of a Wic image and then insert a newly built kernel. - This saves you time from having to rebuild the entire image - each time you modify the kernel. - <note> - In order to use Wic to manipulate a Wic image as in - this example, your development machine must have the - <filename>mtools</filename> package installed. - </note> - </para> - - <para> - The following example examines the contents of the Wic - image, deletes the existing kernel, and then inserts a - new kernel: - <orderedlist> - <listitem><para> - <emphasis>List the Partitions:</emphasis> - Use the <filename>wic ls</filename> command to list - all the partitions in the Wic image: - <literallayout class='monospaced'> - $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic - Num Start End Size Fstype - 1 1048576 25041919 23993344 fat16 - 2 25165824 72157183 46991360 ext4 - </literallayout> - The previous output shows two partitions in the - <filename>core-image-minimal-qemux86.wic</filename> - image. - </para></listitem> - <listitem><para> - <emphasis>Examine a Particular Partition:</emphasis> - Use the <filename>wic ls</filename> command again - but in a different form to examine a particular - partition. - <note> - You can get command usage on any Wic command - using the following form: - <literallayout class='monospaced'> - $ wic help <replaceable>command</replaceable> - </literallayout> - For example, the following command shows you - the various ways to use the - <filename>wic ls</filename> command: - <literallayout class='monospaced'> - $ wic help ls - </literallayout> - </note> - The following command shows what is in Partition - one: - <literallayout class='monospaced'> - $ wic ls tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1 - Volume in drive : is boot - Volume Serial Number is E894-1809 - Directory for ::/ - - libcom32 c32 186500 2017-10-09 16:06 - libutil c32 24148 2017-10-09 16:06 - syslinux cfg 220 2017-10-09 16:06 - vesamenu c32 27104 2017-10-09 16:06 - vmlinuz 6904608 2017-10-09 16:06 - 5 files 7 142 580 bytes - 16 582 656 bytes free - </literallayout> - The previous output shows five files, with the - <filename>vmlinuz</filename> being the kernel. - <note> - If you see the following error, you need to - update or create a - <filename>~/.mtoolsrc</filename> file and - be sure to have the line “mtools_skip_check=1“ - in the file. - Then, run the Wic command again: - <literallayout class='monospaced'> - ERROR: _exec_cmd: /usr/bin/mdir -i /tmp/wic-parttfokuwra ::/ returned '1' instead of 0 - output: Total number of sectors (47824) not a multiple of sectors per track (32)! - Add mtools_skip_check=1 to your .mtoolsrc file to skip this test - </literallayout> - </note> - </para></listitem> - <listitem><para> - <emphasis>Remove the Old Kernel:</emphasis> - Use the <filename>wic rm</filename> command to - remove the <filename>vmlinuz</filename> file - (kernel): - <literallayout class='monospaced'> - $ wic rm tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Add In the New Kernel:</emphasis> - Use the <filename>wic cp</filename> command to - add the updated kernel to the Wic image. - Depending on how you built your kernel, it could - be in different places. - If you used <filename>devtool</filename> and - an SDK to build your kernel, it resides in the - <filename>tmp/work</filename> directory of the - extensible SDK. - If you used <filename>make</filename> to build the - kernel, the kernel will be in the - <filename>workspace/sources</filename> area. - </para> - - <para>The following example assumes - <filename>devtool</filename> was used to build - the kernel: - <literallayout class='monospaced'> - cp ~/poky_sdk/tmp/work/qemux86-poky-linux/linux-yocto/4.12.12+git999-r0/linux-yocto-4.12.12+git999/arch/x86/boot/bzImage \ - ~/poky/build/tmp/deploy/images/qemux86/core-image-minimal-qemux86.wic:1/vmlinuz - </literallayout> - Once the new kernel is added back into the image, - you can use the <filename>dd</filename> - command or - <link linkend='flashing-images-using-bmaptool'><filename>bmaptool</filename></link> - to flash your wic image onto an SD card - or USB stick and test your target. - <note> - Using <filename>bmaptool</filename> is - generally 10 to 20 times faster than using - <filename>dd</filename>. - </note> - </para></listitem> - </orderedlist> - </para> - </section> - </section> - </section> - - <section id='flashing-images-using-bmaptool'> - <title>Flashing Images Using <filename>bmaptool</filename></title> - - <para> - A fast and easy way to flash an image to a bootable device - is to use Bmaptool, which is integrated into the OpenEmbedded - build system. - Bmaptool is a generic tool that creates a file's block map (bmap) - and then uses that map to copy the file. - As compared to traditional tools such as dd or cp, Bmaptool - can copy (or flash) large files like raw system image files - much faster. - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - If you are using Ubuntu or Debian distributions, you - can install the <filename>bmap-tools</filename> package - using the following command and then use the tool - without specifying <filename>PATH</filename> even from - the root account: - <literallayout class='monospaced'> - $ sudo apt-get install bmap-tools - </literallayout> - </para></listitem> - <listitem><para> - If you are unable to install the - <filename>bmap-tools</filename> package, you will - need to build Bmaptool before using it. - Use the following command: - <literallayout class='monospaced'> - $ bitbake bmap-tools-native - </literallayout> - </para></listitem> - </itemizedlist> - </note> - </para> - - <para> - Following, is an example that shows how to flash a Wic image. - Realize that while this example uses a Wic image, you can use - Bmaptool to flash any type of image. - Use these steps to flash an image using Bmaptool: - <orderedlist> - <listitem><para> - <emphasis>Update your <filename>local.conf</filename> File:</emphasis> - You need to have the following set in your - <filename>local.conf</filename> file before building - your image: - <literallayout class='monospaced'> - IMAGE_FSTYPES += "wic wic.bmap" - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Get Your Image:</emphasis> - Either have your image ready (pre-built with the - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FSTYPES'><filename>IMAGE_FSTYPES</filename></ulink> - setting previously mentioned) or take the step to build - the image: - <literallayout class='monospaced'> - $ bitbake <replaceable>image</replaceable> - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Flash the Device:</emphasis> - Flash the device with the image by using Bmaptool - depending on your particular setup. - The following commands assume the image resides in the - Build Directory's <filename>deploy/images/</filename> - area: - <itemizedlist> - <listitem><para> - If you have write access to the media, use this - command form: - <literallayout class='monospaced'> - $ oe-run-native bmap-tools-native bmaptool copy <replaceable>build-directory</replaceable>/tmp/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>.wic /dev/sd<replaceable>X</replaceable> - </literallayout> - </para></listitem> - <listitem><para> - If you do not have write access to the media, set - your permissions first and then use the same - command form: - <literallayout class='monospaced'> - $ sudo chmod 666 /dev/sd<replaceable>X</replaceable> - $ oe-run-native bmap-tools-native bmaptool copy <replaceable>build-directory</replaceable>/tmp/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>.wic /dev/sd<replaceable>X</replaceable> - </literallayout> - </para></listitem> - </itemizedlist> - </para></listitem> - </orderedlist> - </para> - - <para> - For help on the <filename>bmaptool</filename> command, use the - following command: - <literallayout class='monospaced'> - $ bmaptool --help - </literallayout> - </para> - </section> - - <section id='making-images-more-secure'> - <title>Making Images More Secure</title> - - <para> - Security is of increasing concern for embedded devices. - Consider the issues and problems discussed in just this - sampling of work found across the Internet: - <itemizedlist> - <listitem><para><emphasis> - "<ulink url='https://www.schneier.com/blog/archives/2014/01/security_risks_9.html'>Security Risks of Embedded Systems</ulink>"</emphasis> - by Bruce Schneier - </para></listitem> - <listitem><para><emphasis> - "<ulink url='http://census2012.sourceforge.net/paper.html'>Internet Census 2012</ulink>"</emphasis> - by Carna Botnet</para></listitem> - <listitem><para><emphasis> - "<ulink url='http://elinux.org/images/6/6f/Security-issues.pdf'>Security Issues for Embedded Devices</ulink>"</emphasis> - by Jake Edge - </para></listitem> - </itemizedlist> - </para> - - <para> - When securing your image is of concern, there are steps, tools, - and variables that you can consider to help you reach the - security goals you need for your particular device. - Not all situations are identical when it comes to making an - image secure. - Consequently, this section provides some guidance and suggestions - for consideration when you want to make your image more secure. - <note> - Because the security requirements and risks are - different for every type of device, this section cannot - provide a complete reference on securing your custom OS. - It is strongly recommended that you also consult other sources - of information on embedded Linux system hardening and on - security. - </note> - </para> - - <section id='general-considerations'> - <title>General Considerations</title> - - <para> - General considerations exist that help you create more - secure images. - You should consider the following suggestions to help - make your device more secure: - <itemizedlist> - <listitem><para> - Scan additional code you are adding to the system - (e.g. application code) by using static analysis - tools. - Look for buffer overflows and other potential - security problems. - </para></listitem> - <listitem><para> - Pay particular attention to the security for - any web-based administration interface. - </para> - <para>Web interfaces typically need to perform - administrative functions and tend to need to run with - elevated privileges. - Thus, the consequences resulting from the interface's - security becoming compromised can be serious. - Look for common web vulnerabilities such as - cross-site-scripting (XSS), unvalidated inputs, - and so forth.</para> - <para>As with system passwords, the default credentials - for accessing a web-based interface should not be the - same across all devices. - This is particularly true if the interface is enabled - by default as it can be assumed that many end-users - will not change the credentials. - </para></listitem> - <listitem><para> - Ensure you can update the software on the device to - mitigate vulnerabilities discovered in the future. - This consideration especially applies when your - device is network-enabled. - </para></listitem> - <listitem><para> - Ensure you remove or disable debugging functionality - before producing the final image. - For information on how to do this, see the - "<link linkend='considerations-specific-to-the-openembedded-build-system'>Considerations Specific to the OpenEmbedded Build System</link>" - section. - </para></listitem> - <listitem><para> - Ensure you have no network services listening that - are not needed. - </para></listitem> - <listitem><para> - Remove any software from the image that is not needed. - </para></listitem> - <listitem><para> - Enable hardware support for secure boot functionality - when your device supports this functionality. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='security-flags'> - <title>Security Flags</title> - - <para> - The Yocto Project has security flags that you can enable that - help make your build output more secure. - The security flags are in the - <filename>meta/conf/distro/include/security_flags.inc</filename> - file in your - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - (e.g. <filename>poky</filename>). - <note> - Depending on the recipe, certain security flags are enabled - and disabled by default. - </note> - </para> - - <para> -<!-- - The GCC/LD flags in <filename>security_flags.inc</filename> - enable more secure code generation. - By including the <filename>security_flags.inc</filename> - file, you enable flags to the compiler and linker that cause - them to generate more secure code. - <note> - The GCC/LD flags are enabled by default in the - <filename>poky-lsb</filename> distribution. - </note> ---> - Use the following line in your - <filename>local.conf</filename> file or in your custom - distribution configuration file to enable the security - compiler and linker flags for your build: - <literallayout class='monospaced'> - require conf/distro/include/security_flags.inc - </literallayout> - </para> - </section> - - <section id='considerations-specific-to-the-openembedded-build-system'> - <title>Considerations Specific to the OpenEmbedded Build System</title> - - <para> - You can take some steps that are specific to the - OpenEmbedded build system to make your images more secure: - <itemizedlist> - <listitem><para> - Ensure "debug-tweaks" is not one of your selected - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>. - When creating a new project, the default is to provide you - with an initial <filename>local.conf</filename> file that - enables this feature using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink> variable with the line: - <literallayout class='monospaced'> - EXTRA_IMAGE_FEATURES = "debug-tweaks" - </literallayout> - To disable that feature, simply comment out that line in your - <filename>local.conf</filename> file, or - make sure <filename>IMAGE_FEATURES</filename> does not contain - "debug-tweaks" before producing your final image. - Among other things, leaving this in place sets the - root password as blank, which makes logging in for - debugging or inspection easy during - development but also means anyone can easily log in - during production. - </para></listitem> - <listitem><para> - It is possible to set a root password for the image - and also to set passwords for any extra users you might - add (e.g. administrative or service type users). - When you set up passwords for multiple images or - users, you should not duplicate passwords. - </para> - <para> - To set up passwords, use the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-extrausers'><filename>extrausers</filename></ulink> - class, which is the preferred method. - For an example on how to set up both root and user - passwords, see the - "<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-extrausers'><filename>extrausers.bbclass</filename></ulink>" - section. - <note> - When adding extra user accounts or setting a - root password, be cautious about setting the - same password on every device. - If you do this, and the password you have set - is exposed, then every device is now potentially - compromised. - If you need this access but want to ensure - security, consider setting a different, - random password for each device. - Typically, you do this as a separate step after - you deploy the image onto the device. - </note> - </para></listitem> - <listitem><para> - Consider enabling a Mandatory Access Control (MAC) - framework such as SMACK or SELinux and tuning it - appropriately for your device's usage. - You can find more information in the - <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/meta-selinux/'><filename>meta-selinux</filename></ulink> - layer. - </para></listitem> - </itemizedlist> - </para> - - <para> - </para> - </section> - - <section id='tools-for-hardening-your-image'> - <title>Tools for Hardening Your Image</title> - - <para> - The Yocto Project provides tools for making your image - more secure. - You can find these tools in the - <filename>meta-security</filename> layer of the - <ulink url='&YOCTO_GIT_URL;'>Yocto Project Source Repositories</ulink>. - </para> - </section> - </section> - - <section id='creating-your-own-distribution'> - <title>Creating Your Own Distribution</title> - - <para> - When you build an image using the Yocto Project and - do not alter any distribution - <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink>, - you are creating a Poky distribution. - If you wish to gain more control over package alternative - selections, compile-time options, and other low-level - configurations, you can create your own distribution. - </para> - - <para> - To create your own distribution, the basic steps consist of - creating your own distribution layer, creating your own - distribution configuration file, and then adding any needed - code and Metadata to the layer. - The following steps provide some more detail: - <itemizedlist> - <listitem><para><emphasis>Create a layer for your new distro:</emphasis> - Create your distribution layer so that you can keep your - Metadata and code for the distribution separate. - It is strongly recommended that you create and use your own - layer for configuration and code. - Using your own layer as compared to just placing - configurations in a <filename>local.conf</filename> - configuration file makes it easier to reproduce the same - build configuration when using multiple build machines. - See the - "<link linkend='creating-a-general-layer-using-the-bitbake-layers-script'>Creating a General Layer Using the <filename>bitbake-layers</filename> Script</link>" - section for information on how to quickly set up a layer. - </para></listitem> - <listitem><para><emphasis>Create the distribution configuration file:</emphasis> - The distribution configuration file needs to be created in - the <filename>conf/distro</filename> directory of your - layer. - You need to name it using your distribution name - (e.g. <filename>mydistro.conf</filename>). - <note> - The - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> - variable in your - <filename>local.conf</filename> file determines the - name of your distribution. - </note></para> - <para>You can split out parts of your configuration file - into include files and then "require" them from within - your distribution configuration file. - Be sure to place the include files in the - <filename>conf/distro/include</filename> directory of - your layer. - A common example usage of include files would be to - separate out the selection of desired version and revisions - for individual recipes. -</para> - <para>Your configuration file needs to set the following - required variables: - <literallayout class='monospaced'> - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_NAME'><filename>DISTRO_NAME</filename></ulink> - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_VERSION'><filename>DISTRO_VERSION</filename></ulink> - </literallayout> - These following variables are optional and you typically - set them from the distribution configuration file: - <literallayout class='monospaced'> - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RDEPENDS'><filename>DISTRO_EXTRA_RDEPENDS</filename></ulink> - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_EXTRA_RRECOMMENDS'><filename>DISTRO_EXTRA_RRECOMMENDS</filename></ulink> - <ulink url='&YOCTO_DOCS_REF_URL;#var-TCLIBC'><filename>TCLIBC</filename></ulink> - </literallayout> - <tip> - If you want to base your distribution configuration file - on the very basic configuration from OE-Core, you - can use - <filename>conf/distro/defaultsetup.conf</filename> as - a reference and just include variables that differ - as compared to <filename>defaultsetup.conf</filename>. - Alternatively, you can create a distribution - configuration file from scratch using the - <filename>defaultsetup.conf</filename> file - or configuration files from other distributions - such as Poky or Angstrom as references. - </tip></para></listitem> - <listitem><para><emphasis>Provide miscellaneous variables:</emphasis> - Be sure to define any other variables for which you want to - create a default or enforce as part of the distribution - configuration. - You can include nearly any variable from the - <filename>local.conf</filename> file. - The variables you use are not limited to the list in the - previous bulleted item.</para></listitem> - <listitem><para><emphasis>Point to Your distribution configuration file:</emphasis> - In your <filename>local.conf</filename> file in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>, - set your - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO'><filename>DISTRO</filename></ulink> - variable to point to your distribution's configuration file. - For example, if your distribution's configuration file is - named <filename>mydistro.conf</filename>, then you point - to it as follows: - <literallayout class='monospaced'> - DISTRO = "mydistro" - </literallayout></para></listitem> - <listitem><para><emphasis>Add more to the layer if necessary:</emphasis> - Use your layer to hold other information needed for the - distribution: - <itemizedlist> - <listitem><para>Add recipes for installing - distro-specific configuration files that are not - already installed by another recipe. - If you have distro-specific configuration files - that are included by an existing recipe, you should - add an append file (<filename>.bbappend</filename>) - for those. - For general information and recommendations - on how to add recipes to your layer, see the - "<link linkend='creating-your-own-layer'>Creating Your Own Layer</link>" - and - "<link linkend='best-practices-to-follow-when-creating-layers'>Following Best Practices When Creating Layers</link>" - sections.</para></listitem> - <listitem><para>Add any image recipes that are specific - to your distribution.</para></listitem> - <listitem><para>Add a <filename>psplash</filename> - append file for a branded splash screen. - For information on append files, see the - "<link linkend='using-bbappend-files'>Using .bbappend Files in Your Layer</link>" - section.</para></listitem> - <listitem><para>Add any other append files to make - custom changes that are specific to individual - recipes.</para></listitem> - </itemizedlist></para></listitem> - </itemizedlist> - </para> - </section> - - <section id='creating-a-custom-template-configuration-directory'> - <title>Creating a Custom Template Configuration Directory</title> - - <para> - If you are producing your own customized version - of the build system for use by other users, you might - want to customize the message shown by the setup script or - you might want to change the template configuration files (i.e. - <filename>local.conf</filename> and - <filename>bblayers.conf</filename>) that are created in - a new build directory. - </para> - - <para> - The OpenEmbedded build system uses the environment variable - <filename>TEMPLATECONF</filename> to locate the directory - from which it gathers configuration information that ultimately - ends up in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - <filename>conf</filename> directory. - By default, <filename>TEMPLATECONF</filename> is set as - follows in the <filename>poky</filename> repository: - <literallayout class='monospaced'> - TEMPLATECONF=${TEMPLATECONF:-meta-poky/conf} - </literallayout> - This is the directory used by the build system to find templates - from which to build some key configuration files. - If you look at this directory, you will see the - <filename>bblayers.conf.sample</filename>, - <filename>local.conf.sample</filename>, and - <filename>conf-notes.txt</filename> files. - The build system uses these files to form the respective - <filename>bblayers.conf</filename> file, - <filename>local.conf</filename> file, and display the list of - BitBake targets when running the setup script. - </para> - - <para> - To override these default configuration files with - configurations you want used within every new - Build Directory, simply set the - <filename>TEMPLATECONF</filename> variable to your directory. - The <filename>TEMPLATECONF</filename> variable is set in the - <filename>.templateconf</filename> file, which is in the - top-level - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - folder (e.g. <filename>poky</filename>). - Edit the <filename>.templateconf</filename> so that it can locate - your directory. - </para> - - <para> - Best practices dictate that you should keep your - template configuration directory in your custom distribution layer. - For example, suppose you have a layer named - <filename>meta-mylayer</filename> located in your home directory - and you want your template configuration directory named - <filename>myconf</filename>. - Changing the <filename>.templateconf</filename> as follows - causes the OpenEmbedded build system to look in your directory - and base its configuration files on the - <filename>*.sample</filename> configuration files it finds. - The final configuration files (i.e. - <filename>local.conf</filename> and - <filename>bblayers.conf</filename> ultimately still end up in - your Build Directory, but they are based on your - <filename>*.sample</filename> files. - <literallayout class='monospaced'> - TEMPLATECONF=${TEMPLATECONF:-meta-mylayer/myconf} - </literallayout> - </para> - - <para> - Aside from the <filename>*.sample</filename> configuration files, - the <filename>conf-notes.txt</filename> also resides in the - default <filename>meta-poky/conf</filename> directory. - The script that sets up the build environment - (i.e. - <ulink url="&YOCTO_DOCS_REF_URL;#structure-core-script"><filename>&OE_INIT_FILE;</filename></ulink>) - uses this file to display BitBake targets as part of the script - output. - Customizing this <filename>conf-notes.txt</filename> file is a - good way to make sure your list of custom targets appears - as part of the script's output. - </para> - - <para> - Here is the default list of targets displayed as a result of - running either of the setup scripts: - <literallayout class='monospaced'> - You can now run 'bitbake <target>' - - Common targets are: - core-image-minimal - core-image-sato - meta-toolchain - meta-ide-support - </literallayout> - </para> - - <para> - Changing the listed common targets is as easy as editing your - version of <filename>conf-notes.txt</filename> in your - custom template configuration directory and making sure you - have <filename>TEMPLATECONF</filename> set to your directory. - </para> - </section> - - <section id='dev-saving-memory-during-a-build'> - <title>Conserving Disk Space During Builds</title> - - <para> - To help conserve disk space during builds, you can add the - following statement to your project's - <filename>local.conf</filename> configuration file found in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>: - <literallayout class='monospaced'> - INHERIT += "rm_work" - </literallayout> - Adding this statement deletes the work directory used for building - a recipe once the recipe is built. - For more information on "rm_work", see the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-rm-work'><filename>rm_work</filename></ulink> - class in the Yocto Project Reference Manual. - </para> - </section> - - <section id='working-with-packages'> - <title>Working with Packages</title> - - <para> - This section describes a few tasks that involve packages: - <itemizedlist> - <listitem><para> - <link linkend='excluding-packages-from-an-image'>Excluding packages from an image</link> - </para></listitem> - <listitem><para> - <link linkend='incrementing-a-binary-package-version'>Incrementing a binary package version</link> - </para></listitem> - <listitem><para> - <link linkend='handling-optional-module-packaging'>Handling optional module packaging</link> - </para></listitem> - <listitem><para> - <link linkend='using-runtime-package-management'>Using runtime package management</link> - </para></listitem> - <listitem><para> - <link linkend='generating-and-using-signed-packages'>Generating and using signed packages</link> - </para></listitem> - <listitem><para> - <link linkend='testing-packages-with-ptest'>Setting up and running package test (ptest)</link> - </para></listitem> - <listitem><para> - <link linkend='creating-node-package-manager-npm-packages'>Creating node package manager (NPM) packages</link> - </para></listitem> - <listitem><para> - <link linkend='adding-custom-metadata-to-packages'>Adding custom metadata to packages</link> - </para></listitem> - </itemizedlist> - </para> - - <section id='excluding-packages-from-an-image'> - <title>Excluding Packages from an Image</title> - - <para> - You might find it necessary to prevent specific packages - from being installed into an image. - If so, you can use several variables to direct the build - system to essentially ignore installing recommended packages - or to not install a package at all. - </para> - - <para> - The following list introduces variables you can use to - prevent packages from being installed into your image. - Each of these variables only works with IPK and RPM - package types. - Support for Debian packages does not exist. - Also, you can use these variables from your - <filename>local.conf</filename> file or attach them to a - specific image recipe by using a recipe name override. - For more detail on the variables, see the descriptions in the - Yocto Project Reference Manual's glossary chapter. - <itemizedlist> - <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-BAD_RECOMMENDATIONS'><filename>BAD_RECOMMENDATIONS</filename></ulink>: - Use this variable to specify "recommended-only" - packages that you do not want installed. - </para></listitem> - <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-NO_RECOMMENDATIONS'><filename>NO_RECOMMENDATIONS</filename></ulink>: - Use this variable to prevent all "recommended-only" - packages from being installed. - </para></listitem> - <listitem><para><ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_EXCLUDE'><filename>PACKAGE_EXCLUDE</filename></ulink>: - Use this variable to prevent specific packages from - being installed regardless of whether they are - "recommended-only" or not. - You need to realize that the build process could - fail with an error when you - prevent the installation of a package whose presence - is required by an installed package. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='incrementing-a-binary-package-version'> - <title>Incrementing a Package Version</title> - - <para> - This section provides some background on how binary package - versioning is accomplished and presents some of the services, - variables, and terminology involved. - </para> - - <para> - In order to understand binary package versioning, you need - to consider the following: - <itemizedlist> - <listitem><para> - Binary Package: The binary package that is eventually - built and installed into an image. - </para></listitem> - <listitem><para> - Binary Package Version: The binary package version - is composed of two components - a version and a - revision. - <note> - Technically, a third component, the "epoch" (i.e. - <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>) - is involved but this discussion for the most part - ignores <filename>PE</filename>. - </note> - The version and revision are taken from the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> - variables, respectively. - </para></listitem> - <listitem><para> - <filename>PV</filename>: The recipe version. - <filename>PV</filename> represents the version of the - software being packaged. - Do not confuse <filename>PV</filename> with the - binary package version. - </para></listitem> - <listitem><para> - <filename>PR</filename>: The recipe revision. - </para></listitem> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>: - The OpenEmbedded build system uses this string - to help define the value of <filename>PV</filename> - when the source code revision needs to be included - in it. - </para></listitem> - <listitem><para> - <ulink url='https://wiki.yoctoproject.org/wiki/PR_Service'>PR Service</ulink>: - A network-based service that helps automate keeping - package feeds compatible with existing package - manager applications such as RPM, APT, and OPKG. - </para></listitem> - </itemizedlist> - </para> - - <para> - Whenever the binary package content changes, the binary package - version must change. - Changing the binary package version is accomplished by changing - or "bumping" the <filename>PR</filename> and/or - <filename>PV</filename> values. - Increasing these values occurs one of two ways: - <itemizedlist> - <listitem><para>Automatically using a Package Revision - Service (PR Service). - </para></listitem> - <listitem><para>Manually incrementing the - <filename>PR</filename> and/or - <filename>PV</filename> variables. - </para></listitem> - </itemizedlist> - </para> - - <para> - Given a primary challenge of any build system and its users - is how to maintain a package feed that is compatible with - existing package manager applications such as RPM, APT, and - OPKG, using an automated system is much preferred over a - manual system. - In either system, the main requirement is that binary package - version numbering increases in a linear fashion and that a - number of version components exist that support that linear - progression. - For information on how to ensure package revisioning remains - linear, see the - "<link linkend='automatically-incrementing-a-binary-package-revision-number'>Automatically Incrementing a Binary Package Revision Number</link>" - section. - </para> - - <para> - The following three sections provide related information on the - PR Service, the manual method for "bumping" - <filename>PR</filename> and/or <filename>PV</filename>, and - on how to ensure binary package revisioning remains linear. - </para> - - <section id='working-with-a-pr-service'> - <title>Working With a PR Service</title> - - <para> - As mentioned, attempting to maintain revision numbers in the - <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink> - is error prone, inaccurate, and causes problems for people - submitting recipes. - Conversely, the PR Service automatically generates - increasing numbers, particularly the revision field, - which removes the human element. - <note> - For additional information on using a PR Service, you - can see the - <ulink url='&YOCTO_WIKI_URL;/wiki/PR_Service'>PR Service</ulink> - wiki page. - </note> - </para> - - <para> - The Yocto Project uses variables in order of - decreasing priority to facilitate revision numbering (i.e. - <ulink url='&YOCTO_DOCS_REF_URL;#var-PE'><filename>PE</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink>, and - <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> - for epoch, version, and revision, respectively). - The values are highly dependent on the policies and - procedures of a given distribution and package feed. - </para> - - <para> - Because the OpenEmbedded build system uses - "<ulink url='&YOCTO_DOCS_OM_URL;#overview-checksums'>signatures</ulink>", - which are unique to a given build, the build system - knows when to rebuild packages. - All the inputs into a given task are represented by a - signature, which can trigger a rebuild when different. - Thus, the build system itself does not rely on the - <filename>PR</filename>, <filename>PV</filename>, and - <filename>PE</filename> numbers to trigger a rebuild. - The signatures, however, can be used to generate - these values. - </para> - - <para> - The PR Service works with both - <filename>OEBasic</filename> and - <filename>OEBasicHash</filename> generators. - The value of <filename>PR</filename> bumps when the - checksum changes and the different generator mechanisms - change signatures under different circumstances. - </para> - - <para> - As implemented, the build system includes values from - the PR Service into the <filename>PR</filename> field as - an addition using the form "<filename>.x</filename>" so - <filename>r0</filename> becomes <filename>r0.1</filename>, - <filename>r0.2</filename> and so forth. - This scheme allows existing <filename>PR</filename> values - to be used for whatever reasons, which include manual - <filename>PR</filename> bumps, should it be necessary. - </para> - - <para> - By default, the PR Service is not enabled or running. - Thus, the packages generated are just "self consistent". - The build system adds and removes packages and - there are no guarantees about upgrade paths but images - will be consistent and correct with the latest changes. - </para> - - <para> - The simplest form for a PR Service is for it to exist - for a single host development system that builds the - package feed (building system). - For this scenario, you can enable a local PR Service by - setting - <ulink url='&YOCTO_DOCS_REF_URL;#var-PRSERV_HOST'><filename>PRSERV_HOST</filename></ulink> - in your <filename>local.conf</filename> file in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>: - <literallayout class='monospaced'> - PRSERV_HOST = "localhost:0" - </literallayout> - Once the service is started, packages will automatically - get increasing <filename>PR</filename> values and - BitBake takes care of starting and stopping the server. - </para> - - <para> - If you have a more complex setup where multiple host - development systems work against a common, shared package - feed, you have a single PR Service running and it is - connected to each building system. - For this scenario, you need to start the PR Service using - the <filename>bitbake-prserv</filename> command: - <literallayout class='monospaced'> - bitbake-prserv --host <replaceable>ip</replaceable> --port <replaceable>port</replaceable> --start - </literallayout> - In addition to hand-starting the service, you need to - update the <filename>local.conf</filename> file of each - building system as described earlier so each system - points to the server and port. - </para> - - <para> - It is also recommended you use build history, which adds - some sanity checks to binary package versions, in - conjunction with the server that is running the PR Service. - To enable build history, add the following to each building - system's <filename>local.conf</filename> file: - <literallayout class='monospaced'> - # It is recommended to activate "buildhistory" for testing the PR service - INHERIT += "buildhistory" - BUILDHISTORY_COMMIT = "1" - </literallayout> - For information on build history, see the - "<link linkend='maintaining-build-output-quality'>Maintaining Build Output Quality</link>" - section. - </para> - - <note> - <para> - The OpenEmbedded build system does not maintain - <filename>PR</filename> information as part of the - shared state (sstate) packages. - If you maintain an sstate feed, its expected that either - all your building systems that contribute to the sstate - feed use a shared PR Service, or you do not run a PR - Service on any of your building systems. - Having some systems use a PR Service while others do - not leads to obvious problems. - </para> - - <para> - For more information on shared state, see the - "<ulink url='&YOCTO_DOCS_OM_URL;#shared-state-cache'>Shared State Cache</ulink>" - section in the Yocto Project Overview and Concepts - Manual. - </para> - </note> - </section> - - <section id='manually-bumping-pr'> - <title>Manually Bumping PR</title> - - <para> - The alternative to setting up a PR Service is to manually - "bump" the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink> - variable. - </para> - - <para> - If a committed change results in changing the package - output, then the value of the PR variable needs to be - increased (or "bumped") as part of that commit. - For new recipes you should add the <filename>PR</filename> - variable and set its initial value equal to "r0", which is - the default. - Even though the default value is "r0", the practice of - adding it to a new recipe makes it harder to forget to bump - the variable when you make changes to the recipe in future. - </para> - - <para> - If you are sharing a common <filename>.inc</filename> file - with multiple recipes, you can also use the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-INC_PR'>INC_PR</ulink></filename> - variable to ensure that the recipes sharing the - <filename>.inc</filename> file are rebuilt when the - <filename>.inc</filename> file itself is changed. - The <filename>.inc</filename> file must set - <filename>INC_PR</filename> (initially to "r0"), and all - recipes referring to it should set <filename>PR</filename> - to "${INC_PR}.0" initially, incrementing the last number - when the recipe is changed. - If the <filename>.inc</filename> file is changed then its - <filename>INC_PR</filename> should be incremented. - </para> - - <para> - When upgrading the version of a binary package, assuming the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PV'>PV</ulink></filename> - changes, the <filename>PR</filename> variable should be - reset to "r0" (or "${INC_PR}.0" if you are using - <filename>INC_PR</filename>). - </para> - - <para> - Usually, version increases occur only to binary packages. - However, if for some reason <filename>PV</filename> changes - but does not increase, you can increase the - <filename><ulink url='&YOCTO_DOCS_REF_URL;#var-PE'>PE</ulink></filename> - variable (Package Epoch). - The <filename>PE</filename> variable defaults to "0". - </para> - - <para> - Binary package version numbering strives to follow the - <ulink url='http://www.debian.org/doc/debian-policy/ch-controlfields.html'> - Debian Version Field Policy Guidelines</ulink>. - These guidelines define how versions are compared and what - "increasing" a version means. - </para> - </section> - - <section id='automatically-incrementing-a-binary-package-revision-number'> - <title>Automatically Incrementing a Package Version Number</title> - - <para> - When fetching a repository, BitBake uses the - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink> - variable to determine the specific source code revision - from which to build. - You set the <filename>SRCREV</filename> variable to - <ulink url='&YOCTO_DOCS_REF_URL;#var-AUTOREV'><filename>AUTOREV</filename></ulink> - to cause the OpenEmbedded build system to automatically use the - latest revision of the software: - <literallayout class='monospaced'> - SRCREV = "${AUTOREV}" - </literallayout> - </para> - - <para> - Furthermore, you need to reference <filename>SRCPV</filename> - in <filename>PV</filename> in order to automatically update - the version whenever the revision of the source code - changes. - Here is an example: - <literallayout class='monospaced'> - PV = "1.0+git${SRCPV}" - </literallayout> - The OpenEmbedded build system substitutes - <filename>SRCPV</filename> with the following: - <literallayout class='monospaced'> - AUTOINC+<replaceable>source_code_revision</replaceable> - </literallayout> - The build system replaces the <filename>AUTOINC</filename> with - a number. - The number used depends on the state of the PR Service: - <itemizedlist> - <listitem><para> - If PR Service is enabled, the build system increments - the number, which is similar to the behavior of - <ulink url='&YOCTO_DOCS_REF_URL;#var-PR'><filename>PR</filename></ulink>. - This behavior results in linearly increasing package - versions, which is desirable. - Here is an example: - <literallayout class='monospaced'> - hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk - hello-world-git_0.0+git1+dd2f5c3565-r0.0_armv7a-neon.ipk - </literallayout> - </para></listitem> - <listitem><para> - If PR Service is not enabled, the build system - replaces the <filename>AUTOINC</filename> - placeholder with zero (i.e. "0"). - This results in changing the package version since - the source revision is included. - However, package versions are not increased linearly. - Here is an example: - <literallayout class='monospaced'> - hello-world-git_0.0+git0+b6558dd387-r0.0_armv7a-neon.ipk - hello-world-git_0.0+git0+dd2f5c3565-r0.0_armv7a-neon.ipk - </literallayout> - </para></listitem> - </itemizedlist> - </para> - - <para> - In summary, the OpenEmbedded build system does not track the - history of binary package versions for this purpose. - <filename>AUTOINC</filename>, in this case, is comparable to - <filename>PR</filename>. - If PR server is not enabled, <filename>AUTOINC</filename> - in the package version is simply replaced by "0". - If PR server is enabled, the build system keeps track of the - package versions and bumps the number when the package - revision changes. - </para> - </section> - </section> - - <section id='handling-optional-module-packaging'> - <title>Handling Optional Module Packaging</title> - - <para> - Many pieces of software split functionality into optional - modules (or plugins) and the plugins that are built - might depend on configuration options. - To avoid having to duplicate the logic that determines what - modules are available in your recipe or to avoid having - to package each module by hand, the OpenEmbedded build system - provides functionality to handle module packaging dynamically. - </para> - - <para> - To handle optional module packaging, you need to do two things: - <itemizedlist> - <listitem><para>Ensure the module packaging is actually - done.</para></listitem> - <listitem><para>Ensure that any dependencies on optional - modules from other recipes are satisfied by your recipe. - </para></listitem> - </itemizedlist> - </para> - - <section id='making-sure-the-packaging-is-done'> - <title>Making Sure the Packaging is Done</title> - - <para> - To ensure the module packaging actually gets done, you use - the <filename>do_split_packages</filename> function within - the <filename>populate_packages</filename> Python function - in your recipe. - The <filename>do_split_packages</filename> function - searches for a pattern of files or directories under a - specified path and creates a package for each one it finds - by appending to the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES'><filename>PACKAGES</filename></ulink> - variable and setting the appropriate values for - <filename>FILES_packagename</filename>, - <filename>RDEPENDS_packagename</filename>, - <filename>DESCRIPTION_packagename</filename>, and so forth. - Here is an example from the <filename>lighttpd</filename> - recipe: - <literallayout class='monospaced'> - python populate_packages_prepend () { - lighttpd_libdir = d.expand('${libdir}') - do_split_packages(d, lighttpd_libdir, '^mod_(.*)\.so$', - 'lighttpd-module-%s', 'Lighttpd module for %s', - extra_depends='') - } - </literallayout> - The previous example specifies a number of things in the - call to <filename>do_split_packages</filename>. - <itemizedlist> - <listitem><para>A directory within the files installed - by your recipe through <filename>do_install</filename> - in which to search.</para></listitem> - <listitem><para>A regular expression used to match module - files in that directory. - In the example, note the parentheses () that mark - the part of the expression from which the module - name should be derived.</para></listitem> - <listitem><para>A pattern to use for the package names. - </para></listitem> - <listitem><para>A description for each package. - </para></listitem> - <listitem><para>An empty string for - <filename>extra_depends</filename>, which disables - the default dependency on the main - <filename>lighttpd</filename> package. - Thus, if a file in <filename>${libdir}</filename> - called <filename>mod_alias.so</filename> is found, - a package called <filename>lighttpd-module-alias</filename> - is created for it and the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DESCRIPTION'><filename>DESCRIPTION</filename></ulink> - is set to "Lighttpd module for alias".</para></listitem> - </itemizedlist> - </para> - - <para> - Often, packaging modules is as simple as the previous - example. - However, more advanced options exist that you can use - within <filename>do_split_packages</filename> to modify its - behavior. - And, if you need to, you can add more logic by specifying - a hook function that is called for each package. - It is also perfectly acceptable to call - <filename>do_split_packages</filename> multiple times if - you have more than one set of modules to package. - </para> - - <para> - For more examples that show how to use - <filename>do_split_packages</filename>, see the - <filename>connman.inc</filename> file in the - <filename>meta/recipes-connectivity/connman/</filename> - directory of the <filename>poky</filename> - <ulink url='&YOCTO_DOCS_OM_URL;#yocto-project-repositories'>source repository</ulink>. - You can also find examples in - <filename>meta/classes/kernel.bbclass</filename>. - </para> - - <para> - Following is a reference that shows - <filename>do_split_packages</filename> mandatory and - optional arguments: - <literallayout class='monospaced'> - Mandatory arguments - - root - The path in which to search - file_regex - Regular expression to match searched files. - Use parentheses () to mark the part of this - expression that should be used to derive the - module name (to be substituted where %s is - used in other function arguments as noted below) - output_pattern - Pattern to use for the package names. Must - include %s. - description - Description to set for each package. Must - include %s. - - Optional arguments - - postinst - Postinstall script to use for all packages - (as a string) - recursive - True to perform a recursive search - default - False - hook - A hook function to be called for every match. - The function will be called with the following - arguments (in the order listed): - - f - Full path to the file/directory match - pkg - The package name - file_regex - As above - output_pattern - As above - modulename - The module name derived using file_regex - - extra_depends - Extra runtime dependencies (RDEPENDS) to be - set for all packages. The default value of None - causes a dependency on the main package - (${PN}) - if you do not want this, pass empty - string '' for this parameter. - aux_files_pattern - Extra item(s) to be added to FILES for each - package. Can be a single string item or a list - of strings for multiple items. Must include %s. - postrm - postrm script to use for all packages (as a - string) - allow_dirs - True to allow directories to be matched - - default False - prepend - If True, prepend created packages to PACKAGES - instead of the default False which appends them - match_path - match file_regex on the whole relative path to - the root rather than just the file name - aux_files_pattern_verbatim - Extra item(s) to be added to FILES for each - package, using the actual derived module name - rather than converting it to something legal - for a package name. Can be a single string item - or a list of strings for multiple items. Must - include %s. - allow_links - True to allow symlinks to be matched - default - False - summary - Summary to set for each package. Must include %s; - defaults to description if not set. - </literallayout> - </para> - </section> - - <section id='satisfying-dependencies'> - <title>Satisfying Dependencies</title> - - <para> - The second part for handling optional module packaging - is to ensure that any dependencies on optional modules - from other recipes are satisfied by your recipe. - You can be sure these dependencies are satisfied by - using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGES_DYNAMIC'><filename>PACKAGES_DYNAMIC</filename></ulink> variable. - Here is an example that continues with the - <filename>lighttpd</filename> recipe shown earlier: - <literallayout class='monospaced'> - PACKAGES_DYNAMIC = "lighttpd-module-.*" - </literallayout> - The name specified in the regular expression can of - course be anything. - In this example, it is <filename>lighttpd-module-</filename> - and is specified as the prefix to ensure that any - <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink> - and <ulink url='&YOCTO_DOCS_REF_URL;#var-RRECOMMENDS'><filename>RRECOMMENDS</filename></ulink> - on a package name starting with the prefix are satisfied - during build time. - If you are using <filename>do_split_packages</filename> - as described in the previous section, the value you put in - <filename>PACKAGES_DYNAMIC</filename> should correspond to - the name pattern specified in the call to - <filename>do_split_packages</filename>. - </para> - </section> - </section> - - <section id='using-runtime-package-management'> - <title>Using Runtime Package Management</title> - - <para> - During a build, BitBake always transforms a recipe into one or - more packages. - For example, BitBake takes the <filename>bash</filename> recipe - and produces a number of packages (e.g. - <filename>bash</filename>, <filename>bash-bashbug</filename>, - <filename>bash-completion</filename>, - <filename>bash-completion-dbg</filename>, - <filename>bash-completion-dev</filename>, - <filename>bash-completion-extra</filename>, - <filename>bash-dbg</filename>, and so forth). - Not all generated packages are included in an image. - </para> - - <para> - In several situations, you might need to update, add, remove, - or query the packages on a target device at runtime - (i.e. without having to generate a new image). - Examples of such situations include: - <itemizedlist> - <listitem><para> - You want to provide in-the-field updates to deployed - devices (e.g. security updates). - </para></listitem> - <listitem><para> - You want to have a fast turn-around development cycle - for one or more applications that run on your device. - </para></listitem> - <listitem><para> - You want to temporarily install the "debug" packages - of various applications on your device so that - debugging can be greatly improved by allowing - access to symbols and source debugging. - </para></listitem> - <listitem><para> - You want to deploy a more minimal package selection of - your device but allow in-the-field updates to add a - larger selection for customization. - </para></listitem> - </itemizedlist> - </para> - - <para> - In all these situations, you have something similar to a more - traditional Linux distribution in that in-field devices - are able to receive pre-compiled packages from a server for - installation or update. - Being able to install these packages on a running, - in-field device is what is termed "runtime package - management". - </para> - - <para> - In order to use runtime package management, you - need a host or server machine that serves up the pre-compiled - packages plus the required metadata. - You also need package manipulation tools on the target. - The build machine is a likely candidate to act as the server. - However, that machine does not necessarily have to be the - package server. - The build machine could push its artifacts to another machine - that acts as the server (e.g. Internet-facing). - In fact, doing so is advantageous for a production - environment as getting the packages away from the - development system's build directory prevents accidental - overwrites. - </para> - - <para> - A simple build that targets just one device produces - more than one package database. - In other words, the packages produced by a build are separated - out into a couple of different package groupings based on - criteria such as the target's CPU architecture, the target - board, or the C library used on the target. - For example, a build targeting the <filename>qemux86</filename> - device produces the following three package databases: - <filename>noarch</filename>, <filename>i586</filename>, and - <filename>qemux86</filename>. - If you wanted your <filename>qemux86</filename> device to be - aware of all the packages that were available to it, - you would need to point it to each of these databases - individually. - In a similar way, a traditional Linux distribution usually is - configured to be aware of a number of software repositories - from which it retrieves packages. - </para> - - <para> - Using runtime package management is completely optional and - not required for a successful build or deployment in any - way. - But if you want to make use of runtime package management, - you need to do a couple things above and beyond the basics. - The remainder of this section describes what you need to do. - </para> - - <section id='runtime-package-management-build'> - <title>Build Considerations</title> - - <para> - This section describes build considerations of which you - need to be aware in order to provide support for runtime - package management. - </para> - - <para> - When BitBake generates packages, it needs to know - what format or formats to use. - In your configuration, you use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink> - variable to specify the format: - <orderedlist> - <listitem><para> - Open the <filename>local.conf</filename> file - inside your - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - (e.g. <filename>~/poky/build/conf/local.conf</filename>). - </para></listitem> - <listitem><para> - Select the desired package format as follows: - <literallayout class='monospaced'> - PACKAGE_CLASSES ?= “package_<replaceable>packageformat</replaceable>” - </literallayout> - where <replaceable>packageformat</replaceable> - can be "ipk", "rpm", "deb", or "tar" which are the - supported package formats. - <note> - Because the Yocto Project supports four - different package formats, you can set the - variable with more than one argument. - However, the OpenEmbedded build system only - uses the first argument when creating an image - or Software Development Kit (SDK). - </note> - </para></listitem> - </orderedlist> - </para> - - <para> - If you would like your image to start off with a basic - package database containing the packages in your current - build as well as to have the relevant tools available on the - target for runtime package management, you can include - "package-management" in the - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> - variable. - Including "package-management" in this configuration - variable ensures that when the image is assembled for your - target, the image includes the currently-known package - databases as well as the target-specific tools required - for runtime package management to be performed on the - target. - However, this is not strictly necessary. - You could start your image off without any databases - but only include the required on-target package - tool(s). - As an example, you could include "opkg" in your - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink> - variable if you are using the IPK package format. - You can then initialize your target's package database(s) - later once your image is up and running. - </para> - - <para> - Whenever you perform any sort of build step that can - potentially generate a package or modify existing - package, it is always a good idea to re-generate the - package index after the build by using the following - command: - <literallayout class='monospaced'> - $ bitbake package-index - </literallayout> - It might be tempting to build the package and the - package index at the same time with a command such as - the following: - <literallayout class='monospaced'> - $ bitbake <replaceable>some-package</replaceable> package-index - </literallayout> - Do not do this as BitBake does not schedule the package - index for after the completion of the package you are - building. - Consequently, you cannot be sure of the package index - including information for the package you just built. - Thus, be sure to run the package update step separately - after building any packages. - </para> - - <para> - You can use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>, - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink> - variables to pre-configure target images to use a package - feed. - If you do not define these variables, then manual steps - as described in the subsequent sections are necessary to - configure the target. - You should set these variables before building the image - in order to produce a correctly configured image. - </para> - - <para> - When your build is complete, your packages reside in the - <filename>${TMPDIR}/deploy/<replaceable>packageformat</replaceable></filename> - directory. - For example, if - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink><filename>}</filename> - is <filename>tmp</filename> and your selected package type - is RPM, then your RPM packages are available in - <filename>tmp/deploy/rpm</filename>. - </para> - </section> - - <section id='runtime-package-management-server'> - <title>Host or Server Machine Setup</title> - - <para> - Although other protocols are possible, a server using HTTP - typically serves packages. - If you want to use HTTP, then set up and configure a - web server such as Apache 2, lighttpd, or - SimpleHTTPServer on the machine serving the packages. - </para> - - <para> - To keep things simple, this section describes how to set - up a SimpleHTTPServer web server to share package feeds - from the developer's machine. - Although this server might not be the best for a production - environment, the setup is simple and straight forward. - Should you want to use a different server more suited for - production (e.g. Apache 2, Lighttpd, or Nginx), take the - appropriate steps to do so. - </para> - - <para> - From within the build directory where you have built an - image based on your packaging choice (i.e. the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_CLASSES'><filename>PACKAGE_CLASSES</filename></ulink> - setting), simply start the server. - The following example assumes a build directory of - <filename>~/poky/build/tmp/deploy/rpm</filename> and a - <filename>PACKAGE_CLASSES</filename> setting of - "package_rpm": - <literallayout class='monospaced'> - $ cd ~/poky/build/tmp/deploy/rpm - $ python -m SimpleHTTPServer - </literallayout> - </para> - </section> - - <section id='runtime-package-management-target'> - <title>Target Setup</title> - - <para> - Setting up the target differs depending on the - package management system. - This section provides information for RPM, IPK, and DEB. - </para> - - <section id='runtime-package-management-target-rpm'> - <title>Using RPM</title> - - <para> - The - <ulink url='https://en.wikipedia.org/wiki/DNF_(software)'>Dandified Packaging Tool</ulink> - (DNF) performs runtime package management of RPM - packages. - In order to use DNF for runtime package management, - you must perform an initial setup on the target - machine for cases where the - <filename>PACKAGE_FEED_*</filename> variables were not - set as part of the image that is running on the - target. - This means if you built your image and did not not use - these variables as part of the build and your image is - now running on the target, you need to perform the - steps in this section if you want to use runtime - package management. - <note> - For information on the - <filename>PACKAGE_FEED_*</filename> variables, see - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>, - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink> - in the Yocto Project Reference Manual variables - glossary. - </note> - </para> - - <para> - On the target, you must inform DNF that package - databases are available. - You do this by creating a file named - <filename>/etc/yum.repos.d/oe-packages.repo</filename> - and defining the <filename>oe-packages</filename>. - </para> - - <para> - As an example, assume the target is able to use the - following package databases: - <filename>all</filename>, <filename>i586</filename>, - and <filename>qemux86</filename> from a server named - <filename>my.server</filename>. - The specifics for setting up the web server are up to - you. - The critical requirement is that the URIs in the - target repository configuration point to the - correct remote location for the feeds. - <note><title>Tip</title> - For development purposes, you can point the web - server to the build system's - <filename>deploy</filename> directory. - However, for production use, it is better to copy - the package directories to a location outside of - the build area and use that location. - Doing so avoids situations where the build system - overwrites or changes the - <filename>deploy</filename> directory. - </note> - </para> - - <para> - When telling DNF where to look for the package - databases, you must declare individual locations - per architecture or a single location used for all - architectures. - You cannot do both: - <itemizedlist> - <listitem><para> - <emphasis>Create an Explicit List of Architectures:</emphasis> - Define individual base URLs to identify where - each package database is located: - <literallayout class='monospaced'> - [oe-packages] - baseurl=http://my.server/rpm/i586 http://my.server/rpm/qemux86 http://my.server/rpm/all - </literallayout> - This example informs DNF about individual - package databases for all three architectures. - </para></listitem> - <listitem><para> - <emphasis>Create a Single (Full) Package Index:</emphasis> - Define a single base URL that identifies where - a full package database is located: - <literallayout class='monospaced'> - [oe-packages] - baseurl=http://my.server/rpm - </literallayout> - This example informs DNF about a single package - database that contains all the package index - information for all supported architectures. - </para></listitem> - </itemizedlist> - </para> - - <para> - Once you have informed DNF where to find the package - databases, you need to fetch them: - <literallayout class='monospaced'> - # dnf makecache - </literallayout> - DNF is now able to find, install, and upgrade packages - from the specified repository or repositories. - <note> - See the - <ulink url='http://dnf.readthedocs.io/en/latest/'>DNF documentation</ulink> - for additional information. - </note> - </para> - </section> - - <section id='runtime-package-management-target-ipk'> - <title>Using IPK</title> - - <para> - The <filename>opkg</filename> application performs - runtime package management of IPK packages. - You must perform an initial setup for - <filename>opkg</filename> on the target machine - if the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>, and - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink> - variables have not been set or the target image was - built before the variables were set. - </para> - - <para> - The <filename>opkg</filename> application uses - configuration files to find available package - databases. - Thus, you need to create a configuration file inside - the <filename>/etc/opkg/</filename> direction, which - informs <filename>opkg</filename> of any repository - you want to use. - </para> - - <para> - As an example, suppose you are serving packages from a - <filename>ipk/</filename> directory containing the - <filename>i586</filename>, - <filename>all</filename>, and - <filename>qemux86</filename> databases through an - HTTP server named <filename>my.server</filename>. - On the target, create a configuration file - (e.g. <filename>my_repo.conf</filename>) inside the - <filename>/etc/opkg/</filename> directory containing - the following: - <literallayout class='monospaced'> - src/gz all http://my.server/ipk/all - src/gz i586 http://my.server/ipk/i586 - src/gz qemux86 http://my.server/ipk/qemux86 - </literallayout> - Next, instruct <filename>opkg</filename> to fetch - the repository information: - <literallayout class='monospaced'> - # opkg update - </literallayout> - The <filename>opkg</filename> application is now able - to find, install, and upgrade packages from the - specified repository. - </para> - </section> - - <section id='runtime-package-management-target-deb'> - <title>Using DEB</title> - - <para> - The <filename>apt</filename> application performs - runtime package management of DEB packages. - This application uses a source list file to find - available package databases. - You must perform an initial setup for - <filename>apt</filename> on the target machine - if the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_ARCHS'><filename>PACKAGE_FEED_ARCHS</filename></ulink>, - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_BASE_PATHS'><filename>PACKAGE_FEED_BASE_PATHS</filename></ulink>, and - <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_FEED_URIS'><filename>PACKAGE_FEED_URIS</filename></ulink> - variables have not been set or the target image was - built before the variables were set. - </para> - - <para> - To inform <filename>apt</filename> of the repository - you want to use, you might create a list file (e.g. - <filename>my_repo.list</filename>) inside the - <filename>/etc/apt/sources.list.d/</filename> - directory. - As an example, suppose you are serving packages from a - <filename>deb/</filename> directory containing the - <filename>i586</filename>, - <filename>all</filename>, and - <filename>qemux86</filename> databases through an - HTTP server named <filename>my.server</filename>. - The list file should contain: - <literallayout class='monospaced'> - deb http://my.server/deb/all ./ - deb http://my.server/deb/i586 ./ - deb http://my.server/deb/qemux86 ./ - </literallayout> - Next, instruct the <filename>apt</filename> - application to fetch the repository information: - <literallayout class='monospaced'> - # apt-get update - </literallayout> - After this step, <filename>apt</filename> is able - to find, install, and upgrade packages from the - specified repository. - </para> - </section> - </section> - </section> - - <section id='generating-and-using-signed-packages'> - <title>Generating and Using Signed Packages</title> - <para> - In order to add security to RPM packages used during a build, - you can take steps to securely sign them. - Once a signature is verified, the OpenEmbedded build system - can use the package in the build. - If security fails for a signed package, the build system - aborts the build. - </para> - - <para> - This section describes how to sign RPM packages during a build - and how to use signed package feeds (repositories) when - doing a build. - </para> - - <section id='signing-rpm-packages'> - <title>Signing RPM Packages</title> - - <para> - To enable signing RPM packages, you must set up the - following configurations in either your - <filename>local.config</filename> or - <filename>distro.config</filename> file: - <literallayout class='monospaced'> - # Inherit sign_rpm.bbclass to enable signing functionality - INHERIT += " sign_rpm" - # Define the GPG key that will be used for signing. - RPM_GPG_NAME = "<replaceable>key_name</replaceable>" - # Provide passphrase for the key - RPM_GPG_PASSPHRASE = "<replaceable>passphrase</replaceable>" - </literallayout> - <note> - Be sure to supply appropriate values for both - <replaceable>key_name</replaceable> and - <replaceable>passphrase</replaceable> - </note> - Aside from the - <filename>RPM_GPG_NAME</filename> and - <filename>RPM_GPG_PASSPHRASE</filename> variables in the - previous example, two optional variables related to signing - exist: - <itemizedlist> - <listitem><para> - <emphasis><filename>GPG_BIN</filename>:</emphasis> - Specifies a <filename>gpg</filename> binary/wrapper - that is executed when the package is signed. - </para></listitem> - <listitem><para> - <emphasis><filename>GPG_PATH</filename>:</emphasis> - Specifies the <filename>gpg</filename> home - directory used when the package is signed. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='processing-package-feeds'> - <title>Processing Package Feeds</title> - - <para> - In addition to being able to sign RPM packages, you can - also enable signed package feeds for IPK and RPM packages. - </para> - - <para> - The steps you need to take to enable signed package feed - use are similar to the steps used to sign RPM packages. - You must define the following in your - <filename>local.config</filename> or - <filename>distro.config</filename> file: - <literallayout class='monospaced'> - INHERIT += "sign_package_feed" - PACKAGE_FEED_GPG_NAME = "<replaceable>key_name</replaceable>" - PACKAGE_FEED_GPG_PASSPHRASE_FILE = "<replaceable>path_to_file_containing_passphrase</replaceable>" - </literallayout> - For signed package feeds, the passphrase must exist in a - separate file, which is pointed to by the - <filename>PACKAGE_FEED_GPG_PASSPHRASE_FILE</filename> - variable. - Regarding security, keeping a plain text passphrase out of - the configuration is more secure. - </para> - - <para> - Aside from the - <filename>PACKAGE_FEED_GPG_NAME</filename> and - <filename>PACKAGE_FEED_GPG_PASSPHRASE_FILE</filename> - variables, three optional variables related to signed - package feeds exist: - <itemizedlist> - <listitem><para> - <emphasis><filename>GPG_BIN</filename>:</emphasis> - Specifies a <filename>gpg</filename> binary/wrapper - that is executed when the package is signed. - </para></listitem> - <listitem><para> - <emphasis><filename>GPG_PATH</filename>:</emphasis> - Specifies the <filename>gpg</filename> home - directory used when the package is signed. - </para></listitem> - <listitem><para> - <emphasis><filename>PACKAGE_FEED_GPG_SIGNATURE_TYPE</filename>:</emphasis> - Specifies the type of <filename>gpg</filename> - signature. - This variable applies only to RPM and IPK package - feeds. - Allowable values for the - <filename>PACKAGE_FEED_GPG_SIGNATURE_TYPE</filename> - are "ASC", which is the default and specifies ascii - armored, and "BIN", which specifies binary. - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id='testing-packages-with-ptest'> - <title>Testing Packages With ptest</title> - - <para> - A Package Test (ptest) runs tests against packages built - by the OpenEmbedded build system on the target machine. - A ptest contains at least two items: the actual test, and - a shell script (<filename>run-ptest</filename>) that starts - the test. - The shell script that starts the test must not contain - the actual test - the script only starts the test. - On the other hand, the test can be anything from a simple - shell script that runs a binary and checks the output to - an elaborate system of test binaries and data files. - </para> - - <para> - The test generates output in the format used by - Automake: - <literallayout class='monospaced'> - <replaceable>result</replaceable>: <replaceable>testname</replaceable> - </literallayout> - where the result can be <filename>PASS</filename>, - <filename>FAIL</filename>, or <filename>SKIP</filename>, - and the testname can be any identifying string. - </para> - - <para> - For a list of Yocto Project recipes that are already - enabled with ptest, see the - <ulink url='https://wiki.yoctoproject.org/wiki/Ptest'>Ptest</ulink> - wiki page. - <note> - A recipe is "ptest-enabled" if it inherits the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-ptest'><filename>ptest</filename></ulink> - class. - </note> - </para> - - <section id='adding-ptest-to-your-build'> - <title>Adding ptest to Your Build</title> - - <para> - To add package testing to your build, add the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink> - and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink> - variables to your <filename>local.conf</filename> file, - which is found in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>: - <literallayout class='monospaced'> - DISTRO_FEATURES_append = " ptest" - EXTRA_IMAGE_FEATURES += "ptest-pkgs" - </literallayout> - Once your build is complete, the ptest files are installed - into the - <filename>/usr/lib/<replaceable>package</replaceable>/ptest</filename> - directory within the image, where - <filename><replaceable>package</replaceable></filename> - is the name of the package. - </para> - </section> - - <section id='running-ptest'> - <title>Running ptest</title> - - <para> - The <filename>ptest-runner</filename> package installs a - shell script that loops through all installed ptest test - suites and runs them in sequence. - Consequently, you might want to add this package to - your image. - </para> - </section> - - <section id='getting-your-package-ready'> - <title>Getting Your Package Ready</title> - - <para> - In order to enable a recipe to run installed ptests - on target hardware, - you need to prepare the recipes that build the packages - you want to test. - Here is what you have to do for each recipe: - <itemizedlist> - <listitem><para><emphasis>Be sure the recipe - inherits the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-ptest'><filename>ptest</filename></ulink> - class:</emphasis> - Include the following line in each recipe: - <literallayout class='monospaced'> - inherit ptest - </literallayout> - </para></listitem> - <listitem><para><emphasis>Create <filename>run-ptest</filename>:</emphasis> - This script starts your test. - Locate the script where you will refer to it - using - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink>. - Here is an example that starts a test for - <filename>dbus</filename>: - <literallayout class='monospaced'> - #!/bin/sh - cd test - make -k runtest-TESTS - </literallayout> - </para></listitem> - <listitem><para><emphasis>Ensure dependencies are - met:</emphasis> - If the test adds build or runtime dependencies - that normally do not exist for the package - (such as requiring "make" to run the test suite), - use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-RDEPENDS'><filename>RDEPENDS</filename></ulink> - variables in your recipe in order for the package - to meet the dependencies. - Here is an example where the package has a runtime - dependency on "make": - <literallayout class='monospaced'> - RDEPENDS_${PN}-ptest += "make" - </literallayout> - </para></listitem> - <listitem><para><emphasis>Add a function to build the - test suite:</emphasis> - Not many packages support cross-compilation of - their test suites. - Consequently, you usually need to add a - cross-compilation function to the package. - </para> - - <para>Many packages based on Automake compile and - run the test suite by using a single command - such as <filename>make check</filename>. - However, the host <filename>make check</filename> - builds and runs on the same computer, while - cross-compiling requires that the package is built - on the host but executed for the target - architecture (though often, as in the case for - ptest, the execution occurs on the host). - The built version of Automake that ships with the - Yocto Project includes a patch that separates - building and execution. - Consequently, packages that use the unaltered, - patched version of <filename>make check</filename> - automatically cross-compiles.</para> - <para>Regardless, you still must add a - <filename>do_compile_ptest</filename> function to - build the test suite. - Add a function similar to the following to your - recipe: - <literallayout class='monospaced'> - do_compile_ptest() { - oe_runmake buildtest-TESTS - } - </literallayout> - </para></listitem> - <listitem><para><emphasis>Ensure special configurations - are set:</emphasis> - If the package requires special configurations - prior to compiling the test code, you must - insert a <filename>do_configure_ptest</filename> - function into the recipe. - </para></listitem> - <listitem><para><emphasis>Install the test - suite:</emphasis> - The <filename>ptest</filename> class - automatically copies the file - <filename>run-ptest</filename> to the target and - then runs make <filename>install-ptest</filename> - to run the tests. - If this is not enough, you need to create a - <filename>do_install_ptest</filename> function and - make sure it gets called after the - "make install-ptest" completes. - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id='creating-node-package-manager-npm-packages'> - <title>Creating Node Package Manager (NPM) Packages</title> - - <para> - <ulink url='https://en.wikipedia.org/wiki/Npm_(software)'>NPM</ulink> - is a package manager for the JavaScript programming - language. - The Yocto Project supports the NPM - <ulink url='&YOCTO_DOCS_BB_URL;#bb-fetchers'>fetcher</ulink>. - You can use this fetcher in combination with - <ulink url='&YOCTO_DOCS_REF_URL;#ref-devtool-reference'><filename>devtool</filename></ulink> - to create recipes that produce NPM packages. - </para> - - <para> - Two workflows exist that allow you to create NPM packages - using <filename>devtool</filename>: the NPM registry modules - method and the NPM project code method. - <note> - While it is possible to create NPM recipes manually, - using <filename>devtool</filename> is far simpler. - </note> - Additionally, some requirements and caveats exist. - </para> - - <section id='npm-package-creation-requirements'> - <title>Requirements and Caveats</title> - - <para> - You need to be aware of the following before using - <filename>devtool</filename> to create NPM packages: - <itemizedlist> - <listitem><para> - Of the two methods that you can use - <filename>devtool</filename> to create NPM - packages, the registry approach is slightly - simpler. - However, you might consider the project - approach because you do not have to publish - your module in the NPM registry - (<ulink url='https://docs.npmjs.com/misc/registry'><filename>npm-registry</filename></ulink>), - which is NPM's public registry. - </para></listitem> - <listitem><para> - Be familiar with - <ulink url='&YOCTO_DOCS_REF_URL;#ref-devtool-reference'><filename>devtool</filename></ulink>. - </para></listitem> - <listitem><para> - The NPM host tools need the native - <filename>nodejs-npm</filename> package, which - is part of the OpenEmbedded environment. - You need to get the package by cloning the - <ulink url='https://github.com/openembedded/meta-openembedded'></ulink> - repository out of GitHub. - Be sure to add the path to your local copy to - your <filename>bblayers.conf</filename> file. - </para></listitem> - <listitem><para> - <filename>devtool</filename> cannot detect - native libraries in module dependencies. - Consequently, you must manually add packages - to your recipe. - </para></listitem> - <listitem><para> - While deploying NPM packages, - <filename>devtool</filename> cannot determine - which dependent packages are missing on the - target (e.g. the node runtime - <filename>nodejs</filename>). - Consequently, you need to find out what - files are missing and be sure they are on the - target. - </para></listitem> - <listitem><para> - Although you might not need NPM to run your - node package, it is useful to have NPM on your - target. - The NPM package name is - <filename>nodejs-npm</filename>. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='npm-using-the-registry-modules-method'> - <title>Using the Registry Modules Method</title> - - <para> - This section presents an example that uses the - <filename>cute-files</filename> module, which is a - file browser web application. - <note> - You must know the <filename>cute-files</filename> - module version. - </note> - </para> - - <para> - The first thing you need to do is use - <filename>devtool</filename> and the NPM fetcher to - create the recipe: - <literallayout class='monospaced'> - $ devtool add "npm://registry.npmjs.org;package=cute-files;version=1.0.2" - </literallayout> - The <filename>devtool add</filename> command runs - <filename>recipetool create</filename> and uses the - same fetch URI to download each dependency and capture - license details where possible. - The result is a generated recipe. - </para> - - <para> - The recipe file is fairly simple and contains every - license that <filename>recipetool</filename> finds - and includes the licenses in the recipe's - <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink> - variables. - You need to examine the variables and look for those - with "unknown" in the - <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink> - field. - You need to track down the license information for - "unknown" modules and manually add the information to the - recipe. - </para> - - <para> - <filename>recipetool</filename> creates a "shrinkwrap" file - for your recipe. - Shrinkwrap files capture the version of all dependent - modules. - Many packages do not provide shrinkwrap files. - <filename>recipetool</filename> create a shrinkwrap - file as it runs. - <note> - A package is created for each sub-module. - This policy is the only practical way to have the - licenses for all of the dependencies represented - in the license manifest of the image. - </note> - </para> - - <para> - The <filename>devtool edit-recipe</filename> command - lets you take a look at the recipe: - <literallayout class='monospaced'> - $ devtool edit-recipe cute-files - SUMMARY = "Turn any folder on your computer into a cute file browser, available on the local network." - LICENSE = "MIT & ISC & Unknown" - LIC_FILES_CHKSUM = "file://LICENSE;md5=71d98c0a1db42956787b1909c74a86ca \ - file://node_modules/toidentifier/LICENSE;md5=1a261071a044d02eb6f2bb47f51a3502 \ - file://node_modules/debug/LICENSE;md5=ddd815a475e7338b0be7a14d8ee35a99 \ - ... - - SRC_URI = " \ - npm://registry.npmjs.org/;package=cute-files;version=${PV} \ - npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ - " - - S = "${WORKDIR}/npm" - - inherit npm - - LICENSE_${PN} = "MIT" - LICENSE_${PN}-accepts = "MIT" - LICENSE_${PN}-array-flatten = "MIT" - ... - LICENSE_${PN}-vary = "MIT" - </literallayout> - Three key points exist in the previous example: - <itemizedlist> - <listitem><para> - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - uses the NPM scheme so that the NPM fetcher - is used. - </para></listitem> - <listitem><para> - <filename>recipetool</filename> collects all - the license information. - If a sub-module's license is unavailable, - the sub-module's name appears in the comments. - </para></listitem> - <listitem><para> - The <filename>inherit npm</filename> statement - causes the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-npm'><filename>npm</filename></ulink> - class to package up all the modules. - </para></listitem> - </itemizedlist> - </para> - - <para> - You can run the following command to build the - <filename>cute-files</filename> package: - <literallayout class='monospaced'> - $ devtool build cute-files - </literallayout> - Remember that <filename>nodejs</filename> must be - installed on the target before your package. - </para> - - <para> - Assuming 192.168.7.2 for the target's IP address, use - the following command to deploy your package: - <literallayout class='monospaced'> - $ devtool deploy-target -s cute-files root@192.168.7.2 - </literallayout> - Once the package is installed on the target, you can - test the application: - <note> - Because of a know issue, you cannot simply run - <filename>cute-files</filename> as you would if you - had run <filename>npm install</filename>. - </note> - <literallayout class='monospaced'> - $ cd /usr/lib/node_modules/cute-files - $ node cute-files.js - </literallayout> - On a browser, go to - <filename>http://192.168.7.2:3000</filename> and you - see the following: - <imagedata fileref="figures/cute-files-npm-example.png" align="center" width="6in" depth="4in" /> - </para> - - <para> - You can find the recipe in - <filename>workspace/recipes/cute-files</filename>. - You can use the recipe in any layer you choose. - </para> - </section> - - <section id='npm-using-the-npm-projects-method'> - <title>Using the NPM Projects Code Method</title> - - <para> - Although it is useful to package modules already in the - NPM registry, adding <filename>node.js</filename> projects - under development is a more common developer use case. - </para> - - <para> - This section covers the NPM projects code method, which is - very similar to the "registry" approach described in the - previous section. - In the NPM projects method, you provide - <filename>devtool</filename> with an URL that points to the - source files. - </para> - - <para> - Replicating the same example, (i.e. - <filename>cute-files</filename>) use the following command: - <literallayout class='monospaced'> - $ devtool add https://github.com/martinaglv/cute-files.git - </literallayout> - The recipe this command generates is very similar to the - recipe created in the previous section. - However, the <filename>SRC_URI</filename> looks like the - following: - <literallayout class='monospaced'> - SRC_URI = " \ - git://github.com/martinaglv/cute-files.git;protocol=https \ - npmsw://${THISDIR}/${BPN}/npm-shrinkwrap.json \ - " - </literallayout> - In this example, the main module is taken from the Git - repository and dependents are taken from the NPM registry. - Other than those differences, the recipe is basically the - same between the two methods. - You can build and deploy the package exactly as described - in the previous section that uses the registry modules - method. - </para> - </section> - </section> - - <section id='adding-custom-metadata-to-packages'> - <title>Adding custom metadata to packages</title> - - <para> - The variable <ulink url='&YOCTO_DOCS_REF_URL;#var-PACKAGE_ADD_METADATA'><filename>PACKAGE_ADD_METADATA</filename></ulink> - can be used to add additional metadata to packages. This is - reflected in the package control/spec file. To take the ipk - format for example, the CONTROL file stored inside would - contain the additional metadata as additional lines. - </para> - - <para> - The variable can be used in multiple ways, including using - suffixes to set it for a specific package type and/or package. - Note that the order of precedence is the same as this list: - <itemizedlist> - <listitem><para> - <filename>PACKAGE_ADD_METADATA_<PKGTYPE>_<PN></filename> - </para></listitem> - <listitem><para> - <filename>PACKAGE_ADD_METADATA_<PKGTYPE></filename> - </para></listitem> - <listitem><para> - <filename>PACKAGE_ADD_METADATA_<PN></filename> - </para></listitem> - <listitem><para> - <filename>PACKAGE_ADD_METADATA</filename> - </para></listitem> - </itemizedlist> - <PKGTYPE> is a parameter and expected to be a - distinct name of specific package type: - <itemizedlist> - <listitem><para>IPK for .ipk packages</para></listitem> - <listitem><para>DEB for .deb packages</para></listitem> - <listitem><para>RPM for .rpm packages</para></listitem> - </itemizedlist> - <PN> is a parameter and expected to be a package name. - </para> - - <para> - The variable can contain multiple [one-line] metadata fields - separated by the literal sequence '\n'. The separator can be - redefined using the variable flag <filename>separator</filename>. - </para> - - <para> - The following is an example that adds two custom fields for - ipk packages: - <literallayout class='monospaced'> - PACKAGE_ADD_METADATA_IPK = "Vendor: CustomIpk\nGroup: Applications/Spreadsheets" - </literallayout> - </para> - </section> - - </section> - - <section id='efficiently-fetching-source-files-during-a-build'> - <title>Efficiently Fetching Source Files During a Build</title> - - <para> - The OpenEmbedded build system works with source files located - through the - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - variable. - When you build something using BitBake, a big part of the operation - is locating and downloading all the source tarballs. - For images, downloading all the source for various packages can - take a significant amount of time. - </para> - - <para> - This section shows you how you can use mirrors to speed up - fetching source files and how you can pre-fetch files all of which - leads to more efficient use of resources and time. - </para> - - <section id='setting-up-effective-mirrors'> - <title>Setting up Effective Mirrors</title> - - <para> - A good deal that goes into a Yocto Project - build is simply downloading all of the source tarballs. - Maybe you have been working with another build system - (OpenEmbedded or Angstrom) for which you have built up a - sizable directory of source tarballs. - Or, perhaps someone else has such a directory for which you - have read access. - If so, you can save time by adding statements to your - configuration file so that the build process checks local - directories first for existing tarballs before checking the - Internet. - </para> - - <para> - Here is an efficient way to set it up in your - <filename>local.conf</filename> file: - <literallayout class='monospaced'> - SOURCE_MIRROR_URL ?= "file:///home/you/your-download-dir/" - INHERIT += "own-mirrors" - BB_GENERATE_MIRROR_TARBALLS = "1" - # BB_NO_NETWORK = "1" - </literallayout> - </para> - - <para> - In the previous example, the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BB_GENERATE_MIRROR_TARBALLS'><filename>BB_GENERATE_MIRROR_TARBALLS</filename></ulink> - variable causes the OpenEmbedded build system to generate - tarballs of the Git repositories and store them in the - <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> - directory. - Due to performance reasons, generating and storing these - tarballs is not the build system's default behavior. - </para> - - <para> - You can also use the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PREMIRRORS'><filename>PREMIRRORS</filename></ulink> - variable. - For an example, see the variable's glossary entry in the - Yocto Project Reference Manual. - </para> - </section> - - <section id='getting-source-files-and-suppressing-the-build'> - <title>Getting Source Files and Suppressing the Build</title> - - <para> - Another technique you can use to ready yourself for a - successive string of build operations, is to pre-fetch - all the source files without actually starting a build. - This technique lets you work through any download issues - and ultimately gathers all the source files into your - download directory - <ulink url='&YOCTO_DOCS_REF_URL;#structure-build-downloads'><filename>build/downloads</filename></ulink>, - which is located with - <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink>. - </para> - - <para> - Use the following BitBake command form to fetch all the - necessary sources without starting the build: - <literallayout class='monospaced'> - $ bitbake <replaceable>target</replaceable> --runall=fetch - </literallayout> - This variation of the BitBake command guarantees that you - have all the sources for that BitBake target should you - disconnect from the Internet and want to do the build - later offline. - </para> - </section> - </section> - - <section id="selecting-an-initialization-manager"> - <title>Selecting an Initialization Manager</title> - - <para> - By default, the Yocto Project uses SysVinit as the initialization - manager. - However, support also exists for systemd, - which is a full replacement for init with - parallel starting of services, reduced shell overhead and other - features that are used by many distributions. - </para> - - <para> - Within the system, SysVinit treats system components as services. - These services are maintained as shell scripts stored in the - <filename>/etc/init.d/</filename> directory. - Services organize into different run levels. - This organization is maintained by putting links to the services - in the <filename>/etc/rcN.d/</filename> directories, where - <replaceable>N/</replaceable> is one of the following options: - "S", "0", "1", "2", "3", "4", "5", or "6". - <note> - Each runlevel has a dependency on the previous runlevel. - This dependency allows the services to work properly. - </note> - </para> - - <para> - In comparison, systemd treats components as units. - Using units is a broader concept as compared to using a service. - A unit includes several different types of entities. - Service is one of the types of entities. - The runlevel concept in SysVinit corresponds to the concept of a - target in systemd, where target is also a type of supported unit. - </para> - - <para> - In a SysVinit-based system, services load sequentially (i.e. one - by one) during and parallelization is not supported. - With systemd, services start in parallel. - Needless to say, the method can have an impact on system startup - performance. - </para> - - <para> - If you want to use SysVinit, you do - not have to do anything. - But, if you want to use systemd, you must - take some steps as described in the following sections. - </para> - - <section id='using-systemd-exclusively'> - <title>Using systemd Exclusively</title> - - <para> - Set these variables in your distribution configuration - file as follows: - <literallayout class='monospaced'> - DISTRO_FEATURES_append = " systemd" - VIRTUAL-RUNTIME_init_manager = "systemd" - </literallayout> - You can also prevent the SysVinit - distribution feature from - being automatically enabled as follows: - <literallayout class='monospaced'> - DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit" - </literallayout> - Doing so removes any redundant SysVinit scripts. - </para> - - <para> - To remove initscripts from your image altogether, - set this variable also: - <literallayout class='monospaced'> - VIRTUAL-RUNTIME_initscripts = "" - </literallayout> - </para> - - <para> - For information on the backfill variable, see - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES_BACKFILL_CONSIDERED'><filename>DISTRO_FEATURES_BACKFILL_CONSIDERED</filename></ulink>. - </para> - </section> - - <section id='using-systemd-for-the-main-image-and-using-sysvinit-for-the-rescue-image'> - <title>Using systemd for the Main Image and Using SysVinit for the Rescue Image</title> - - <para> - Set these variables in your distribution configuration - file as follows: - <literallayout class='monospaced'> - DISTRO_FEATURES_append = " systemd" - VIRTUAL-RUNTIME_init_manager = "systemd" - </literallayout> - Doing so causes your main image to use the - <filename>packagegroup-core-boot.bb</filename> recipe and - systemd. - The rescue/minimal image cannot use this package group. - However, it can install SysVinit - and the appropriate packages will have support for both - systemd and SysVinit. - </para> - </section> - </section> - - <section id="selecting-dev-manager"> - <title>Selecting a Device Manager</title> - - <para> - The Yocto Project provides multiple ways to manage the device - manager (<filename>/dev</filename>): - <itemizedlist> - <listitem><para><emphasis>Persistent and Pre-Populated<filename>/dev</filename>:</emphasis> - For this case, the <filename>/dev</filename> directory - is persistent and the required device nodes are created - during the build. - </para></listitem> - <listitem><para><emphasis>Use <filename>devtmpfs</filename> with a Device Manager:</emphasis> - For this case, the <filename>/dev</filename> directory - is provided by the kernel as an in-memory file system and - is automatically populated by the kernel at runtime. - Additional configuration of device nodes is done in user - space by a device manager like - <filename>udev</filename> or - <filename>busybox-mdev</filename>. - </para></listitem> - </itemizedlist> - </para> - - <section id="static-dev-management"> - <title>Using Persistent and Pre-Populated<filename>/dev</filename></title> - - <para> - To use the static method for device population, you need to - set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-USE_DEVFS'><filename>USE_DEVFS</filename></ulink> - variable to "0" as follows: - <literallayout class='monospaced'> - USE_DEVFS = "0" - </literallayout> - </para> - - <para> - The content of the resulting <filename>/dev</filename> - directory is defined in a Device Table file. - The - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_DEVICE_TABLES'><filename>IMAGE_DEVICE_TABLES</filename></ulink> - variable defines the Device Table to use and should be set - in the machine or distro configuration file. - Alternatively, you can set this variable in your - <filename>local.conf</filename> configuration file. - </para> - - <para> - If you do not define the - <filename>IMAGE_DEVICE_TABLES</filename> variable, the default - <filename>device_table-minimal.txt</filename> is used: - <literallayout class='monospaced'> - IMAGE_DEVICE_TABLES = "device_table-mymachine.txt" - </literallayout> - </para> - - <para> - The population is handled by the <filename>makedevs</filename> - utility during image creation: - </para> - </section> - - <section id="devtmpfs-dev-management"> - <title>Using <filename>devtmpfs</filename> and a Device Manager</title> - - <para> - To use the dynamic method for device population, you need to - use (or be sure to set) the - <ulink url='&YOCTO_DOCS_REF_URL;#var-USE_DEVFS'><filename>USE_DEVFS</filename></ulink> - variable to "1", which is the default: - <literallayout class='monospaced'> - USE_DEVFS = "1" - </literallayout> - With this setting, the resulting <filename>/dev</filename> - directory is populated by the kernel using - <filename>devtmpfs</filename>. - Make sure the corresponding kernel configuration variable - <filename>CONFIG_DEVTMPFS</filename> is set when building - you build a Linux kernel. - </para> - - <para> - All devices created by <filename>devtmpfs</filename> will be - owned by <filename>root</filename> and have permissions - <filename>0600</filename>. - </para> - - <para> - To have more control over the device nodes, you can use a - device manager like <filename>udev</filename> or - <filename>busybox-mdev</filename>. - You choose the device manager by defining the - <filename>VIRTUAL-RUNTIME_dev_manager</filename> variable - in your machine or distro configuration file. - Alternatively, you can set this variable in your - <filename>local.conf</filename> configuration file: - <literallayout class='monospaced'> - VIRTUAL-RUNTIME_dev_manager = "udev" - - # Some alternative values - # VIRTUAL-RUNTIME_dev_manager = "busybox-mdev" - # VIRTUAL-RUNTIME_dev_manager = "systemd" - </literallayout> - </para> - </section> - </section> - - <section id="platdev-appdev-srcrev"> - <title>Using an External SCM</title> - - <para> - If you're working on a recipe that pulls from an external Source - Code Manager (SCM), it is possible to have the OpenEmbedded build - system notice new recipe changes added to the SCM and then build - the resulting packages that depend on the new recipes by using - the latest versions. - This only works for SCMs from which it is possible to get a - sensible revision number for changes. - Currently, you can do this with Apache Subversion (SVN), Git, and - Bazaar (BZR) repositories. - </para> - - <para> - To enable this behavior, the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PV'><filename>PV</filename></ulink> - of the recipe needs to reference - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCPV'><filename>SRCPV</filename></ulink>. - Here is an example: - <literallayout class='monospaced'> - PV = "1.2.3+git${SRCPV}" - </literallayout> - Then, you can add the following to your - <filename>local.conf</filename>: - <literallayout class='monospaced'> - SRCREV_pn-<replaceable>PN</replaceable> = "${AUTOREV}" - </literallayout> - <ulink url='&YOCTO_DOCS_REF_URL;#var-PN'><filename>PN</filename></ulink> - is the name of the recipe for which you want to enable automatic source - revision updating. - </para> - - <para> - If you do not want to update your local configuration file, you can - add the following directly to the recipe to finish enabling - the feature: - <literallayout class='monospaced'> - SRCREV = "${AUTOREV}" - </literallayout> - </para> - - <para> - The Yocto Project provides a distribution named - <filename>poky-bleeding</filename>, whose configuration - file contains the line: - <literallayout class='monospaced'> - require conf/distro/include/poky-floating-revisions.inc - </literallayout> - This line pulls in the listed include file that contains - numerous lines of exactly that form: - <literallayout class='monospaced'> - #SRCREV_pn-opkg-native ?= "${AUTOREV}" - #SRCREV_pn-opkg-sdk ?= "${AUTOREV}" - #SRCREV_pn-opkg ?= "${AUTOREV}" - #SRCREV_pn-opkg-utils-native ?= "${AUTOREV}" - #SRCREV_pn-opkg-utils ?= "${AUTOREV}" - SRCREV_pn-gconf-dbus ?= "${AUTOREV}" - SRCREV_pn-matchbox-common ?= "${AUTOREV}" - SRCREV_pn-matchbox-config-gtk ?= "${AUTOREV}" - SRCREV_pn-matchbox-desktop ?= "${AUTOREV}" - SRCREV_pn-matchbox-keyboard ?= "${AUTOREV}" - SRCREV_pn-matchbox-panel-2 ?= "${AUTOREV}" - SRCREV_pn-matchbox-themes-extra ?= "${AUTOREV}" - SRCREV_pn-matchbox-terminal ?= "${AUTOREV}" - SRCREV_pn-matchbox-wm ?= "${AUTOREV}" - SRCREV_pn-settings-daemon ?= "${AUTOREV}" - SRCREV_pn-screenshot ?= "${AUTOREV}" - . - . - . - </literallayout> - These lines allow you to experiment with building a - distribution that tracks the latest development source - for numerous packages. - <note><title>Caution</title> - The <filename>poky-bleeding</filename> distribution - is not tested on a regular basis. - Keep this in mind if you use it. - </note> - </para> - </section> - - <section id='creating-a-read-only-root-filesystem'> - <title>Creating a Read-Only Root Filesystem</title> - - <para> - Suppose, for security reasons, you need to disable - your target device's root filesystem's write permissions - (i.e. you need a read-only root filesystem). - Or, perhaps you are running the device's operating system - from a read-only storage device. - For either case, you can customize your image for - that behavior. - </para> - - <note> - Supporting a read-only root filesystem requires that the system and - applications do not try to write to the root filesystem. - You must configure all parts of the target system to write - elsewhere, or to gracefully fail in the event of attempting to - write to the root filesystem. - </note> - - <section id='creating-the-root-filesystem'> - <title>Creating the Root Filesystem</title> - - <para> - To create the read-only root filesystem, simply add the - "read-only-rootfs" feature to your image, normally in one of two ways. - The first way is to add the "read-only-rootfs" image feature - in the image's recipe file via the - <filename>IMAGE_FEATURES</filename> variable: - <literallayout class='monospaced'> - IMAGE_FEATURES += "read-only-rootfs" - </literallayout> - As an alternative, you can add the same feature from within your - build directory's <filename>local.conf</filename> file with the - associated <filename>EXTRA_IMAGE_FEATURES</filename> variable, as in: - <literallayout class='monospaced'> - EXTRA_IMAGE_FEATURES = "read-only-rootfs" - </literallayout> - </para> - - <para> - For more information on how to use these variables, see the - "<link linkend='usingpoky-extend-customimage-imagefeatures'>Customizing Images Using Custom <filename>IMAGE_FEATURES</filename> and <filename>EXTRA_IMAGE_FEATURES</filename></link>" - section. - For information on the variables, see - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> - and <ulink url='&YOCTO_DOCS_REF_URL;#var-EXTRA_IMAGE_FEATURES'><filename>EXTRA_IMAGE_FEATURES</filename></ulink>. - </para> - </section> - - <section id='post-installation-scripts'> - <title>Post-Installation Scripts</title> - - <para> - It is very important that you make sure all - post-Installation (<filename>pkg_postinst</filename>) scripts - for packages that are installed into the image can be run - at the time when the root filesystem is created during the - build on the host system. - These scripts cannot attempt to run during first-boot on the - target device. - With the "read-only-rootfs" feature enabled, - the build system checks during root filesystem creation to make - sure all post-installation scripts succeed. - If any of these scripts still need to be run after the root - filesystem is created, the build immediately fails. - These build-time checks ensure that the build fails - rather than the target device fails later during its - initial boot operation. - </para> - - <para> - Most of the common post-installation scripts generated by the - build system for the out-of-the-box Yocto Project are engineered - so that they can run during root filesystem creation - (e.g. post-installation scripts for caching fonts). - However, if you create and add custom scripts, you need - to be sure they can be run during this file system creation. - </para> - - <para> - Here are some common problems that prevent - post-installation scripts from running during root filesystem - creation: - <itemizedlist> - <listitem><para> - <emphasis>Not using $D in front of absolute - paths:</emphasis> - The build system defines - <filename>$</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-D'><filename>D</filename></ulink> - when the root filesystem is created. - Furthermore, <filename>$D</filename> is blank when the - script is run on the target device. - This implies two purposes for <filename>$D</filename>: - ensuring paths are valid in both the host and target - environments, and checking to determine which - environment is being used as a method for taking - appropriate actions. - </para></listitem> - <listitem><para> - <emphasis>Attempting to run processes that are - specific to or dependent on the target - architecture:</emphasis> - You can work around these attempts by using native - tools, which run on the host system, - to accomplish the same tasks, or - by alternatively running the processes under QEMU, - which has the <filename>qemu_run_binary</filename> - function. - For more information, see the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-qemu'><filename>qemu</filename></ulink> - class.</para></listitem> - </itemizedlist> - </para> - </section> - - <section id='areas-with-write-access'> - <title>Areas With Write Access</title> - - <para> - With the "read-only-rootfs" feature enabled, - any attempt by the target to write to the root filesystem at - runtime fails. - Consequently, you must make sure that you configure processes - and applications that attempt these types of writes do so - to directories with write access (e.g. - <filename>/tmp</filename> or <filename>/var/run</filename>). - </para> - </section> - </section> - - - - - <section id='maintaining-build-output-quality'> - <title>Maintaining Build Output Quality</title> - - <para> - Many factors can influence the quality of a build. - For example, if you upgrade a recipe to use a new version of an - upstream software package or you experiment with some new - configuration options, subtle changes can occur that you might - not detect until later. - Consider the case where your recipe is using a newer version of - an upstream package. - In this case, a new version of a piece of software might - introduce an optional dependency on another library, which is - auto-detected. - If that library has already been built when the software is - building, the software will link to the built library and that - library will be pulled into your image along with the new - software even if you did not want the library. - </para> - - <para> - The - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-buildhistory'><filename>buildhistory</filename></ulink> - class exists to help you maintain the quality of your build - output. - You can use the class to highlight unexpected and possibly - unwanted changes in the build output. - When you enable build history, it records information about the - contents of each package and image and then commits that - information to a local Git repository where you can examine - the information. - </para> - - <para> - The remainder of this section describes the following: - <itemizedlist> - <listitem><para> - How you can enable and disable build history - </para></listitem> - <listitem><para> - How to understand what the build history contains - </para></listitem> - <listitem><para> - How to limit the information used for build history - </para></listitem> - <listitem><para> - How to examine the build history from both a - command-line and web interface - </para></listitem> - </itemizedlist> - </para> - - <section id='enabling-and-disabling-build-history'> - <title>Enabling and Disabling Build History</title> - - <para> - Build history is disabled by default. - To enable it, add the following <filename>INHERIT</filename> - statement and set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT</filename></ulink> - variable to "1" at the end of your - <filename>conf/local.conf</filename> file found in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>: - <literallayout class='monospaced'> - INHERIT += "buildhistory" - BUILDHISTORY_COMMIT = "1" - </literallayout> - Enabling build history as previously described causes the - OpenEmbedded build system to collect build output information - and commit it as a single commit to a local - <ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink> - repository. - <note> - Enabling build history increases your build times slightly, - particularly for images, and increases the amount of disk - space used during the build. - </note> - </para> - - <para> - You can disable build history by removing the previous - statements from your <filename>conf/local.conf</filename> - file. - </para> - </section> - - <section id='understanding-what-the-build-history-contains'> - <title>Understanding What the Build History Contains</title> - - <para> - Build history information is kept in - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-TOPDIR'><filename>TOPDIR</filename></ulink><filename>}/buildhistory</filename> - in the Build Directory as defined by the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_DIR'><filename>BUILDHISTORY_DIR</filename></ulink> - variable. - The following is an example abbreviated listing: - <imagedata fileref="figures/buildhistory.png" align="center" width="6in" depth="4in" /> - </para> - - <para> - At the top level, a <filename>metadata-revs</filename> - file exists that lists the revisions of the repositories for - the enabled layers when the build was produced. - The rest of the data splits into separate - <filename>packages</filename>, <filename>images</filename> - and <filename>sdk</filename> directories, the contents of - which are described as follows. - </para> - - <section id='build-history-package-information'> - <title>Build History Package Information</title> - - <para> - The history for each package contains a text file that has - name-value pairs with information about the package. - For example, - <filename>buildhistory/packages/i586-poky-linux/busybox/busybox/latest</filename> - contains the following: - <literallayout class='monospaced'> - PV = 1.22.1 - PR = r32 - RPROVIDES = - RDEPENDS = glibc (>= 2.20) update-alternatives-opkg - RRECOMMENDS = busybox-syslog busybox-udhcpc update-rc.d - PKGSIZE = 540168 - FILES = /usr/bin/* /usr/sbin/* /usr/lib/busybox/* /usr/lib/lib*.so.* \ - /etc /com /var /bin/* /sbin/* /lib/*.so.* /lib/udev/rules.d \ - /usr/lib/udev/rules.d /usr/share/busybox /usr/lib/busybox/* \ - /usr/share/pixmaps /usr/share/applications /usr/share/idl \ - /usr/share/omf /usr/share/sounds /usr/lib/bonobo/servers - FILELIST = /bin/busybox /bin/busybox.nosuid /bin/busybox.suid /bin/sh \ - /etc/busybox.links.nosuid /etc/busybox.links.suid - </literallayout> - Most of these name-value pairs correspond to variables - used to produce the package. - The exceptions are <filename>FILELIST</filename>, which - is the actual list of files in the package, and - <filename>PKGSIZE</filename>, which is the total size of - files in the package in bytes. - </para> - - <para> - A file also exists that corresponds to the recipe from - which the package came (e.g. - <filename>buildhistory/packages/i586-poky-linux/busybox/latest</filename>): - <literallayout class='monospaced'> - PV = 1.22.1 - PR = r32 - DEPENDS = initscripts kern-tools-native update-rc.d-native \ - virtual/i586-poky-linux-compilerlibs virtual/i586-poky-linux-gcc \ - virtual/libc virtual/update-alternatives - PACKAGES = busybox-ptest busybox-httpd busybox-udhcpd busybox-udhcpc \ - busybox-syslog busybox-mdev busybox-hwclock busybox-dbg \ - busybox-staticdev busybox-dev busybox-doc busybox-locale busybox - </literallayout> - </para> - - <para> - Finally, for those recipes fetched from a version control - system (e.g., Git), a file exists that lists source - revisions that are specified in the recipe and lists - the actual revisions used during the build. - Listed and actual revisions might differ when - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRCREV'><filename>SRCREV</filename></ulink> - is set to - ${<ulink url='&YOCTO_DOCS_REF_URL;#var-AUTOREV'><filename>AUTOREV</filename></ulink>}. - Here is an example assuming - <filename>buildhistory/packages/qemux86-poky-linux/linux-yocto/latest_srcrev</filename>): - <literallayout class='monospaced'> - # SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" - SRCREV_machine = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" - # SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" - SRCREV_meta = "a227f20eff056e511d504b2e490f3774ab260d6f" - </literallayout> - You can use the - <filename>buildhistory-collect-srcrevs</filename> - command with the <filename>-a</filename> option to - collect the stored <filename>SRCREV</filename> values - from build history and report them in a format suitable for - use in global configuration (e.g., - <filename>local.conf</filename> or a distro include file) - to override floating <filename>AUTOREV</filename> values - to a fixed set of revisions. - Here is some example output from this command: - <literallayout class='monospaced'> - $ buildhistory-collect-srcrevs -a - # i586-poky-linux - SRCREV_pn-glibc = "b8079dd0d360648e4e8de48656c5c38972621072" - SRCREV_pn-glibc-initial = "b8079dd0d360648e4e8de48656c5c38972621072" - SRCREV_pn-opkg-utils = "53274f087565fd45d8452c5367997ba6a682a37a" - SRCREV_pn-kmod = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" - # x86_64-linux - SRCREV_pn-gtk-doc-stub-native = "1dea266593edb766d6d898c79451ef193eb17cfa" - SRCREV_pn-dtc-native = "65cc4d2748a2c2e6f27f1cf39e07a5dbabd80ebf" - SRCREV_pn-update-rc.d-native = "eca680ddf28d024954895f59a241a622dd575c11" - SRCREV_glibc_pn-cross-localedef-native = "b8079dd0d360648e4e8de48656c5c38972621072" - SRCREV_localedef_pn-cross-localedef-native = "c833367348d39dad7ba018990bfdaffaec8e9ed3" - SRCREV_pn-prelink-native = "faa069deec99bf61418d0bab831c83d7c1b797ca" - SRCREV_pn-opkg-utils-native = "53274f087565fd45d8452c5367997ba6a682a37a" - SRCREV_pn-kern-tools-native = "23345b8846fe4bd167efdf1bd8a1224b2ba9a5ff" - SRCREV_pn-kmod-native = "fd56638aed3fe147015bfa10ed4a5f7491303cb4" - # qemux86-poky-linux - SRCREV_machine_pn-linux-yocto = "38cd560d5022ed2dbd1ab0dca9642e47c98a0aa1" - SRCREV_meta_pn-linux-yocto = "a227f20eff056e511d504b2e490f3774ab260d6f" - # all-poky-linux - SRCREV_pn-update-rc.d = "eca680ddf28d024954895f59a241a622dd575c11" - </literallayout> - <note> - Here are some notes on using the - <filename>buildhistory-collect-srcrevs</filename> - command: - <itemizedlist> - <listitem><para> - By default, only values where the - <filename>SRCREV</filename> was not hardcoded - (usually when <filename>AUTOREV</filename> - is used) are reported. - Use the <filename>-a</filename> option to - see all <filename>SRCREV</filename> values. - </para></listitem> - <listitem><para> - The output statements might not have any effect - if overrides are applied elsewhere in the - build system configuration. - Use the <filename>-f</filename> option to add - the <filename>forcevariable</filename> override - to each output line if you need to work around - this restriction. - </para></listitem> - <listitem><para> - The script does apply special handling when - building for multiple machines. - However, the script does place a comment before - each set of values that specifies which - triplet to which they belong as previously - shown (e.g., - <filename>i586-poky-linux</filename>). - </para></listitem> - </itemizedlist> - </note> - </para> - </section> - - <section id='build-history-image-information'> - <title>Build History Image Information</title> - - <para> - The files produced for each image are as follows: - <itemizedlist> - <listitem><para> - <filename>image-files:</filename> - A directory containing selected files from the root - filesystem. - The files are defined by - <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_IMAGE_FILES'><filename>BUILDHISTORY_IMAGE_FILES</filename></ulink>. - </para></listitem> - <listitem><para> - <filename>build-id.txt:</filename> - Human-readable information about the build - configuration and metadata source revisions. - This file contains the full build header as printed - by BitBake. - </para></listitem> - <listitem><para> - <filename>*.dot:</filename> - Dependency graphs for the image that are - compatible with <filename>graphviz</filename>. - </para></listitem> - <listitem><para> - <filename>files-in-image.txt:</filename> - A list of files in the image with permissions, - owner, group, size, and symlink information. - </para></listitem> - <listitem><para> - <filename>image-info.txt:</filename> - A text file containing name-value pairs with - information about the image. - See the following listing example for more - information. - </para></listitem> - <listitem><para> - <filename>installed-package-names.txt:</filename> - A list of installed packages by name only. - </para></listitem> - <listitem><para> - <filename>installed-package-sizes.txt:</filename> - A list of installed packages ordered by size. - </para></listitem> - <listitem><para> - <filename>installed-packages.txt:</filename> - A list of installed packages with full package - filenames. - </para></listitem> - </itemizedlist> - <note> - Installed package information is able to be gathered - and produced even if package management is disabled - for the final image. - </note> - </para> - - <para> - Here is an example of <filename>image-info.txt</filename>: - <literallayout class='monospaced'> - DISTRO = poky - DISTRO_VERSION = 1.7 - USER_CLASSES = buildstats image-mklibs image-prelink - IMAGE_CLASSES = image_types - IMAGE_FEATURES = debug-tweaks - IMAGE_LINGUAS = - IMAGE_INSTALL = packagegroup-core-boot run-postinsts - BAD_RECOMMENDATIONS = - NO_RECOMMENDATIONS = - PACKAGE_EXCLUDE = - ROOTFS_POSTPROCESS_COMMAND = write_package_manifest; license_create_manifest; \ - write_image_manifest ; buildhistory_list_installed_image ; \ - buildhistory_get_image_installed ; ssh_allow_empty_password; \ - postinst_enable_logging; rootfs_update_timestamp ; ssh_disable_dns_lookup ; - IMAGE_POSTPROCESS_COMMAND = buildhistory_get_imageinfo ; - IMAGESIZE = 6900 - </literallayout> - Other than <filename>IMAGESIZE</filename>, which is the - total size of the files in the image in Kbytes, the - name-value pairs are variables that may have influenced the - content of the image. - This information is often useful when you are trying to - determine why a change in the package or file - listings has occurred. - </para> - </section> - - <section id='using-build-history-to-gather-image-information-only'> - <title>Using Build History to Gather Image Information Only</title> - - <para> - As you can see, build history produces image information, - including dependency graphs, so you can see why something - was pulled into the image. - If you are just interested in this information and not - interested in collecting specific package or SDK - information, you can enable writing only image information - without any history by adding the following to your - <filename>conf/local.conf</filename> file found in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>: - <literallayout class='monospaced'> - INHERIT += "buildhistory" - BUILDHISTORY_COMMIT = "0" - BUILDHISTORY_FEATURES = "image" - </literallayout> - Here, you set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_FEATURES'><filename>BUILDHISTORY_FEATURES</filename></ulink> - variable to use the image feature only. - </para> - </section> - - <section id='build-history-sdk-information'> - <title>Build History SDK Information</title> - - <para> - Build history collects similar information on the contents - of SDKs - (e.g. <filename>bitbake -c populate_sdk imagename</filename>) - as compared to information it collects for images. - Furthermore, this information differs depending on whether - an extensible or standard SDK is being produced. - </para> - - <para> - The following list shows the files produced for SDKs: - <itemizedlist> - <listitem><para> - <filename>files-in-sdk.txt:</filename> - A list of files in the SDK with permissions, - owner, group, size, and symlink information. - This list includes both the host and target parts - of the SDK. - </para></listitem> - <listitem><para> - <filename>sdk-info.txt:</filename> - A text file containing name-value pairs with - information about the SDK. - See the following listing example for more - information. - </para></listitem> - <listitem><para> - <filename>sstate-task-sizes.txt:</filename> - A text file containing name-value pairs with - information about task group sizes - (e.g. <filename>do_populate_sysroot</filename> - tasks have a total size). - The <filename>sstate-task-sizes.txt</filename> file - exists only when an extensible SDK is created. - </para></listitem> - <listitem><para> - <filename>sstate-package-sizes.txt:</filename> - A text file containing name-value pairs with - information for the shared-state packages and - sizes in the SDK. - The <filename>sstate-package-sizes.txt</filename> - file exists only when an extensible SDK is created. - </para></listitem> - <listitem><para> - <filename>sdk-files:</filename> - A folder that contains copies of the files - mentioned in - <filename>BUILDHISTORY_SDK_FILES</filename> if the - files are present in the output. - Additionally, the default value of - <filename>BUILDHISTORY_SDK_FILES</filename> is - specific to the extensible SDK although you can - set it differently if you would like to pull in - specific files from the standard SDK.</para> - - <para>The default files are - <filename>conf/local.conf</filename>, - <filename>conf/bblayers.conf</filename>, - <filename>conf/auto.conf</filename>, - <filename>conf/locked-sigs.inc</filename>, and - <filename>conf/devtool.conf</filename>. - Thus, for an extensible SDK, these files get - copied into the <filename>sdk-files</filename> - directory. - </para></listitem> - <listitem><para> - The following information appears under - each of the <filename>host</filename> - and <filename>target</filename> directories - for the portions of the SDK that run on the host - and on the target, respectively: - <note> - The following files for the most part are empty - when producing an extensible SDK because this - type of SDK is not constructed from packages - as is the standard SDK. - </note> - <itemizedlist> - <listitem><para> - <filename>depends.dot:</filename> - Dependency graph for the SDK that is - compatible with - <filename>graphviz</filename>. - </para></listitem> - <listitem><para> - <filename>installed-package-names.txt:</filename> - A list of installed packages by name only. - </para></listitem> - <listitem><para> - <filename>installed-package-sizes.txt:</filename> - A list of installed packages ordered by size. - </para></listitem> - <listitem><para> - <filename>installed-packages.txt:</filename> - A list of installed packages with full - package filenames. - </para></listitem> - </itemizedlist> - </para></listitem> - </itemizedlist> - </para> - - <para> - Here is an example of <filename>sdk-info.txt</filename>: - <literallayout class='monospaced'> - DISTRO = poky - DISTRO_VERSION = 1.3+snapshot-20130327 - SDK_NAME = poky-glibc-i686-arm - SDK_VERSION = 1.3+snapshot - SDKMACHINE = - SDKIMAGE_FEATURES = dev-pkgs dbg-pkgs - BAD_RECOMMENDATIONS = - SDKSIZE = 352712 - </literallayout> - Other than <filename>SDKSIZE</filename>, which is the - total size of the files in the SDK in Kbytes, the - name-value pairs are variables that might have influenced - the content of the SDK. - This information is often useful when you are trying to - determine why a change in the package or file listings - has occurred. - </para> - </section> - - <section id='examining-build-history-information'> - <title>Examining Build History Information</title> - - <para> - You can examine build history output from the command - line or from a web interface. - </para> - - <para> - To see any changes that have occurred (assuming you have - <ulink url='&YOCTO_DOCS_REF_URL;#var-BUILDHISTORY_COMMIT'><filename>BUILDHISTORY_COMMIT</filename></ulink><filename> = "1"</filename>), - you can simply use any Git command that allows you to - view the history of a repository. - Here is one method: - <literallayout class='monospaced'> - $ git log -p - </literallayout> - You need to realize, however, that this method does show - changes that are not significant (e.g. a package's size - changing by a few bytes). - </para> - - <para> - A command-line tool called - <filename>buildhistory-diff</filename> does exist, though, - that queries the Git repository and prints just the - differences that might be significant in human-readable - form. - Here is an example: - <literallayout class='monospaced'> - $ ~/poky/poky/scripts/buildhistory-diff . HEAD^ - Changes to images/qemux86_64/glibc/core-image-minimal (files-in-image.txt): - /etc/anotherpkg.conf was added - /sbin/anotherpkg was added - * (installed-package-names.txt): - * anotherpkg was added - Changes to images/qemux86_64/glibc/core-image-minimal (installed-package-names.txt): - anotherpkg was added - packages/qemux86_64-poky-linux/v86d: PACKAGES: added "v86d-extras" - * PR changed from "r0" to "r1" - * PV changed from "0.1.10" to "0.1.12" - packages/qemux86_64-poky-linux/v86d/v86d: PKGSIZE changed from 110579 to 144381 (+30%) - * PR changed from "r0" to "r1" - * PV changed from "0.1.10" to "0.1.12" - </literallayout> - <note> - The <filename>buildhistory-diff</filename> tool - requires the <filename>GitPython</filename> package. - Be sure to install it using Pip3 as follows: - <literallayout class='monospaced'> - $ pip3 install GitPython --user - </literallayout> - Alternatively, you can install - <filename>python3-git</filename> using the appropriate - distribution package manager (e.g. - <filename>apt-get</filename>, <filename>dnf</filename>, - or <filename>zipper</filename>). - </note> - </para> - - <para> - To see changes to the build history using a web interface, - follow the instruction in the <filename>README</filename> - file here. - <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/buildhistory-web/'></ulink>. - </para> - - <para> - Here is a sample screenshot of the interface: - <imagedata fileref="figures/buildhistory-web.png" align="center" scalefit="1" width="130%" contentdepth="130%" /> - </para> - </section> - </section> - </section> - - <section id="performing-automated-runtime-testing"> - <title>Performing Automated Runtime Testing</title> - - <para> - The OpenEmbedded build system makes available a series of automated - tests for images to verify runtime functionality. - You can run these tests on either QEMU or actual target hardware. - Tests are written in Python making use of the - <filename>unittest</filename> module, and the majority of them - run commands on the target system over SSH. - This section describes how you set up the environment to use these - tests, run available tests, and write and add your own tests. - </para> - - <para> - For information on the test and QA infrastructure available - within the Yocto Project, see the - "<ulink url='&YOCTO_DOCS_REF_URL;#testing-and-quality-assurance'>Testing and Quality Assurance</ulink>" - section in the Yocto Project Reference Manual. - </para> - - <section id='enabling-tests'> - <title>Enabling Tests</title> - - <para> - Depending on whether you are planning to run tests using - QEMU or on the hardware, you have to take - different steps to enable the tests. - See the following subsections for information on how to - enable both types of tests. - </para> - - <section id='qemu-image-enabling-tests'> - <title>Enabling Runtime Tests on QEMU</title> - - <para> - In order to run tests, you need to do the following: - <itemizedlist> - <listitem><para><emphasis>Set up to avoid interaction - with <filename>sudo</filename> for networking:</emphasis> - To accomplish this, you must do one of the - following: - <itemizedlist> - <listitem><para>Add - <filename>NOPASSWD</filename> for your user - in <filename>/etc/sudoers</filename> either for - all commands or just for - <filename>runqemu-ifup</filename>. - You must provide the full path as that can - change if you are using multiple clones of the - source repository. - <note> - On some distributions, you also need to - comment out "Defaults requiretty" in - <filename>/etc/sudoers</filename>. - </note></para></listitem> - <listitem><para>Manually configure a tap interface - for your system.</para></listitem> - <listitem><para>Run as root the script in - <filename>scripts/runqemu-gen-tapdevs</filename>, - which should generate a list of tap devices. - This is the option typically chosen for - Autobuilder-type environments. - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - Be sure to use an absolute path - when calling this script - with sudo. - </para></listitem> - <listitem><para> - The package recipe - <filename>qemu-helper-native</filename> - is required to run this script. - Build the package using the - following command: - <literallayout class='monospaced'> - $ bitbake qemu-helper-native - </literallayout> - </para></listitem> - </itemizedlist> - </note> - </para></listitem> - </itemizedlist></para></listitem> - <listitem><para><emphasis>Set the - <filename>DISPLAY</filename> variable:</emphasis> - You need to set this variable so that you have an X - server available (e.g. start - <filename>vncserver</filename> for a headless machine). - </para></listitem> - <listitem><para><emphasis>Be sure your host's firewall - accepts incoming connections from - 192.168.7.0/24:</emphasis> - Some of the tests (in particular DNF tests) start - an HTTP server on a random high number port, - which is used to serve files to the target. - The DNF module serves - <filename>${WORKDIR}/oe-rootfs-repo</filename> - so it can run DNF channel commands. - That means your host's firewall - must accept incoming connections from 192.168.7.0/24, - which is the default IP range used for tap devices - by <filename>runqemu</filename>.</para></listitem> - <listitem><para><emphasis>Be sure your host has the - correct packages installed:</emphasis> - Depending your host's distribution, you need - to have the following packages installed: - <itemizedlist> - <listitem><para>Ubuntu and Debian: - <filename>sysstat</filename> and - <filename>iproute2</filename> - </para></listitem> - <listitem><para>OpenSUSE: - <filename>sysstat</filename> and - <filename>iproute2</filename> - </para></listitem> - <listitem><para>Fedora: - <filename>sysstat</filename> and - <filename>iproute</filename> - </para></listitem> - <listitem><para>CentOS: - <filename>sysstat</filename> and - <filename>iproute</filename> - </para></listitem> - </itemizedlist> - </para></listitem> - </itemizedlist> - </para> - - <para> - Once you start running the tests, the following happens: - <orderedlist> - <listitem><para>A copy of the root filesystem is written - to <filename>${WORKDIR}/testimage</filename>. - </para></listitem> - <listitem><para>The image is booted under QEMU using the - standard <filename>runqemu</filename> script. - </para></listitem> - <listitem><para>A default timeout of 500 seconds occurs - to allow for the boot process to reach the login prompt. - You can change the timeout period by setting - <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_QEMUBOOT_TIMEOUT'><filename>TEST_QEMUBOOT_TIMEOUT</filename></ulink> - in the <filename>local.conf</filename> file. - </para></listitem> - <listitem><para>Once the boot process is reached and the - login prompt appears, the tests run. - The full boot log is written to - <filename>${WORKDIR}/testimage/qemu_boot_log</filename>. - </para></listitem> - <listitem><para>Each test module loads in the order found - in <filename>TEST_SUITES</filename>. - You can find the full output of the commands run over - SSH in - <filename>${WORKDIR}/testimgage/ssh_target_log</filename>. - </para></listitem> - <listitem><para>If no failures occur, the task running the - tests ends successfully. - You can find the output from the - <filename>unittest</filename> in the task log at - <filename>${WORKDIR}/temp/log.do_testimage</filename>. - </para></listitem> - </orderedlist> - </para> - </section> - - <section id='hardware-image-enabling-tests'> - <title>Enabling Runtime Tests on Hardware</title> - - <para> - The OpenEmbedded build system can run tests on real - hardware, and for certain devices it can also deploy - the image to be tested onto the device beforehand. - </para> - - <para> - For automated deployment, a "master image" is installed - onto the hardware once as part of setup. - Then, each time tests are to be run, the following - occurs: - <orderedlist> - <listitem><para>The master image is booted into and - used to write the image to be tested to - a second partition. - </para></listitem> - <listitem><para>The device is then rebooted using an - external script that you need to provide. - </para></listitem> - <listitem><para>The device boots into the image to be - tested. - </para></listitem> - </orderedlist> - </para> - - <para> - When running tests (independent of whether the image - has been deployed automatically or not), the device is - expected to be connected to a network on a - pre-determined IP address. - You can either use static IP addresses written into - the image, or set the image to use DHCP and have your - DHCP server on the test network assign a known IP address - based on the MAC address of the device. - </para> - - <para> - In order to run tests on hardware, you need to set - <filename>TEST_TARGET</filename> to an appropriate value. - For QEMU, you do not have to change anything, the default - value is "qemu". - For running tests on hardware, the following options exist: - <itemizedlist> - <listitem><para><emphasis>"simpleremote":</emphasis> - Choose "simpleremote" if you are going to - run tests on a target system that is already - running the image to be tested and is available - on the network. - You can use "simpleremote" in conjunction - with either real hardware or an image running - within a separately started QEMU or any - other virtual machine manager. - </para></listitem> - <listitem><para><emphasis>"SystemdbootTarget":</emphasis> - Choose "SystemdbootTarget" if your hardware is - an EFI-based machine with - <filename>systemd-boot</filename> as bootloader and - <filename>core-image-testmaster</filename> - (or something similar) is installed. - Also, your hardware under test must be in a - DHCP-enabled network that gives it the same IP - address for each reboot.</para> - <para>If you choose "SystemdbootTarget", there are - additional requirements and considerations. - See the - "<link linkend='selecting-systemdboottarget'>Selecting SystemdbootTarget</link>" - section, which follows, for more information. - </para></listitem> - <listitem><para><emphasis>"BeagleBoneTarget":</emphasis> - Choose "BeagleBoneTarget" if you are deploying - images and running tests on the BeagleBone - "Black" or original "White" hardware. - For information on how to use these tests, see the - comments at the top of the BeagleBoneTarget - <filename>meta-yocto-bsp/lib/oeqa/controllers/beaglebonetarget.py</filename> - file. - </para></listitem> - <listitem><para><emphasis>"EdgeRouterTarget":</emphasis> - Choose "EdgeRouterTarget" is you are deploying - images and running tests on the Ubiquiti Networks - EdgeRouter Lite. - For information on how to use these tests, see the - comments at the top of the EdgeRouterTarget - <filename>meta-yocto-bsp/lib/oeqa/controllers/edgeroutertarget.py</filename> - file. - </para></listitem> - <listitem><para><emphasis>"GrubTarget":</emphasis> - Choose the "supports deploying images and running - tests on any generic PC that boots using GRUB. - For information on how to use these tests, see the - comments at the top of the GrubTarget - <filename>meta-yocto-bsp/lib/oeqa/controllers/grubtarget.py</filename> - file. - </para></listitem> - <listitem><para><emphasis>"<replaceable>your-target</replaceable>":</emphasis> - Create your own custom target if you want to run - tests when you are deploying images and running - tests on a custom machine within your BSP layer. - To do this, you need to add a Python unit that - defines the target class under - <filename>lib/oeqa/controllers/</filename> within - your layer. - You must also provide an empty - <filename>__init__.py</filename>. - For examples, see files in - <filename>meta-yocto-bsp/lib/oeqa/controllers/</filename>. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='selecting-systemdboottarget'> - <title>Selecting SystemdbootTarget</title> - - <para> - If you did not set <filename>TEST_TARGET</filename> to - "SystemdbootTarget", then you do not need any information - in this section. - You can skip down to the - "<link linkend='qemu-image-running-tests'>Running Tests</link>" - section. - </para> - - <para> - If you did set <filename>TEST_TARGET</filename> to - "SystemdbootTarget", you also need to perform a one-time - setup of your master image by doing the following: - <orderedlist> - <listitem><para><emphasis>Set <filename>EFI_PROVIDER</filename>:</emphasis> - Be sure that <filename>EFI_PROVIDER</filename> - is as follows: - <literallayout class='monospaced'> - EFI_PROVIDER = "systemd-boot" - </literallayout> - </para></listitem> - <listitem><para><emphasis>Build the master image:</emphasis> - Build the <filename>core-image-testmaster</filename> - image. - The <filename>core-image-testmaster</filename> - recipe is provided as an example for a - "master" image and you can customize the image - recipe as you would any other recipe. - </para> - <para>Here are the image recipe requirements: - <itemizedlist> - <listitem><para>Inherits - <filename>core-image</filename> - so that kernel modules are installed. - </para></listitem> - <listitem><para>Installs normal linux utilities - not busybox ones (e.g. - <filename>bash</filename>, - <filename>coreutils</filename>, - <filename>tar</filename>, - <filename>gzip</filename>, and - <filename>kmod</filename>). - </para></listitem> - <listitem><para>Uses a custom - Initial RAM Disk (initramfs) image with a - custom installer. - A normal image that you can install usually - creates a single rootfs partition. - This image uses another installer that - creates a specific partition layout. - Not all Board Support Packages (BSPs) - can use an installer. - For such cases, you need to manually create - the following partition layout on the - target: - <itemizedlist> - <listitem><para>First partition mounted - under <filename>/boot</filename>, - labeled "boot". - </para></listitem> - <listitem><para>The main rootfs - partition where this image gets - installed, which is mounted under - <filename>/</filename>. - </para></listitem> - <listitem><para>Another partition - labeled "testrootfs" where test - images get deployed. - </para></listitem> - </itemizedlist> - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para><emphasis>Install image:</emphasis> - Install the image that you just built on the target - system. - </para></listitem> - </orderedlist> - </para> - - <para> - The final thing you need to do when setting - <filename>TEST_TARGET</filename> to "SystemdbootTarget" is - to set up the test image: - <orderedlist> - <listitem><para><emphasis>Set up your <filename>local.conf</filename> file:</emphasis> - Make sure you have the following statements in - your <filename>local.conf</filename> file: - <literallayout class='monospaced'> - IMAGE_FSTYPES += "tar.gz" - INHERIT += "testimage" - TEST_TARGET = "SystemdbootTarget" - TEST_TARGET_IP = "192.168.2.3" - </literallayout> - </para></listitem> - <listitem><para><emphasis>Build your test image:</emphasis> - Use BitBake to build the image: - <literallayout class='monospaced'> - $ bitbake core-image-sato - </literallayout> - </para></listitem> - </orderedlist> - </para> - </section> - - <section id='power-control'> - <title>Power Control</title> - - <para> - For most hardware targets other than "simpleremote", - you can control power: - <itemizedlist> - <listitem><para> - You can use - <filename>TEST_POWERCONTROL_CMD</filename> - together with - <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename> - as a command that runs on the host and does power - cycling. - The test code passes one argument to that command: - off, on or cycle (off then on). - Here is an example that could appear in your - <filename>local.conf</filename> file: - <literallayout class='monospaced'> - TEST_POWERCONTROL_CMD = "powercontrol.exp test 10.11.12.1 nuc1" - </literallayout> - In this example, the expect script does the - following: - <literallayout class='monospaced'> - ssh test@10.11.12.1 "pyctl nuc1 <replaceable>arg</replaceable>" - </literallayout> - It then runs a Python script that controls power - for a label called <filename>nuc1</filename>. - <note> - You need to customize - <filename>TEST_POWERCONTROL_CMD</filename> - and - <filename>TEST_POWERCONTROL_EXTRA_ARGS</filename> - for your own setup. - The one requirement is that it accepts - "on", "off", and "cycle" as the last argument. - </note> - </para></listitem> - <listitem><para> - When no command is defined, it connects to the - device over SSH and uses the classic reboot command - to reboot the device. - Classic reboot is fine as long as the machine - actually reboots (i.e. the SSH test has not - failed). - It is useful for scenarios where you have a simple - setup, typically with a single board, and where - some manual interaction is okay from time to time. - </para></listitem> - </itemizedlist> - If you have no hardware to automatically perform power - control but still wish to experiment with automated - hardware testing, you can use the dialog-power-control - script that shows a dialog prompting you to perform the - required power action. - This script requires either KDialog or Zenity to be - installed. - To use this script, set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_POWERCONTROL_CMD'><filename>TEST_POWERCONTROL_CMD</filename></ulink> - variable as follows: - <literallayout class='monospaced'> - TEST_POWERCONTROL_CMD = "${COREBASE}/scripts/contrib/dialog-power-control" - </literallayout> - </para> - </section> - - <section id='serial-console-connection'> - <title>Serial Console Connection</title> - - <para> - For test target classes requiring a serial console - to interact with the bootloader (e.g. BeagleBoneTarget, - EdgeRouterTarget, and GrubTarget), you need to - specify a command to use to connect to the serial console - of the target machine by using the - <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_CMD'><filename>TEST_SERIALCONTROL_CMD</filename></ulink> - variable and optionally the - <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SERIALCONTROL_EXTRA_ARGS'><filename>TEST_SERIALCONTROL_EXTRA_ARGS</filename></ulink> - variable. - </para> - - <para> - These cases could be a serial terminal program if the - machine is connected to a local serial port, or a - <filename>telnet</filename> or - <filename>ssh</filename> command connecting to a remote - console server. - Regardless of the case, the command simply needs to - connect to the serial console and forward that connection - to standard input and output as any normal terminal - program does. - For example, to use the picocom terminal program on - serial device <filename>/dev/ttyUSB0</filename> - at 115200bps, you would set the variable as follows: - <literallayout class='monospaced'> - TEST_SERIALCONTROL_CMD = "picocom /dev/ttyUSB0 -b 115200" - </literallayout> - For local devices where the serial port device disappears - when the device reboots, an additional "serdevtry" wrapper - script is provided. - To use this wrapper, simply prefix the terminal command - with - <filename>${COREBASE}/scripts/contrib/serdevtry</filename>: - <literallayout class='monospaced'> - TEST_SERIALCONTROL_CMD = "${COREBASE}/scripts/contrib/serdevtry picocom -b -115200 /dev/ttyUSB0" - </literallayout> - </para> - </section> - </section> - - <section id="qemu-image-running-tests"> - <title>Running Tests</title> - - <para> - You can start the tests automatically or manually: - <itemizedlist> - <listitem><para><emphasis>Automatically running tests:</emphasis> - To run the tests automatically after the - OpenEmbedded build system successfully creates an image, - first set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-TESTIMAGE_AUTO'><filename>TESTIMAGE_AUTO</filename></ulink> - variable to "1" in your <filename>local.conf</filename> - file in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>: - <literallayout class='monospaced'> - TESTIMAGE_AUTO = "1" - </literallayout> - Next, build your image. - If the image successfully builds, the tests run: - <literallayout class='monospaced'> - bitbake core-image-sato - </literallayout></para></listitem> - <listitem><para><emphasis>Manually running tests:</emphasis> - To manually run the tests, first globally inherit the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage*'><filename>testimage</filename></ulink> - class by editing your <filename>local.conf</filename> - file: - <literallayout class='monospaced'> - INHERIT += "testimage" - </literallayout> - Next, use BitBake to run the tests: - <literallayout class='monospaced'> - bitbake -c testimage <replaceable>image</replaceable> - </literallayout></para></listitem> - </itemizedlist> - </para> - - <para> - All test files reside in - <filename>meta/lib/oeqa/runtime</filename> in the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. - A test name maps directly to a Python module. - Each test module may contain a number of individual tests. - Tests are usually grouped together by the area - tested (e.g tests for systemd reside in - <filename>meta/lib/oeqa/runtime/systemd.py</filename>). - </para> - - <para> - You can add tests to any layer provided you place them in the - proper area and you extend - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink> - in the <filename>local.conf</filename> file as normal. - Be sure that tests reside in - <filename><replaceable>layer</replaceable>/lib/oeqa/runtime</filename>. - <note> - Be sure that module names do not collide with module names - used in the default set of test modules in - <filename>meta/lib/oeqa/runtime</filename>. - </note> - </para> - - <para> - You can change the set of tests run by appending or overriding - <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink> - variable in <filename>local.conf</filename>. - Each name in <filename>TEST_SUITES</filename> represents a - required test for the image. - Test modules named within <filename>TEST_SUITES</filename> - cannot be skipped even if a test is not suitable for an image - (e.g. running the RPM tests on an image without - <filename>rpm</filename>). - Appending "auto" to <filename>TEST_SUITES</filename> causes the - build system to try to run all tests that are suitable for the - image (i.e. each test module may elect to skip itself). - </para> - - <para> - The order you list tests in <filename>TEST_SUITES</filename> - is important and influences test dependencies. - Consequently, tests that depend on other tests should be added - after the test on which they depend. - For example, since the <filename>ssh</filename> test - depends on the - <filename>ping</filename> test, "ssh" needs to come after - "ping" in the list. - The test class provides no re-ordering or dependency handling. - <note> - Each module can have multiple classes with multiple test - methods. - And, Python <filename>unittest</filename> rules apply. - </note> - </para> - - <para> - Here are some things to keep in mind when running tests: - <itemizedlist> - <listitem><para>The default tests for the image are defined - as: - <literallayout class='monospaced'> - DEFAULT_TEST_SUITES_pn-<replaceable>image</replaceable> = "ping ssh df connman syslog xorg scp vnc date rpm dnf dmesg" - </literallayout></para></listitem> - <listitem><para>Add your own test to the list of the - by using the following: - <literallayout class='monospaced'> - TEST_SUITES_append = " mytest" - </literallayout></para></listitem> - <listitem><para>Run a specific list of tests as follows: - <literallayout class='monospaced'> - TEST_SUITES = "test1 test2 test3" - </literallayout> - Remember, order is important. - Be sure to place a test that is dependent on another test - later in the order.</para></listitem> - </itemizedlist> - </para> - </section> - - <section id="exporting-tests"> - <title>Exporting Tests</title> - - <para> - You can export tests so that they can run independently of - the build system. - Exporting tests is required if you want to be able to hand - the test execution off to a scheduler. - You can only export tests that are defined in - <ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_SUITES'><filename>TEST_SUITES</filename></ulink>. - </para> - - <para> - If your image is already built, make sure the following are set - in your <filename>local.conf</filename> file: - <literallayout class='monospaced'> - INHERIT +="testexport" - TEST_TARGET_IP = "<replaceable>IP-address-for-the-test-target</replaceable>" - TEST_SERVER_IP = "<replaceable>IP-address-for-the-test-server</replaceable>" - </literallayout> - You can then export the tests with the following BitBake - command form: - <literallayout class='monospaced'> - $ bitbake <replaceable>image</replaceable> -c testexport - </literallayout> - Exporting the tests places them in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - in - <filename>tmp/testexport/</filename><replaceable>image</replaceable>, - which is controlled by the - <filename>TEST_EXPORT_DIR</filename> variable. - </para> - - <para> - You can now run the tests outside of the build environment: - <literallayout class='monospaced'> - $ cd tmp/testexport/<replaceable>image</replaceable> - $ ./runexported.py testdata.json - </literallayout> - </para> - - <para> - Here is a complete example that shows IP addresses and uses - the <filename>core-image-sato</filename> image: - <literallayout class='monospaced'> - INHERIT +="testexport" - TEST_TARGET_IP = "192.168.7.2" - TEST_SERVER_IP = "192.168.7.1" - </literallayout> - Use BitBake to export the tests: - <literallayout class='monospaced'> - $ bitbake core-image-sato -c testexport - </literallayout> - Run the tests outside of the build environment using the - following: - <literallayout class='monospaced'> - $ cd tmp/testexport/core-image-sato - $ ./runexported.py testdata.json - </literallayout> - </para> - </section> - - <section id="qemu-image-writing-new-tests"> - <title>Writing New Tests</title> - - <para> - As mentioned previously, all new test files need to be in the - proper place for the build system to find them. - New tests for additional functionality outside of the core - should be added to the layer that adds the functionality, in - <filename><replaceable>layer</replaceable>/lib/oeqa/runtime</filename> - (as long as - <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink> - is extended in the layer's - <filename>layer.conf</filename> file as normal). - Just remember the following: - <itemizedlist> - <listitem><para>Filenames need to map directly to test - (module) names. - </para></listitem> - <listitem><para>Do not use module names that - collide with existing core tests. - </para></listitem> - <listitem><para>Minimally, an empty - <filename>__init__.py</filename> file must exist - in the runtime directory. - </para></listitem> - </itemizedlist> - </para> - - <para> - To create a new test, start by copying an existing module - (e.g. <filename>syslog.py</filename> or - <filename>gcc.py</filename> are good ones to use). - Test modules can use code from - <filename>meta/lib/oeqa/utils</filename>, which are helper - classes. - </para> - - <note> - Structure shell commands such that you rely on them and they - return a single code for success. - Be aware that sometimes you will need to parse the output. - See the <filename>df.py</filename> and - <filename>date.py</filename> modules for examples. - </note> - - <para> - You will notice that all test classes inherit - <filename>oeRuntimeTest</filename>, which is found in - <filename>meta/lib/oetest.py</filename>. - This base class offers some helper attributes, which are - described in the following sections: - </para> - - <section id='qemu-image-writing-tests-class-methods'> - <title>Class Methods</title> - - <para> - Class methods are as follows: - <itemizedlist> - <listitem><para><emphasis><filename>hasPackage(pkg)</filename>:</emphasis> - Returns "True" if <filename>pkg</filename> is in the - installed package list of the image, which is based - on the manifest file that is generated during the - <filename>do_rootfs</filename> task. - </para></listitem> - <listitem><para><emphasis><filename>hasFeature(feature)</filename>:</emphasis> - Returns "True" if the feature is in - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink> - or - <ulink url='&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES'><filename>DISTRO_FEATURES</filename></ulink>. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='qemu-image-writing-tests-class-attributes'> - <title>Class Attributes</title> - - <para> - Class attributes are as follows: - <itemizedlist> - <listitem><para><emphasis><filename>pscmd</filename>:</emphasis> - Equals "ps -ef" if <filename>procps</filename> is - installed in the image. - Otherwise, <filename>pscmd</filename> equals - "ps" (busybox). - </para></listitem> - <listitem><para><emphasis><filename>tc</filename>:</emphasis> - The called test context, which gives access to the - following attributes: - <itemizedlist> - <listitem><para><emphasis><filename>d</filename>:</emphasis> - The BitBake datastore, which allows you to - use stuff such as - <filename>oeRuntimeTest.tc.d.getVar("VIRTUAL-RUNTIME_init_manager")</filename>. - </para></listitem> - <listitem><para><emphasis><filename>testslist</filename> and <filename>testsrequired</filename>:</emphasis> - Used internally. - The tests do not need these. - </para></listitem> - <listitem><para><emphasis><filename>filesdir</filename>:</emphasis> - The absolute path to - <filename>meta/lib/oeqa/runtime/files</filename>, - which contains helper files for tests meant - for copying on the target such as small - files written in C for compilation. - </para></listitem> - <listitem><para><emphasis><filename>target</filename>:</emphasis> - The target controller object used to deploy - and start an image on a particular target - (e.g. Qemu, SimpleRemote, and - SystemdbootTarget). - Tests usually use the following: - <itemizedlist> - <listitem><para><emphasis><filename>ip</filename>:</emphasis> - The target's IP address. - </para></listitem> - <listitem><para><emphasis><filename>server_ip</filename>:</emphasis> - The host's IP address, which is - usually used by the DNF test - suite. - </para></listitem> - <listitem><para><emphasis><filename>run(cmd, timeout=None)</filename>:</emphasis> - The single, most used method. - This command is a wrapper for: - <filename>ssh root@host "cmd"</filename>. - The command returns a tuple: - (status, output), which are what - their names imply - the return code - of "cmd" and whatever output - it produces. - The optional timeout argument - represents the number of seconds the - test should wait for "cmd" to - return. - If the argument is "None", the - test uses the default instance's - timeout period, which is 300 - seconds. - If the argument is "0", the test - runs until the command returns. - </para></listitem> - <listitem><para><emphasis><filename>copy_to(localpath, remotepath)</filename>:</emphasis> - <filename>scp localpath root@ip:remotepath</filename>. - </para></listitem> - <listitem><para><emphasis><filename>copy_from(remotepath, localpath)</filename>:</emphasis> - <filename>scp root@host:remotepath localpath</filename>. - </para></listitem> - </itemizedlist></para></listitem> - </itemizedlist></para></listitem> - </itemizedlist> - </para> - </section> - - <section id='qemu-image-writing-tests-instance-attributes'> - <title>Instance Attributes</title> - - <para> - A single instance attribute exists, which is - <filename>target</filename>. - The <filename>target</filename> instance attribute is - identical to the class attribute of the same name, which - is described in the previous section. - This attribute exists as both an instance and class - attribute so tests can use - <filename>self.target.run(cmd)</filename> in instance - methods instead of - <filename>oeRuntimeTest.tc.target.run(cmd)</filename>. - </para> - </section> - </section> - - <section id='installing-packages-in-the-dut-without-the-package-manager'> - <title>Installing Packages in the DUT Without the Package Manager</title> - - <para> - When a test requires a package built by BitBake, it is possible - to install that package. - Installing the package does not require a package manager be - installed in the device under test (DUT). - It does, however, require an SSH connection and the target must - be using the <filename>sshcontrol</filename> class. - <note> - This method uses <filename>scp</filename> to copy files - from the host to the target, which causes permissions and - special attributes to be lost. - </note> - </para> - - <para> - A JSON file is used to define the packages needed by a test. - This file must be in the same path as the file used to define - the tests. - Furthermore, the filename must map directly to the test - module name with a <filename>.json</filename> extension. - </para> - - <para> - The JSON file must include an object with the test name as - keys of an object or an array. - This object (or array of objects) uses the following data: - <itemizedlist> - <listitem><para>"pkg" - A mandatory string that is the - name of the package to be installed. - </para></listitem> - <listitem><para>"rm" - An optional boolean, which defaults - to "false", that specifies to remove the package after - the test. - </para></listitem> - <listitem><para>"extract" - An optional boolean, which - defaults to "false", that specifies if the package must - be extracted from the package format. - When set to "true", the package is not automatically - installed into the DUT. - </para></listitem> - </itemizedlist> - </para> - - <para> - Following is an example JSON file that handles test "foo" - installing package "bar" and test "foobar" installing - packages "foo" and "bar". - Once the test is complete, the packages are removed from the - DUT. - <literallayout class='monospaced'> - { - "foo": { - "pkg": "bar" - }, - "foobar": [ - { - "pkg": "foo", - "rm": true - }, - { - "pkg": "bar", - "rm": true - } - ] - } - </literallayout> - </para> - </section> - </section> - - <section id='usingpoky-debugging-tools-and-techniques'> - <title>Debugging Tools and Techniques</title> - - <para> - The exact method for debugging build failures depends on the nature - of the problem and on the system's area from which the bug - originates. - Standard debugging practices such as comparison against the last - known working version with examination of the changes and the - re-application of steps to identify the one causing the problem are - valid for the Yocto Project just as they are for any other system. - Even though it is impossible to detail every possible potential - failure, this section provides some general tips to aid in - debugging given a variety of situations. - <note><title>Tip</title> - A useful feature for debugging is the error reporting tool. - Configuring the Yocto Project to use this tool causes the - OpenEmbedded build system to produce error reporting commands as - part of the console output. - You can enter the commands after the build completes to log - error information into a common database, that can help you - figure out what might be going wrong. - For information on how to enable and use this feature, see the - "<link linkend='using-the-error-reporting-tool'>Using the Error Reporting Tool</link>" - section. - </note> - </para> - - <para> - The following list shows the debugging topics in the remainder of - this section: - <itemizedlist> - <listitem><para> - "<link linkend='dev-debugging-viewing-logs-from-failed-tasks'>Viewing Logs from Failed Tasks</link>" - describes how to find and view logs from tasks that - failed during the build process. - </para></listitem> - <listitem><para> - "<link linkend='dev-debugging-viewing-variable-values'>Viewing Variable Values</link>" - describes how to use the BitBake <filename>-e</filename> - option to examine variable values after a recipe has been - parsed. - </para></listitem> - <listitem><para> - "<link linkend='viewing-package-information-with-oe-pkgdata-util'>Viewing Package Information with <filename>oe-pkgdata-util</filename></link>" - describes how to use the - <filename>oe-pkgdata-util</filename> utility to query - <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></ulink> - and display package-related information for built - packages. - </para></listitem> - <listitem><para> - "<link linkend='dev-viewing-dependencies-between-recipes-and-tasks'>Viewing Dependencies Between Recipes and Tasks</link>" - describes how to use the BitBake <filename>-g</filename> - option to display recipe dependency information used - during the build. - </para></listitem> - <listitem><para> - "<link linkend='dev-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</link>" - describes how to use the - <filename>bitbake-dumpsig</filename> command in - conjunction with key subdirectories in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink> - to determine variable dependencies. - </para></listitem> - <listitem><para> - "<link linkend='dev-debugging-taskrunning'>Running Specific Tasks</link>" - describes how to use several BitBake options (e.g. - <filename>-c</filename>, <filename>-C</filename>, and - <filename>-f</filename>) to run specific tasks in the - build chain. - It can be useful to run tasks "out-of-order" when trying - isolate build issues. - </para></listitem> - <listitem><para> - "<link linkend='dev-debugging-bitbake'>General BitBake Problems</link>" - describes how to use BitBake's <filename>-D</filename> - debug output option to reveal more about what BitBake is - doing during the build. - </para></listitem> - <listitem><para> - "<link linkend='dev-debugging-buildfile'>Building with No Dependencies</link>" - describes how to use the BitBake <filename>-b</filename> - option to build a recipe while ignoring dependencies. - </para></listitem> - <listitem><para> - "<link linkend='recipe-logging-mechanisms'>Recipe Logging Mechanisms</link>" - describes how to use the many recipe logging functions - to produce debugging output and report errors and warnings. - </para></listitem> - <listitem><para> - "<link linkend='debugging-parallel-make-races'>Debugging Parallel Make Races</link>" - describes how to debug situations where the build consists - of several parts that are run simultaneously and when the - output or result of one part is not ready for use with a - different part of the build that depends on that output. - </para></listitem> - <listitem><para> - "<link linkend='platdev-gdb-remotedebug'>Debugging With the GNU Project Debugger (GDB) Remotely</link>" - describes how to use GDB to allow you to examine running - programs, which can help you fix problems. - </para></listitem> - <listitem><para> - "<link linkend='debugging-with-the-gnu-project-debugger-gdb-on-the-target'>Debugging with the GNU Project Debugger (GDB) on the Target</link>" - describes how to use GDB directly on target hardware for - debugging. - </para></listitem> - <listitem><para> - "<link linkend='dev-other-debugging-others'>Other Debugging Tips</link>" - describes miscellaneous debugging tips that can be useful. - </para></listitem> - </itemizedlist> - </para> - - <section id='dev-debugging-viewing-logs-from-failed-tasks'> - <title>Viewing Logs from Failed Tasks</title> - - <para> - You can find the log for a task in the file - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/temp/log.do_</filename><replaceable>taskname</replaceable>. - For example, the log for the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-compile'><filename>do_compile</filename></ulink> - task of the QEMU minimal image for the x86 machine - (<filename>qemux86</filename>) might be in - <filename>tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/temp/log.do_compile</filename>. - To see the commands - <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink> - ran to generate a log, look at the corresponding - <filename>run.do_</filename><replaceable>taskname</replaceable> - file in the same directory. - </para> - - <para> - <filename>log.do_</filename><replaceable>taskname</replaceable> - and - <filename>run.do_</filename><replaceable>taskname</replaceable> - are actually symbolic links to - <filename>log.do_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable> - and - <filename>log.run_</filename><replaceable>taskname</replaceable><filename>.</filename><replaceable>pid</replaceable>, - where <replaceable>pid</replaceable> is the PID the task had - when it ran. - The symlinks always point to the files corresponding to the most - recent run. - </para> - </section> - - <section id='dev-debugging-viewing-variable-values'> - <title>Viewing Variable Values</title> - - <para> - Sometimes you need to know the value of a variable as a - result of BitBake's parsing step. - This could be because some unexpected behavior occurred - in your project. - Perhaps an attempt to - <ulink url='&YOCTO_DOCS_BB_URL;#modifying-existing-variables'>modify a variable</ulink> - did not work out as expected. - </para> - - <para> - BitBake's <filename>-e</filename> option is used to display - variable values after parsing. - The following command displays the variable values after the - configuration files (i.e. <filename>local.conf</filename>, - <filename>bblayers.conf</filename>, - <filename>bitbake.conf</filename> and so forth) have been - parsed: - <literallayout class='monospaced'> - $ bitbake -e - </literallayout> - The following command displays variable values after a specific - recipe has been parsed. - The variables include those from the configuration as well: - <literallayout class='monospaced'> - $ bitbake -e recipename - </literallayout> - <note><para> - Each recipe has its own private set of variables - (datastore). - Internally, after parsing the configuration, a copy of the - resulting datastore is made prior to parsing each recipe. - This copying implies that variables set in one recipe will - not be visible to other recipes.</para> - - <para>Likewise, each task within a recipe gets a private - datastore based on the recipe datastore, which means that - variables set within one task will not be visible to - other tasks.</para> - </note> - </para> - - <para> - In the output of <filename>bitbake -e</filename>, each - variable is preceded by a description of how the variable - got its value, including temporary values that were later - overriden. - This description also includes variable flags (varflags) set on - the variable. - The output can be very helpful during debugging. - </para> - - <para> - Variables that are exported to the environment are preceded by - <filename>export</filename> in the output of - <filename>bitbake -e</filename>. - See the following example: - <literallayout class='monospaced'> - export CC="i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/ulf/poky/build/tmp/sysroots/qemux86" - </literallayout> - </para> - - <para> - In addition to variable values, the output of the - <filename>bitbake -e</filename> and - <filename>bitbake -e</filename> <replaceable>recipe</replaceable> - commands includes the following information: - <itemizedlist> - <listitem><para> - The output starts with a tree listing all configuration - files and classes included globally, recursively listing - the files they include or inherit in turn. - Much of the behavior of the OpenEmbedded build system - (including the behavior of the - <ulink url='&YOCTO_DOCS_REF_URL;#normal-recipe-build-tasks'>normal recipe build tasks</ulink>) - is implemented in the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-base'><filename>base</filename></ulink> - class and the classes it inherits, rather than being - built into BitBake itself. - </para></listitem> - <listitem><para> - After the variable values, all functions appear in the - output. - For shell functions, variables referenced within the - function body are expanded. - If a function has been modified using overrides or - using override-style operators like - <filename>_append</filename> and - <filename>_prepend</filename>, then the final assembled - function body appears in the output. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id='viewing-package-information-with-oe-pkgdata-util'> - <title>Viewing Package Information with <filename>oe-pkgdata-util</filename></title> - - <para> - You can use the <filename>oe-pkgdata-util</filename> - command-line utility to query - <ulink url='&YOCTO_DOCS_REF_URL;#var-PKGDATA_DIR'><filename>PKGDATA_DIR</filename></ulink> - and display various package-related information. - When you use the utility, you must use it to view information - on packages that have already been built. - </para> - - <para> - Following are a few of the available - <filename>oe-pkgdata-util</filename> subcommands. - <note> - You can use the standard * and ? globbing wildcards as part - of package names and paths. - </note> - <itemizedlist> - <listitem><para> - <filename>oe-pkgdata-util list-pkgs [</filename><replaceable>pattern</replaceable><filename>]</filename>: - Lists all packages that have been built, optionally - limiting the match to packages that match - <replaceable>pattern</replaceable>. - </para></listitem> - <listitem><para> - <filename>oe-pkgdata-util list-pkg-files </filename><replaceable>package</replaceable><filename> ...</filename>: - Lists the files and directories contained in the given - packages. - <note> - <para> - A different way to view the contents of a package is - to look at the - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink><filename>}/packages-split</filename> - directory of the recipe that generates the - package. - This directory is created by the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink> - task and has one subdirectory for each package the - recipe generates, which contains the files stored in - that package.</para> - <para> - If you want to inspect the - <filename>${WORKDIR}/packages-split</filename> - directory, make sure that - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-rm-work'><filename>rm_work</filename></ulink> - is not enabled when you build the recipe. - </para> - </note> - </para></listitem> - <listitem><para> - <filename>oe-pkgdata-util find-path </filename><replaceable>path</replaceable><filename> ...</filename>: - Lists the names of the packages that contain the given - paths. - For example, the following tells us that - <filename>/usr/share/man/man1/make.1</filename> - is contained in the <filename>make-doc</filename> - package: - <literallayout class='monospaced'> - $ oe-pkgdata-util find-path /usr/share/man/man1/make.1 - make-doc: /usr/share/man/man1/make.1 - </literallayout> - </para></listitem> - <listitem><para> - <filename>oe-pkgdata-util lookup-recipe </filename><replaceable>package</replaceable><filename> ...</filename>: - Lists the name of the recipes that - produce the given packages. - </para></listitem> - </itemizedlist> - </para> - - <para> - For more information on the <filename>oe-pkgdata-util</filename> - command, use the help facility: - <literallayout class='monospaced'> - $ oe-pkgdata-util ‐‐help - $ oe-pkgdata-util <replaceable>subcommand</replaceable> --help - </literallayout> - </para> - </section> - - <section id='dev-viewing-dependencies-between-recipes-and-tasks'> - <title>Viewing Dependencies Between Recipes and Tasks</title> - - <para> - Sometimes it can be hard to see why BitBake wants to build other - recipes before the one you have specified. - Dependency information can help you understand why a recipe is - built. - </para> - - <para> - To generate dependency information for a recipe, run the - following command: - <literallayout class='monospaced'> - $ bitbake -g <replaceable>recipename</replaceable> - </literallayout> - This command writes the following files in the current - directory: - <itemizedlist> - <listitem><para> - <filename>pn-buildlist</filename>: A list of - recipes/targets involved in building - <replaceable>recipename</replaceable>. - "Involved" here means that at least one task from the - recipe needs to run when building - <replaceable>recipename</replaceable> from scratch. - Targets that are in - <ulink url='&YOCTO_DOCS_REF_URL;#var-ASSUME_PROVIDED'><filename>ASSUME_PROVIDED</filename></ulink> - are not listed. - </para></listitem> - <listitem><para> - <filename>task-depends.dot</filename>: A graph showing - dependencies between tasks. - </para></listitem> - </itemizedlist> - </para> - - <para> - The graphs are in - <ulink url='https://en.wikipedia.org/wiki/DOT_%28graph_description_language%29'>DOT</ulink> - format and can be converted to images (e.g. using the - <filename>dot</filename> tool from - <ulink url='http://www.graphviz.org/'>Graphviz</ulink>). - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - DOT files use a plain text format. - The graphs generated using the - <filename>bitbake -g</filename> command are often so - large as to be difficult to read without special - pruning (e.g. with Bitbake's - <filename>-I</filename> option) and processing. - Despite the form and size of the graphs, the - corresponding <filename>.dot</filename> files can - still be possible to read and provide useful - information. - </para> - - <para>As an example, the - <filename>task-depends.dot</filename> file contains - lines such as the following: - <literallayout class='monospaced'> - "libxslt.do_configure" -> "libxml2.do_populate_sysroot" - </literallayout> - The above example line reveals that the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-configure'><filename>do_configure</filename></ulink> - task in <filename>libxslt</filename> depends on the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-populate_sysroot'><filename>do_populate_sysroot</filename></ulink> - task in <filename>libxml2</filename>, which is a - normal - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEPENDS'><filename>DEPENDS</filename></ulink> - dependency between the two recipes. - </para></listitem> - <listitem><para> - For an example of how <filename>.dot</filename> - files can be processed, see the - <filename>scripts/contrib/graph-tool</filename> - Python script, which finds and displays paths - between graph nodes. - </para></listitem> - </itemizedlist> - </note> - </para> - - <para> - You can use a different method to view dependency information - by using the following command: - <literallayout class='monospaced'> - $ bitbake -g -u taskexp <replaceable>recipename</replaceable> - </literallayout> - This command displays a GUI window from which you can view - build-time and runtime dependencies for the recipes involved in - building <replaceable>recipename</replaceable>. - </para> - </section> - - <section id='dev-viewing-task-variable-dependencies'> - <title>Viewing Task Variable Dependencies</title> - - <para> - As mentioned in the - "<ulink url='&YOCTO_DOCS_BB_URL;#checksums'>Checksums (Signatures)</ulink>" - section of the BitBake User Manual, BitBake tries to - automatically determine what variables a task depends on so - that it can rerun the task if any values of the variables - change. - This determination is usually reliable. - However, if you do things like construct variable names at - runtime, then you might have to manually declare dependencies - on those variables using <filename>vardeps</filename> as - described in the - "<ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'>Variable Flags</ulink>" - section of the BitBake User Manual. - </para> - - <para> - If you are unsure whether a variable dependency is being - picked up automatically for a given task, you can list the - variable dependencies BitBake has determined by doing the - following: - <orderedlist> - <listitem><para> - Build the recipe containing the task: - <literallayout class='monospaced'> - $ bitbake <replaceable>recipename</replaceable> - </literallayout> - </para></listitem> - <listitem><para> - Inside the - <ulink url='&YOCTO_DOCS_REF_URL;#var-STAMPS_DIR'><filename>STAMPS_DIR</filename></ulink> - directory, find the signature data - (<filename>sigdata</filename>) file that corresponds - to the task. - The <filename>sigdata</filename> files contain a pickled - Python database of all the metadata that went into - creating the input checksum for the task. - As an example, for the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-fetch'><filename>do_fetch</filename></ulink> - task of the <filename>db</filename> recipe, the - <filename>sigdata</filename> file might be found in the - following location: - <literallayout class='monospaced'> - ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 - </literallayout> - For tasks that are accelerated through the shared state - (<ulink url='&YOCTO_DOCS_OM_URL;#shared-state-cache'>sstate</ulink>) - cache, an additional <filename>siginfo</filename> file - is written into - <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink> - along with the cached task output. - The <filename>siginfo</filename> files contain exactly - the same information as <filename>sigdata</filename> - files. - </para></listitem> - <listitem><para> - Run <filename>bitbake-dumpsig</filename> on the - <filename>sigdata</filename> or - <filename>siginfo</filename> file. - Here is an example: - <literallayout class='monospaced'> - $ bitbake-dumpsig ${BUILDDIR}/tmp/stamps/i586-poky-linux/db/6.0.30-r1.do_fetch.sigdata.7c048c18222b16ff0bcee2000ef648b1 - </literallayout> - In the output of the above command, you will find a - line like the following, which lists all the (inferred) - variable dependencies for the task. - This list also includes indirect dependencies from - variables depending on other variables, recursively. - <literallayout class='monospaced'> - Task dependencies: ['PV', 'SRCREV', 'SRC_URI', 'SRC_URI[md5sum]', 'SRC_URI[sha256sum]', 'base_do_fetch'] - </literallayout> - <note> - Functions (e.g. <filename>base_do_fetch</filename>) - also count as variable dependencies. - These functions in turn depend on the variables they - reference. - </note> - The output of <filename>bitbake-dumpsig</filename> also - includes the value each variable had, a list of - dependencies for each variable, and - <ulink url='&YOCTO_DOCS_BB_URL;#var-BB_HASHBASE_WHITELIST'><filename>BB_HASHBASE_WHITELIST</filename></ulink> - information. - </para></listitem> - </orderedlist> - </para> - - <para> - There is also a <filename>bitbake-diffsigs</filename> command - for comparing two <filename>siginfo</filename> or - <filename>sigdata</filename> files. - This command can be helpful when trying to figure out what - changed between two versions of a task. - If you call <filename>bitbake-diffsigs</filename> with just one - file, the command behaves like - <filename>bitbake-dumpsig</filename>. - </para> - - <para> - You can also use BitBake to dump out the signature construction - information without executing tasks by using either of the - following BitBake command-line options: - <literallayout class='monospaced'> - ‐‐dump-signatures=<replaceable>SIGNATURE_HANDLER</replaceable> - -S <replaceable>SIGNATURE_HANDLER</replaceable> - </literallayout> - <note> - Two common values for - <replaceable>SIGNATURE_HANDLER</replaceable> are "none" and - "printdiff", which dump only the signature or compare the - dumped signature with the cached one, respectively. - </note> - Using BitBake with either of these options causes BitBake to - dump out <filename>sigdata</filename> files in the - <filename>stamps</filename> directory for every task it would - have executed instead of building the specified target package. - </para> - </section> - - <section id='dev-viewing-metadata-used-to-create-the-input-signature-of-a-shared-state-task'> - <title>Viewing Metadata Used to Create the Input Signature of a Shared State Task</title> - - <para> - Seeing what metadata went into creating the input signature - of a shared state (sstate) task can be a useful debugging - aid. - This information is available in signature information - (<filename>siginfo</filename>) files in - <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink>. - For information on how to view and interpret information in - <filename>siginfo</filename> files, see the - "<link linkend='dev-viewing-task-variable-dependencies'>Viewing Task Variable Dependencies</link>" - section. - </para> - - <para> - For conceptual information on shared state, see the - "<ulink url='&YOCTO_DOCS_OM_URL;#shared-state'>Shared State</ulink>" - section in the Yocto Project Overview and Concepts Manual. - </para> - </section> - - <section id='dev-invalidating-shared-state-to-force-a-task-to-run'> - <title>Invalidating Shared State to Force a Task to Run</title> - - <para> - The OpenEmbedded build system uses - <ulink url='&YOCTO_DOCS_OM_URL;#overview-checksums'>checksums</ulink> - and - <ulink url='&YOCTO_DOCS_OM_URL;#shared-state'>shared state</ulink> - cache to avoid unnecessarily rebuilding tasks. - Collectively, this scheme is known as "shared state code." - </para> - - <para> - As with all schemes, this one has some drawbacks. - It is possible that you could make implicit changes to your - code that the checksum calculations do not take into - account. - These implicit changes affect a task's output but do not - trigger the shared state code into rebuilding a recipe. - Consider an example during which a tool changes its output. - Assume that the output of <filename>rpmdeps</filename> - changes. - The result of the change should be that all the - <filename>package</filename> and - <filename>package_write_rpm</filename> shared state cache - items become invalid. - However, because the change to the output is - external to the code and therefore implicit, - the associated shared state cache items do not become - invalidated. - In this case, the build process uses the cached items - rather than running the task again. - Obviously, these types of implicit changes can cause - problems. - </para> - - <para> - To avoid these problems during the build, you need to - understand the effects of any changes you make. - Realize that changes you make directly to a function - are automatically factored into the checksum calculation. - Thus, these explicit changes invalidate the associated - area of shared state cache. - However, you need to be aware of any implicit changes that - are not obvious changes to the code and could affect - the output of a given task. - </para> - - <para> - When you identify an implicit change, you can easily - take steps to invalidate the cache and force the tasks - to run. - The steps you can take are as simple as changing a - function's comments in the source code. - For example, to invalidate package shared state files, - change the comment statements of - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-package'><filename>do_package</filename></ulink> - or the comments of one of the functions it calls. - Even though the change is purely cosmetic, it causes the - checksum to be recalculated and forces the build system to - run the task again. - <note> - For an example of a commit that makes a cosmetic - change to invalidate shared state, see this - <ulink url='&YOCTO_GIT_URL;/cgit.cgi/poky/commit/meta/classes/package.bbclass?id=737f8bbb4f27b4837047cb9b4fbfe01dfde36d54'>commit</ulink>. - </note> - </para> - </section> - - <section id='dev-debugging-taskrunning'> - <title>Running Specific Tasks</title> - - <para> - Any given recipe consists of a set of tasks. - The standard BitBake behavior in most cases is: - <filename>do_fetch</filename>, - <filename>do_unpack</filename>, - <filename>do_patch</filename>, - <filename>do_configure</filename>, - <filename>do_compile</filename>, - <filename>do_install</filename>, - <filename>do_package</filename>, - <filename>do_package_write_*</filename>, and - <filename>do_build</filename>. - The default task is <filename>do_build</filename> and any tasks - on which it depends build first. - Some tasks, such as <filename>do_devshell</filename>, are not - part of the default build chain. - If you wish to run a task that is not part of the default build - chain, you can use the <filename>-c</filename> option in - BitBake. - Here is an example: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -c devshell - </literallayout> - </para> - - <para> - The <filename>-c</filename> option respects task dependencies, - which means that all other tasks (including tasks from other - recipes) that the specified task depends on will be run before - the task. - Even when you manually specify a task to run with - <filename>-c</filename>, BitBake will only run the task if it - considers it "out of date". - See the - "<ulink url='&YOCTO_DOCS_OM_URL;#stamp-files-and-the-rerunning-of-tasks'>Stamp Files and the Rerunning of Tasks</ulink>" - section in the Yocto Project Overview and Concepts Manual for - how BitBake determines whether a task is "out of date". - </para> - - <para> - If you want to force an up-to-date task to be rerun (e.g. - because you made manual modifications to the recipe's - <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink> - that you want to try out), then you can use the - <filename>-f</filename> option. - <note> - The reason <filename>-f</filename> is never required when - running the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-devshell'><filename>do_devshell</filename></ulink> - task is because the - <filename>[</filename><ulink url='&YOCTO_DOCS_BB_URL;#variable-flags'><filename>nostamp</filename></ulink><filename>]</filename> - variable flag is already set for the task. - </note> - The following example shows one way you can use the - <filename>-f</filename> option: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop - . - . - make some changes to the source code in the work directory - . - . - $ bitbake matchbox-desktop -c compile -f - $ bitbake matchbox-desktop - </literallayout> - </para> - - <para> - This sequence first builds and then recompiles - <filename>matchbox-desktop</filename>. - The last command reruns all tasks (basically the packaging - tasks) after the compile. - BitBake recognizes that the <filename>do_compile</filename> - task was rerun and therefore understands that the other tasks - also need to be run again. - </para> - - <para> - Another, shorter way to rerun a task and all - <ulink url='&YOCTO_DOCS_REF_URL;#normal-recipe-build-tasks'>normal recipe build tasks</ulink> - that depend on it is to use the <filename>-C</filename> - option. - <note> - This option is upper-cased and is separate from the - <filename>-c</filename> option, which is lower-cased. - </note> - Using this option invalidates the given task and then runs the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-build'><filename>do_build</filename></ulink> - task, which is the default task if no task is given, and the - tasks on which it depends. - You could replace the final two commands in the previous example - with the following single command: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -C compile - </literallayout> - Internally, the <filename>-f</filename> and - <filename>-C</filename> options work by tainting (modifying) the - input checksum of the specified task. - This tainting indirectly causes the task and its - dependent tasks to be rerun through the normal task dependency - mechanisms. - <note> - BitBake explicitly keeps track of which tasks have been - tainted in this fashion, and will print warnings such as the - following for builds involving such tasks: - <literallayout class='monospaced'> - WARNING: /home/ulf/poky/meta/recipes-sato/matchbox-desktop/matchbox-desktop_2.1.bb.do_compile is tainted from a forced run - </literallayout> - The purpose of the warning is to let you know that the work - directory and build output might not be in the clean state - they would be in for a "normal" build, depending on what - actions you took. - To get rid of such warnings, you can remove the work - directory and rebuild the recipe, as follows: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -c clean - $ bitbake matchbox-desktop - </literallayout> - </note> - </para> - - <para> - You can view a list of tasks in a given package by running the - <filename>do_listtasks</filename> task as follows: - <literallayout class='monospaced'> - $ bitbake matchbox-desktop -c listtasks - </literallayout> - The results appear as output to the console and are also in the - file <filename>${WORKDIR}/temp/log.do_listtasks</filename>. - </para> - </section> - - <section id='dev-debugging-bitbake'> - <title>General BitBake Problems</title> - - <para> - You can see debug output from BitBake by using the - <filename>-D</filename> option. - The debug output gives more information about what BitBake - is doing and the reason behind it. - Each <filename>-D</filename> option you use increases the - logging level. - The most common usage is <filename>-DDD</filename>. - </para> - - <para> - The output from - <filename>bitbake -DDD -v</filename> <replaceable>targetname</replaceable> - can reveal why BitBake chose a certain version of a package or - why BitBake picked a certain provider. - This command could also help you in a situation where you think - BitBake did something unexpected. - </para> - </section> - - <section id='dev-debugging-buildfile'> - <title>Building with No Dependencies</title> - - <para> - To build a specific recipe (<filename>.bb</filename> file), - you can use the following command form: - <literallayout class='monospaced'> - $ bitbake -b <replaceable>somepath</replaceable>/<replaceable>somerecipe</replaceable>.bb - </literallayout> - This command form does not check for dependencies. - Consequently, you should use it only when you know existing - dependencies have been met. - <note> - You can also specify fragments of the filename. - In this case, BitBake checks for a unique match. - </note> - </para> - </section> - - <section id='recipe-logging-mechanisms'> - <title>Recipe Logging Mechanisms</title> - - <para> - The Yocto Project provides several logging functions for - producing debugging output and reporting errors and warnings. - For Python functions, the following logging functions exist. - All of these functions log to - <filename>${T}/log.do_</filename><replaceable>task</replaceable>, - and can also log to standard output (stdout) with the right - settings: - <itemizedlist> - <listitem><para> - <filename>bb.plain(</filename><replaceable>msg</replaceable><filename>)</filename>: - Writes <replaceable>msg</replaceable> as is to the - log while also logging to stdout. - </para></listitem> - <listitem><para> - <filename>bb.note(</filename><replaceable>msg</replaceable><filename>)</filename>: - Writes "NOTE: <replaceable>msg</replaceable>" to the - log. - Also logs to stdout if BitBake is called with "-v". - </para></listitem> - <listitem><para> - <filename>bb.debug(</filename><replaceable>level</replaceable><filename>, </filename><replaceable>msg</replaceable><filename>)</filename>: - Writes "DEBUG: <replaceable>msg</replaceable>" to the - log. - Also logs to stdout if the log level is greater than or - equal to <replaceable>level</replaceable>. - See the - "<ulink url='&YOCTO_DOCS_BB_URL;#usage-and-syntax'>-D</ulink>" - option in the BitBake User Manual for more information. - </para></listitem> - <listitem><para> - <filename>bb.warn(</filename><replaceable>msg</replaceable><filename>)</filename>: - Writes "WARNING: <replaceable>msg</replaceable>" to the - log while also logging to stdout. - </para></listitem> - <listitem><para> - <filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename>: - Writes "ERROR: <replaceable>msg</replaceable>" to the - log while also logging to standard out (stdout). - <note> - Calling this function does not cause the task to fail. - </note> - </para></listitem> - <listitem><para> - <filename>bb.fatal(</filename><replaceable>msg</replaceable><filename>)</filename>: - This logging function is similar to - <filename>bb.error(</filename><replaceable>msg</replaceable><filename>)</filename> - but also causes the calling task to fail. - <note> - <filename>bb.fatal()</filename> raises an exception, - which means you do not need to put a "return" - statement after the function. - </note> - </para></listitem> - </itemizedlist> - </para> - - <para> - The same logging functions are also available in shell - functions, under the names - <filename>bbplain</filename>, <filename>bbnote</filename>, - <filename>bbdebug</filename>, <filename>bbwarn</filename>, - <filename>bberror</filename>, and <filename>bbfatal</filename>. - The - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-logging'><filename>logging</filename></ulink> - class implements these functions. - See that class in the - <filename>meta/classes</filename> folder of the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - for information. - </para> - - <section id='logging-with-python'> - <title>Logging With Python</title> - - <para> - When creating recipes using Python and inserting code that - handles build logs, keep in mind the goal is to have - informative logs while keeping the console as "silent" as - possible. - Also, if you want status messages in the log, use the - "debug" loglevel. - </para> - - <para> - Following is an example written in Python. - The code handles logging for a function that determines the - number of tasks needed to be run. - See the - "<ulink url='&YOCTO_DOCS_REF_URL;#ref-tasks-listtasks'><filename>do_listtasks</filename></ulink>" - section for additional information: - <literallayout class='monospaced'> - python do_listtasks() { - bb.debug(2, "Starting to figure out the task list") - if noteworthy_condition: - bb.note("There are 47 tasks to run") - bb.debug(2, "Got to point xyz") - if warning_trigger: - bb.warn("Detected warning_trigger, this might be a problem later.") - if recoverable_error: - bb.error("Hit recoverable_error, you really need to fix this!") - if fatal_error: - bb.fatal("fatal_error detected, unable to print the task list") - bb.plain("The tasks present are abc") - bb.debug(2, "Finished figuring out the tasklist") - } - </literallayout> - </para> - </section> - - <section id='logging-with-bash'> - <title>Logging With Bash</title> - - <para> - When creating recipes using Bash and inserting code that - handles build logs, you have the same goals - informative - with minimal console output. - The syntax you use for recipes written in Bash is similar - to that of recipes written in Python described in the - previous section. - </para> - - <para> - Following is an example written in Bash. - The code logs the progress of the <filename>do_my_function</filename> function. - <literallayout class='monospaced'> - do_my_function() { - bbdebug 2 "Running do_my_function" - if [ exceptional_condition ]; then - bbnote "Hit exceptional_condition" - fi - bbdebug 2 "Got to point xyz" - if [ warning_trigger ]; then - bbwarn "Detected warning_trigger, this might cause a problem later." - fi - if [ recoverable_error ]; then - bberror "Hit recoverable_error, correcting" - fi - if [ fatal_error ]; then - bbfatal "fatal_error detected" - fi - bbdebug 2 "Completed do_my_function" - } - </literallayout> - </para> - </section> - </section> - - <section id='debugging-parallel-make-races'> - <title>Debugging Parallel Make Races</title> - - <para> - A parallel <filename>make</filename> race occurs when the build - consists of several parts that are run simultaneously and - a situation occurs when the output or result of one - part is not ready for use with a different part of the build - that depends on that output. - Parallel make races are annoying and can sometimes be difficult - to reproduce and fix. - However, some simple tips and tricks exist that can help - you debug and fix them. - This section presents a real-world example of an error - encountered on the Yocto Project autobuilder and the process - used to fix it. - <note> - If you cannot properly fix a <filename>make</filename> race - condition, you can work around it by clearing either the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink> - or - <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKEINST'><filename>PARALLEL_MAKEINST</filename></ulink> - variables. - </note> - </para> - - <section id='the-failure'> - <title>The Failure</title> - - <para> - For this example, assume that you are building an image that - depends on the "neard" package. - And, during the build, BitBake runs into problems and - creates the following output. - <note> - This example log file has longer lines artificially - broken to make the listing easier to read. - </note> - If you examine the output or the log file, you see the - failure during <filename>make</filename>: - <literallayout class='monospaced'> - | DEBUG: SITE files ['endian-little', 'bit-32', 'ix86-common', 'common-linux', 'common-glibc', 'i586-linux', 'common'] - | DEBUG: Executing shell function do_compile - | NOTE: make -j 16 - | make --no-print-directory all-am - | /bin/mkdir -p include/near - | /bin/mkdir -p include/near - | /bin/mkdir -p include/near - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/types.h include/near/types.h - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/log.h include/near/log.h - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/plugin.h include/near/plugin.h - | /bin/mkdir -p include/near - | /bin/mkdir -p include/near - | /bin/mkdir -p include/near - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/tag.h include/near/tag.h - | /bin/mkdir -p include/near - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/adapter.h include/near/adapter.h - | /bin/mkdir -p include/near - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/ndef.h include/near/ndef.h - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/tlv.h include/near/tlv.h - | /bin/mkdir -p include/near - | /bin/mkdir -p include/near - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/setting.h include/near/setting.h - | /bin/mkdir -p include/near - | /bin/mkdir -p include/near - | /bin/mkdir -p include/near - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/device.h include/near/device.h - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/nfc_copy.h include/near/nfc_copy.h - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/snep.h include/near/snep.h - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/version.h include/near/version.h - | ln -s /home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/work/i586-poky-linux/neard/ - 0.14-r0/neard-0.14/include/dbus.h include/near/dbus.h - | ./src/genbuiltin nfctype1 nfctype2 nfctype3 nfctype4 p2p > src/builtin.h - | i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/ - build/build/tmp/sysroots/qemux86 -DHAVE_CONFIG_H -I. -I./include -I./src -I./gdbus -I/home/pokybuild/ - yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/glib-2.0 - -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/ - lib/glib-2.0/include -I/home/pokybuild/yocto-autobuilder/yocto-slave/nightly-x86/build/build/ - tmp/sysroots/qemux86/usr/include/dbus-1.0 -I/home/pokybuild/yocto-autobuilder/yocto-slave/ - nightly-x86/build/build/tmp/sysroots/qemux86/usr/lib/dbus-1.0/include -I/home/pokybuild/yocto-autobuilder/ - yocto-slave/nightly-x86/build/build/tmp/sysroots/qemux86/usr/include/libnl3 - -DNEAR_PLUGIN_BUILTIN -DPLUGINDIR=\""/usr/lib/near/plugins"\" - -DCONFIGDIR=\""/etc/neard\"" -O2 -pipe -g -feliminate-unused-debug-types -c - -o tools/snep-send.o tools/snep-send.c - | In file included from tools/snep-send.c:16:0: - | tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory - | #include <near/dbus.h> - | ^ - | compilation terminated. - | make[1]: *** [tools/snep-send.o] Error 1 - | make[1]: *** Waiting for unfinished jobs.... - | make: *** [all] Error 2 - | ERROR: oe_runmake failed - </literallayout> - </para> - </section> - - <section id='reproducing-the-error'> - <title>Reproducing the Error</title> - - <para> - Because race conditions are intermittent, they do not - manifest themselves every time you do the build. - In fact, most times the build will complete without problems - even though the potential race condition exists. - Thus, once the error surfaces, you need a way to reproduce - it. - </para> - - <para> - In this example, compiling the "neard" package is causing - the problem. - So the first thing to do is build "neard" locally. - Before you start the build, set the - <ulink url='&YOCTO_DOCS_REF_URL;#var-PARALLEL_MAKE'><filename>PARALLEL_MAKE</filename></ulink> - variable in your <filename>local.conf</filename> file to - a high number (e.g. "-j 20"). - Using a high value for <filename>PARALLEL_MAKE</filename> - increases the chances of the race condition showing up: - <literallayout class='monospaced'> - $ bitbake neard - </literallayout> - </para> - - <para> - Once the local build for "neard" completes, start a - <filename>devshell</filename> build: - <literallayout class='monospaced'> - $ bitbake neard -c devshell - </literallayout> - For information on how to use a - <filename>devshell</filename>, see the - "<link linkend='platdev-appdev-devshell'>Using a Development Shell</link>" - section. - </para> - - <para> - In the <filename>devshell</filename>, do the following: - <literallayout class='monospaced'> - $ make clean - $ make tools/snep-send.o - </literallayout> - The <filename>devshell</filename> commands cause the failure - to clearly be visible. - In this case, a missing dependency exists for the "neard" - Makefile target. - Here is some abbreviated, sample output with the - missing dependency clearly visible at the end: - <literallayout class='monospaced'> - i586-poky-linux-gcc -m32 -march=i586 --sysroot=/home/scott-lenovo/...... - . - . - . - tools/snep-send.c - In file included from tools/snep-send.c:16:0: - tools/../src/near.h:41:23: fatal error: near/dbus.h: No such file or directory - #include <near/dbus.h> - ^ - compilation terminated. - make: *** [tools/snep-send.o] Error 1 - $ - </literallayout> - </para> - </section> - - <section id='creating-a-patch-for-the-fix'> - <title>Creating a Patch for the Fix</title> - - <para> - Because there is a missing dependency for the Makefile - target, you need to patch the - <filename>Makefile.am</filename> file, which is generated - from <filename>Makefile.in</filename>. - You can use Quilt to create the patch: - <literallayout class='monospaced'> - $ quilt new parallelmake.patch - Patch patches/parallelmake.patch is now on top - $ quilt add Makefile.am - File Makefile.am added to patch patches/parallelmake.patch - </literallayout> - For more information on using Quilt, see the - "<link linkend='using-a-quilt-workflow'>Using Quilt in Your Workflow</link>" - section. - </para> - - <para> - At this point you need to make the edits to - <filename>Makefile.am</filename> to add the missing - dependency. - For our example, you have to add the following line - to the file: - <literallayout class='monospaced'> - tools/snep-send.$(OBJEXT): include/near/dbus.h - </literallayout> - </para> - - <para> - Once you have edited the file, use the - <filename>refresh</filename> command to create the patch: - <literallayout class='monospaced'> - $ quilt refresh - Refreshed patch patches/parallelmake.patch - </literallayout> - Once the patch file exists, you need to add it back to the - originating recipe folder. - Here is an example assuming a top-level - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - named <filename>poky</filename>: - <literallayout class='monospaced'> - $ cp patches/parallelmake.patch poky/meta/recipes-connectivity/neard/neard - </literallayout> - The final thing you need to do to implement the fix in the - build is to update the "neard" recipe (i.e. - <filename>neard-0.14.bb</filename>) so that the - <ulink url='&YOCTO_DOCS_REF_URL;#var-SRC_URI'><filename>SRC_URI</filename></ulink> - statement includes the patch file. - The recipe file is in the folder above the patch. - Here is what the edited <filename>SRC_URI</filename> - statement would look like: - <literallayout class='monospaced'> - SRC_URI = "${KERNELORG_MIRROR}/linux/network/nfc/${BPN}-${PV}.tar.xz \ - file://neard.in \ - file://neard.service.in \ - file://parallelmake.patch \ - " - </literallayout> - </para> - - <para> - With the patch complete and moved to the correct folder and - the <filename>SRC_URI</filename> statement updated, you can - exit the <filename>devshell</filename>: - <literallayout class='monospaced'> - $ exit - </literallayout> - </para> - </section> - - <section id='testing-the-build'> - <title>Testing the Build</title> - - <para> - With everything in place, you can get back to trying the - build again locally: - <literallayout class='monospaced'> - $ bitbake neard - </literallayout> - This build should succeed. - </para> - - <para> - Now you can open up a <filename>devshell</filename> again - and repeat the clean and make operations as follows: - <literallayout class='monospaced'> - $ bitbake neard -c devshell - $ make clean - $ make tools/snep-send.o - </literallayout> - The build should work without issue. - </para> - - <para> - As with all solved problems, if they originated upstream, - you need to submit the fix for the recipe in OE-Core and - upstream so that the problem is taken care of at its - source. - See the - "<link linkend='how-to-submit-a-change'>Submitting a Change to the Yocto Project</link>" - section for more information. - </para> - </section> - </section> - - <section id="platdev-gdb-remotedebug"> - <title>Debugging With the GNU Project Debugger (GDB) Remotely</title> - - <para> - GDB allows you to examine running programs, which in turn helps - you to understand and fix problems. - It also allows you to perform post-mortem style analysis of - program crashes. - GDB is available as a package within the Yocto Project and is - installed in SDK images by default. - See the - "<ulink url='&YOCTO_DOCS_REF_URL;#ref-images'>Images</ulink>" - chapter in the Yocto Project Reference Manual for a description of - these images. - You can find information on GDB at - <ulink url="http://sourceware.org/gdb/"/>. - <note><title>Tip</title> - For best results, install debug (<filename>-dbg</filename>) - packages for the applications you are going to debug. - Doing so makes extra debug symbols available that give you - more meaningful output. - </note> - </para> - - <para> - Sometimes, due to memory or disk space constraints, it is not - possible to use GDB directly on the remote target to debug - applications. - These constraints arise because GDB needs to load the debugging - information and the binaries of the process being debugged. - Additionally, GDB needs to perform many computations to locate - information such as function names, variable names and values, - stack traces and so forth - even before starting the debugging - process. - These extra computations place more load on the target system - and can alter the characteristics of the program being debugged. - </para> - - <para> - To help get past the previously mentioned constraints, you can - use gdbserver, which runs on the remote target and does not - load any debugging information from the debugged process. - Instead, a GDB instance processes the debugging information that - is run on a remote computer - the host GDB. - The host GDB then sends control commands to gdbserver to make - it stop or start the debugged program, as well as read or write - memory regions of that debugged program. - All the debugging information loaded and processed as well - as all the heavy debugging is done by the host GDB. - Offloading these processes gives the gdbserver running on the - target a chance to remain small and fast. - </para> - - <para> - Because the host GDB is responsible for loading the debugging - information and for doing the necessary processing to make - actual debugging happen, you have to make sure the host can - access the unstripped binaries complete with their debugging - information and also be sure the target is compiled with no - optimizations. - The host GDB must also have local access to all the libraries - used by the debugged program. - Because gdbserver does not need any local debugging information, - the binaries on the remote target can remain stripped. - However, the binaries must also be compiled without optimization - so they match the host's binaries. - </para> - - <para> - To remain consistent with GDB documentation and terminology, - the binary being debugged on the remote target machine is - referred to as the "inferior" binary. - For documentation on GDB see the - <ulink url="http://sourceware.org/gdb/documentation/">GDB site</ulink>. - </para> - - <para> - The following steps show you how to debug using the GNU project - debugger. - <orderedlist> - <listitem><para> - <emphasis>Configure your build system to construct the - companion debug filesystem:</emphasis></para> - - <para>In your <filename>local.conf</filename> file, set - the following: - <literallayout class='monospaced'> - IMAGE_GEN_DEBUGFS = "1" - IMAGE_FSTYPES_DEBUGFS = "tar.bz2" - </literallayout> - These options cause the OpenEmbedded build system - to generate a special companion filesystem fragment, - which contains the matching source and debug symbols to - your deployable filesystem. - The build system does this by looking at what is in the - deployed filesystem, and pulling the corresponding - <filename>-dbg</filename> packages.</para> - - <para>The companion debug filesystem is not a complete - filesystem, but only contains the debug fragments. - This filesystem must be combined with the full filesystem - for debugging. - Subsequent steps in this procedure show how to combine - the partial filesystem with the full filesystem. - </para></listitem> - <listitem><para> - <emphasis>Configure the system to include gdbserver in - the target filesystem:</emphasis></para> - - <para>Make the following addition in either your - <filename>local.conf</filename> file or in an image - recipe: - <literallayout class='monospaced'> - IMAGE_INSTALL_append = “ gdbserver" - </literallayout> - The change makes sure the <filename>gdbserver</filename> - package is included. - </para></listitem> - <listitem><para> - <emphasis>Build the environment:</emphasis></para> - - <para>Use the following command to construct the image - and the companion Debug Filesystem: - <literallayout class='monospaced'> - $ bitbake <replaceable>image</replaceable> - </literallayout> - Build the cross GDB component and make it available - for debugging. - Build the SDK that matches the image. - Building the SDK is best for a production build - that can be used later for debugging, especially - during long term maintenance: - <literallayout class='monospaced'> - $ bitbake -c populate_sdk <replaceable>image</replaceable> - </literallayout></para> - - <para>Alternatively, you can build the minimal - toolchain components that match the target. - Doing so creates a smaller than typical SDK and only - contains a minimal set of components with which to - build simple test applications, as well as run the - debugger: - <literallayout class='monospaced'> - $ bitbake meta-toolchain - </literallayout></para> - - <para>A final method is to build Gdb itself within - the build system: - <literallayout class='monospaced'> - $ bitbake gdb-cross-<replaceable>architecture</replaceable> - </literallayout> - Doing so produces a temporary copy of - <filename>cross-gdb</filename> you can use for - debugging during development. - While this is the quickest approach, the two previous - methods in this step are better when considering - long-term maintenance strategies. - <note> - If you run - <filename>bitbake gdb-cross</filename>, the - OpenEmbedded build system suggests the actual - image (e.g. <filename>gdb-cross-i586</filename>). - The suggestion is usually the actual name you want - to use. - </note> - </para></listitem> - <listitem><para> - <emphasis>Set up the</emphasis> <filename>debugfs</filename></para> - - <para>Run the following commands to set up the - <filename>debugfs</filename>: - <literallayout class='monospaced'> - $ mkdir debugfs - $ cd debugfs - $ tar xvfj <replaceable>build-dir</replaceable>/tmp-glibc/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>.rootfs.tar.bz2 - $ tar xvfj <replaceable>build-dir</replaceable>/tmp-glibc/deploy/images/<replaceable>machine</replaceable>/<replaceable>image</replaceable>-dbg.rootfs.tar.bz2 - </literallayout> - </para></listitem> - <listitem><para> - <emphasis>Set up GDB</emphasis></para> - - <para>Install the SDK (if you built one) and then - source the correct environment file. - Sourcing the environment file puts the SDK in your - <filename>PATH</filename> environment variable.</para> - - <para>If you are using the build system, Gdb is - located in - <replaceable>build-dir</replaceable>/tmp/sysroots/<replaceable>host</replaceable>/usr/bin/<replaceable>architecture</replaceable>/<replaceable>architecture</replaceable>-gdb - </para></listitem> - <listitem><para> - <emphasis>Boot the target:</emphasis></para> - - <para>For information on how to run QEMU, see the - <ulink url='http://wiki.qemu.org/Documentation/GettingStartedDevelopers'>QEMU Documentation</ulink>. - <note> - Be sure to verify that your host can access the - target via TCP. - </note> - </para></listitem> - <listitem><para> - <emphasis>Debug a program:</emphasis></para> - - <para>Debugging a program involves running gdbserver - on the target and then running Gdb on the host. - The example in this step debugs - <filename>gzip</filename>: - <literallayout class='monospaced'> - root@qemux86:~# gdbserver localhost:1234 /bin/gzip —help - </literallayout> - For additional gdbserver options, see the - <ulink url='https://www.gnu.org/software/gdb/documentation/'>GDB Server Documentation</ulink>. - </para> - - <para>After running gdbserver on the target, you need - to run Gdb on the host and configure it and connect to - the target. - Use these commands: - <literallayout class='monospaced'> - $ cd <replaceable>directory-holding-the-debugfs-directory</replaceable> - $ <replaceable>arch</replaceable>-gdb - - (gdb) set sysroot debugfs - (gdb) set substitute-path /usr/src/debug debugfs/usr/src/debug - (gdb) target remote <replaceable>IP-of-target</replaceable>:1234 - </literallayout> - At this point, everything should automatically load - (i.e. matching binaries, symbols and headers). - <note> - The Gdb <filename>set</filename> commands in the - previous example can be placed into the users - <filename>~/.gdbinit</filename> file. - Upon starting, Gdb automatically runs whatever - commands are in that file. - </note> - </para></listitem> - <listitem><para> - <emphasis>Deploying without a full image - rebuild:</emphasis></para> - - <para>In many cases, during development you want a - quick method to deploy a new binary to the target and - debug it, without waiting for a full image build. - </para> - - <para>One approach to solving this situation is to - just build the component you want to debug. - Once you have built the component, copy the - executable directly to both the target and the - host <filename>debugfs</filename>.</para> - - <para>If the binary is processed through the debug - splitting in OpenEmbedded, you should also - copy the debug items (i.e. <filename>.debug</filename> - contents and corresponding - <filename>/usr/src/debug</filename> files) - from the work directory. - Here is an example: - <literallayout class='monospaced'> - $ bitbake bash - $ bitbake -c devshell bash - $ cd .. - $ scp packages-split/bash/bin/bash <replaceable>target</replaceable>:/bin/bash - $ cp -a packages-split/bash-dbg/* <replaceable>path</replaceable>/debugfs - </literallayout> - </para></listitem> - </orderedlist> - </para> - </section> - - <section id='debugging-with-the-gnu-project-debugger-gdb-on-the-target'> - <title>Debugging with the GNU Project Debugger (GDB) on the Target</title> - - <para> - The previous section addressed using GDB remotely for debugging - purposes, which is the most usual case due to the inherent - hardware limitations on many embedded devices. - However, debugging in the target hardware itself is also - possible with more powerful devices. - This section describes what you need to do in order to support - using GDB to debug on the target hardware. - </para> - - <para> - To support this kind of debugging, you need do the following: - <itemizedlist> - <listitem><para> - Ensure that GDB is on the target. - You can do this by adding "gdb" to - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_INSTALL'><filename>IMAGE_INSTALL</filename></ulink>: - <literallayout class='monospaced'> - IMAGE_INSTALL_append = " gdb" - </literallayout> - Alternatively, you can add "tools-debug" to - <ulink url='&YOCTO_DOCS_REF_URL;#var-IMAGE_FEATURES'><filename>IMAGE_FEATURES</filename></ulink>: - <literallayout class='monospaced'> - IMAGE_FEATURES_append = " tools-debug" - </literallayout> - </para></listitem> - <listitem><para> - Ensure that debug symbols are present. - You can make sure these symbols are present by - installing <filename>-dbg</filename>: - <literallayout class='monospaced'> - IMAGE_INSTALL_append = " <replaceable>packagename</replaceable>-dbg" - </literallayout> - Alternatively, you can do the following to include all - the debug symbols: - <literallayout class='monospaced'> - IMAGE_FEATURES_append = " dbg-pkgs" - </literallayout> - </para></listitem> - </itemizedlist> - <note> - To improve the debug information accuracy, you can reduce - the level of optimization used by the compiler. - For example, when adding the following line to your - <filename>local.conf</filename> file, you will reduce - optimization from - <ulink url='&YOCTO_DOCS_REF_URL;#var-FULL_OPTIMIZATION'><filename>FULL_OPTIMIZATION</filename></ulink> - of "-O2" to - <ulink url='&YOCTO_DOCS_REF_URL;#var-DEBUG_OPTIMIZATION'><filename>DEBUG_OPTIMIZATION</filename></ulink> - of "-O -fno-omit-frame-pointer": - <literallayout class='monospaced'> - DEBUG_BUILD = "1" - </literallayout> - Consider that this will reduce the application's performance - and is recommended only for debugging purposes. - </note> - </para> - </section> - - <section id='dev-other-debugging-others'> - <title>Other Debugging Tips</title> - - <para> - Here are some other tips that you might find useful: - <itemizedlist> - <listitem><para> - When adding new packages, it is worth watching for - undesirable items making their way into compiler command - lines. - For example, you do not want references to local system - files like - <filename>/usr/lib/</filename> or - <filename>/usr/include/</filename>. - </para></listitem> - <listitem><para> - If you want to remove the <filename>psplash</filename> - boot splashscreen, - add <filename>psplash=false</filename> to the kernel - command line. - Doing so prevents <filename>psplash</filename> from - loading and thus allows you to see the console. - It is also possible to switch out of the splashscreen by - switching the virtual console (e.g. Fn+Left or Fn+Right - on a Zaurus). - </para></listitem> - <listitem><para> - Removing - <ulink url='&YOCTO_DOCS_REF_URL;#var-TMPDIR'><filename>TMPDIR</filename></ulink> - (usually <filename>tmp/</filename>, within the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>) - can often fix temporary build issues. - Removing <filename>TMPDIR</filename> is usually a - relatively cheap operation, because task output will be - cached in - <ulink url='&YOCTO_DOCS_REF_URL;#var-SSTATE_DIR'><filename>SSTATE_DIR</filename></ulink> - (usually <filename>sstate-cache/</filename>, which is - also in the Build Directory). - <note> - Removing <filename>TMPDIR</filename> might be a - workaround rather than a fix. - Consequently, trying to determine the underlying - cause of an issue before removing the directory is - a good idea. - </note> - </para></listitem> - <listitem><para> - Understanding how a feature is used in practice within - existing recipes can be very helpful. - It is recommended that you configure some method that - allows you to quickly search through files.</para> - - <para>Using GNU Grep, you can use the following shell - function to recursively search through common - recipe-related files, skipping binary files, - <filename>.git</filename> directories, and the - Build Directory (assuming its name starts with - "build"): - <literallayout class='monospaced'> - g() { - grep -Ir \ - --exclude-dir=.git \ - --exclude-dir='build*' \ - --include='*.bb*' \ - --include='*.inc*' \ - --include='*.conf*' \ - --include='*.py*' \ - "$@" - } - </literallayout> - Following are some usage examples: - <literallayout class='monospaced'> - $ g FOO # Search recursively for "FOO" - $ g -i foo # Search recursively for "foo", ignoring case - $ g -w FOO # Search recursively for "FOO" as a word, ignoring e.g. "FOOBAR" - </literallayout> - If figuring out how some feature works requires a lot of - searching, it might indicate that the documentation - should be extended or improved. - In such cases, consider filing a documentation bug using - the Yocto Project implementation of - <ulink url='https://bugzilla.yoctoproject.org/'>Bugzilla</ulink>. - For information on how to submit a bug against - the Yocto Project, see the Yocto Project Bugzilla - <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>wiki page</ulink> - and the - "<link linkend='submitting-a-defect-against-the-yocto-project'>Submitting a Defect Against the Yocto Project</link>" - section. - <note> - The manuals might not be the right place to document - variables that are purely internal and have a - limited scope (e.g. internal variables used to - implement a single <filename>.bbclass</filename> - file). - </note> - </para></listitem> - </itemizedlist> - </para> - </section> - </section> - - <section id='making-changes-to-the-yocto-project'> - <title>Making Changes to the Yocto Project</title> - - <para> - Because the Yocto Project is an open-source, community-based - project, you can effect changes to the project. - This section presents procedures that show you how to submit - a defect against the project and how to submit a change. - </para> - - <section id='submitting-a-defect-against-the-yocto-project'> - <title>Submitting a Defect Against the Yocto Project</title> - - <para> - Use the Yocto Project implementation of - <ulink url='http://www.bugzilla.org/about/'>Bugzilla</ulink> - to submit a defect (bug) against the Yocto Project. - For additional information on this implementation of Bugzilla see the - "<ulink url='&YOCTO_DOCS_REF_URL;#resources-bugtracker'>Yocto Project Bugzilla</ulink>" - section in the Yocto Project Reference Manual. - For more detail on any of the following steps, see the Yocto Project - <ulink url='&YOCTO_WIKI_URL;/wiki/Bugzilla_Configuration_and_Bug_Tracking'>Bugzilla wiki page</ulink>. - </para> - - <para> - Use the following general steps to submit a bug" - - <orderedlist> - <listitem><para> - Open the Yocto Project implementation of - <ulink url='&YOCTO_BUGZILLA_URL;'>Bugzilla</ulink>. - </para></listitem> - <listitem><para> - Click "File a Bug" to enter a new bug. - </para></listitem> - <listitem><para> - Choose the appropriate "Classification", "Product", and - "Component" for which the bug was found. - Bugs for the Yocto Project fall into one of several - classifications, which in turn break down into several - products and components. - For example, for a bug against the - <filename>meta-intel</filename> layer, you would choose - "Build System, Metadata & Runtime", "BSPs", and - "bsps-meta-intel", respectively. - </para></listitem> - <listitem><para> - Choose the "Version" of the Yocto Project for which you found - the bug (e.g. &DISTRO;). - </para></listitem> - <listitem><para> - Determine and select the "Severity" of the bug. - The severity indicates how the bug impacted your work. - </para></listitem> - <listitem><para> - Choose the "Hardware" that the bug impacts. - </para></listitem> - <listitem><para> - Choose the "Architecture" that the bug impacts. - </para></listitem> - <listitem><para> - Choose a "Documentation change" item for the bug. - Fixing a bug might or might not affect the Yocto Project - documentation. - If you are unsure of the impact to the documentation, select - "Don't Know". - </para></listitem> - <listitem><para> - Provide a brief "Summary" of the bug. - Try to limit your summary to just a line or two and be sure - to capture the essence of the bug. - </para></listitem> - <listitem><para> - Provide a detailed "Description" of the bug. - You should provide as much detail as you can about the context, - behavior, output, and so forth that surrounds the bug. - You can even attach supporting files for output from logs by - using the "Add an attachment" button. - </para></listitem> - <listitem><para> - Click the "Submit Bug" button submit the bug. - A new Bugzilla number is assigned to the bug and the defect - is logged in the bug tracking system. - </para></listitem> - </orderedlist> - Once you file a bug, the bug is processed by the Yocto Project Bug - Triage Team and further details concerning the bug are assigned - (e.g. priority and owner). - You are the "Submitter" of the bug and any further categorization, - progress, or comments on the bug result in Bugzilla sending you an - automated email concerning the particular change or progress to the - bug. - </para> - </section> - - <section id='how-to-submit-a-change'> - <title>Submitting a Change to the Yocto Project</title> - - <para> - Contributions to the Yocto Project and OpenEmbedded are very welcome. - Because the system is extremely configurable and flexible, we recognize - that developers will want to extend, configure or optimize it for - their specific uses. - </para> - - <para> - The Yocto Project uses a mailing list and a patch-based workflow - that is similar to the Linux kernel but contains important - differences. - In general, a mailing list exists through which you can submit - patches. - You should send patches to the appropriate mailing list so that they - can be reviewed and merged by the appropriate maintainer. - The specific mailing list you need to use depends on the - location of the code you are changing. - Each component (e.g. layer) should have a - <filename>README</filename> file that indicates where to send - the changes and which process to follow. - </para> - - <para> - You can send the patch to the mailing list using whichever approach - you feel comfortable with to generate the patch. - Once sent, the patch is usually reviewed by the community at large. - If somebody has concerns with the patch, they will usually voice - their concern over the mailing list. - If a patch does not receive any negative reviews, the maintainer of - the affected layer typically takes the patch, tests it, and then - based on successful testing, merges the patch. - </para> - - <para id='figuring-out-the-mailing-list-to-use'> - The "poky" repository, which is the Yocto Project's reference build - environment, is a hybrid repository that contains several - individual pieces (e.g. BitBake, Metadata, documentation, - and so forth) built using the combo-layer tool. - The upstream location used for submitting changes varies by - component: - <itemizedlist> - <listitem><para> - <emphasis>Core Metadata:</emphasis> - Send your patch to the - <ulink url='http://lists.openembedded.org/mailman/listinfo/openembedded-core'>openembedded-core</ulink> - mailing list. For example, a change to anything under - the <filename>meta</filename> or - <filename>scripts</filename> directories should be sent - to this mailing list. - </para></listitem> - <listitem><para> - <emphasis>BitBake:</emphasis> - For changes to BitBake (i.e. anything under the - <filename>bitbake</filename> directory), send your patch - to the - <ulink url='http://lists.openembedded.org/mailman/listinfo/bitbake-devel'>bitbake-devel</ulink> - mailing list. - </para></listitem> - <listitem><para> - <emphasis>"meta-*" trees:</emphasis> - These trees contain Metadata. - Use the - <ulink url='https://lists.yoctoproject.org/listinfo/poky'>poky</ulink> - mailing list. - </para></listitem> - </itemizedlist> - </para> - - <para> - For changes to other layers hosted in the Yocto Project source - repositories (i.e. <filename>yoctoproject.org</filename>), tools, - and the Yocto Project documentation, use the - <ulink url='https://lists.yoctoproject.org/listinfo/yocto'>Yocto Project</ulink> - general mailing list. - <note> - Sometimes a layer's documentation specifies to use a - particular mailing list. - If so, use that list. - </note> - For additional recipes that do not fit into the core Metadata, you - should determine which layer the recipe should go into and submit - the change in the manner recommended by the documentation (e.g. - the <filename>README</filename> file) supplied with the layer. - If in doubt, please ask on the Yocto general mailing list or on - the openembedded-devel mailing list. - </para> - - <para> - You can also push a change upstream and request a maintainer to - pull the change into the component's upstream repository. - You do this by pushing to a contribution repository that is upstream. - See the - "<ulink url='&YOCTO_DOCS_OM_URL;#gs-git-workflows-and-the-yocto-project'>Git Workflows and the Yocto Project</ulink>" - section in the Yocto Project Overview and Concepts Manual for additional - concepts on working in the Yocto Project development environment. - </para> - - <para> - Two commonly used testing repositories exist for - OpenEmbedded-Core: - <itemizedlist> - <listitem><para> - <emphasis>"ross/mut" branch:</emphasis> - The "mut" (master-under-test) tree - exists in the <filename>poky-contrib</filename> repository - in the - <ulink url='&YOCTO_GIT_URL;'>Yocto Project source repositories</ulink>. - </para></listitem> - <listitem><para> - <emphasis>"master-next" branch:</emphasis> - This branch is part of the main - "poky" repository in the Yocto Project source repositories. - </para></listitem> - </itemizedlist> - Maintainers use these branches to test submissions prior to merging - patches. - Thus, you can get an idea of the status of a patch based on - whether the patch has been merged into one of these branches. - <note> - This system is imperfect and changes can sometimes get lost in the - flow. - Asking about the status of a patch or change is reasonable if the - change has been idle for a while with no feedback. - The Yocto Project does have plans to use - <ulink url='https://en.wikipedia.org/wiki/Patchwork_(software)'>Patchwork</ulink> - to track the status of patches and also to automatically preview - patches. - </note> - </para> - - <para> - The following sections provide procedures for submitting a change. - </para> - - <section id='pushing-a-change-upstream'> - <title>Using Scripts to Push a Change Upstream and Request a Pull</title> - - <para> - Follow this procedure to push a change to an upstream "contrib" - Git repository: - <note> - You can find general Git information on how to push a change - upstream in the - <ulink url='http://git-scm.com/book/en/v2/Distributed-Git-Distributed-Workflows'>Git Community Book</ulink>. - </note> - <orderedlist> - <listitem><para> - <emphasis>Make Your Changes Locally:</emphasis> - Make your changes in your local Git repository. - You should make small, controlled, isolated changes. - Keeping changes small and isolated aids review, - makes merging/rebasing easier and keeps the change - history clean should anyone need to refer to it in - future. - </para></listitem> - <listitem><para> - <emphasis>Stage Your Changes:</emphasis> - Stage your changes by using the <filename>git add</filename> - command on each file you changed. - </para></listitem> - <listitem><para id='making-sure-you-have-correct-commit-information'> - <emphasis>Commit Your Changes:</emphasis> - Commit the change by using the - <filename>git commit</filename> command. - Make sure your commit information follows standards by - following these accepted conventions: - <itemizedlist> - <listitem><para> - Be sure to include a "Signed-off-by:" line in the - same style as required by the Linux kernel. - Adding this line signifies that you, the submitter, - have agreed to the Developer's Certificate of - Origin 1.1 as follows: - <literallayout class='monospaced'> - Developer's Certificate of Origin 1.1 - - By making a contribution to this project, I certify that: - - (a) The contribution was created in whole or in part by me and I - have the right to submit it under the open source license - indicated in the file; or - - (b) The contribution is based upon previous work that, to the best - of my knowledge, is covered under an appropriate open source - license and I have the right under that license to submit that - work with modifications, whether created in whole or in part - by me, under the same open source license (unless I am - permitted to submit under a different license), as indicated - in the file; or - - (c) The contribution was provided directly to me by some other - person who certified (a), (b) or (c) and I have not modified - it. - - (d) I understand and agree that this project and the contribution - are public and that a record of the contribution (including all - personal information I submit with it, including my sign-off) is - maintained indefinitely and may be redistributed consistent with - this project or the open source license(s) involved. - </literallayout> - </para></listitem> - <listitem><para> - Provide a single-line summary of the change. - and, - if more explanation is needed, provide more - detail in the body of the commit. - This summary is typically viewable in the - "shortlist" of changes. - Thus, providing something short and descriptive - that gives the reader a summary of the change is - useful when viewing a list of many commits. - You should prefix this short description with the - recipe name (if changing a recipe), or else with - the short form path to the file being changed. - </para></listitem> - <listitem><para> - For the body of the commit message, provide - detailed information that describes what you - changed, why you made the change, and the approach - you used. - It might also be helpful if you mention how you - tested the change. - Provide as much detail as you can in the body of - the commit message. - <note> - You do not need to provide a more detailed - explanation of a change if the change is - minor to the point of the single line - summary providing all the information. - </note> - </para></listitem> - <listitem><para> - If the change addresses a specific bug or issue - that is associated with a bug-tracking ID, - include a reference to that ID in your detailed - description. - For example, the Yocto Project uses a specific - convention for bug references - any commit that - addresses a specific bug should use the following - form for the detailed description. - Be sure to use the actual bug-tracking ID from - Bugzilla for - <replaceable>bug-id</replaceable>: - <literallayout class='monospaced'> - Fixes [YOCTO #<replaceable>bug-id</replaceable>] - - <replaceable>detailed description of change</replaceable> - </literallayout> - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <emphasis>Push Your Commits to a "Contrib" Upstream:</emphasis> - If you have arranged for permissions to push to an - upstream contrib repository, push the change to that - repository: - <literallayout class='monospaced'> - $ git push <replaceable>upstream_remote_repo</replaceable> <replaceable>local_branch_name</replaceable> - </literallayout> - For example, suppose you have permissions to push into the - upstream <filename>meta-intel-contrib</filename> - repository and you are working in a local branch named - <replaceable>your_name</replaceable><filename>/README</filename>. - The following command pushes your local commits to the - <filename>meta-intel-contrib</filename> upstream - repository and puts the commit in a branch named - <replaceable>your_name</replaceable><filename>/README</filename>: - <literallayout class='monospaced'> - $ git push meta-intel-contrib <replaceable>your_name</replaceable>/README - </literallayout> - </para></listitem> - <listitem><para id='push-determine-who-to-notify'> - <emphasis>Determine Who to Notify:</emphasis> - Determine the maintainer or the mailing list - that you need to notify for the change.</para> - - <para>Before submitting any change, you need to be sure - who the maintainer is or what mailing list that you need - to notify. - Use either these methods to find out: - <itemizedlist> - <listitem><para> - <emphasis>Maintenance File:</emphasis> - Examine the <filename>maintainers.inc</filename> - file, which is located in the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - at - <filename>meta/conf/distro/include</filename>, - to see who is responsible for code. - </para></listitem> - <listitem><para> - <emphasis>Search by File:</emphasis> - Using <ulink url='&YOCTO_DOCS_OM_URL;#git'>Git</ulink>, - you can enter the following command to bring up a - short list of all commits against a specific file: - <literallayout class='monospaced'> - git shortlog -- <replaceable>filename</replaceable> - </literallayout> - Just provide the name of the file for which you - are interested. - The information returned is not ordered by history - but does include a list of everyone who has - committed grouped by name. - From the list, you can see who is responsible for - the bulk of the changes against the file. - </para></listitem> - <listitem><para> - <emphasis>Examine the List of Mailing Lists:</emphasis> - For a list of the Yocto Project and related mailing - lists, see the - "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing lists</ulink>" - section in the Yocto Project Reference Manual. - </para></listitem> - </itemizedlist> - </para></listitem> - <listitem><para> - <emphasis>Make a Pull Request:</emphasis> - Notify the maintainer or the mailing list that you have - pushed a change by making a pull request.</para> - - <para>The Yocto Project provides two scripts that - conveniently let you generate and send pull requests to the - Yocto Project. - These scripts are <filename>create-pull-request</filename> - and <filename>send-pull-request</filename>. - You can find these scripts in the - <filename>scripts</filename> directory within the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - (e.g. <filename>~/poky/scripts</filename>). - </para> - - <para>Using these scripts correctly formats the requests - without introducing any whitespace or HTML formatting. - The maintainer that receives your patches either directly - or through the mailing list needs to be able to save and - apply them directly from your emails. - Using these scripts is the preferred method for sending - patches.</para> - - <para>First, create the pull request. - For example, the following command runs the script, - specifies the upstream repository in the contrib directory - into which you pushed the change, and provides a subject - line in the created patch files: - <literallayout class='monospaced'> - $ ~/poky/scripts/create-pull-request -u meta-intel-contrib -s "Updated Manual Section Reference in README" - </literallayout> - Running this script forms - <filename>*.patch</filename> files in a folder named - <filename>pull-</filename><replaceable>PID</replaceable> - in the current directory. - One of the patch files is a cover letter.</para> - - <para>Before running the - <filename>send-pull-request</filename> script, you must - edit the cover letter patch to insert information about - your change. - After editing the cover letter, send the pull request. - For example, the following command runs the script and - specifies the patch directory and email address. - In this example, the email address is a mailing list: - <literallayout class='monospaced'> - $ ~/poky/scripts/send-pull-request -p ~/meta-intel/pull-10565 -t meta-intel@yoctoproject.org - </literallayout> - You need to follow the prompts as the script is - interactive. - <note> - For help on using these scripts, simply provide the - <filename>-h</filename> argument as follows: - <literallayout class='monospaced'> - $ poky/scripts/create-pull-request -h - $ poky/scripts/send-pull-request -h - </literallayout> - </note> - </para></listitem> - </orderedlist> - </para> - </section> - - <section id='submitting-a-patch'> - <title>Using Email to Submit a Patch</title> - - <para> - You can submit patches without using the - <filename>create-pull-request</filename> and - <filename>send-pull-request</filename> scripts described in the - previous section. - However, keep in mind, the preferred method is to use the scripts. - </para> - - <para> - Depending on the components changed, you need to submit the email - to a specific mailing list. - For some guidance on which mailing list to use, see the - <link linkend='figuring-out-the-mailing-list-to-use'>list</link> - at the beginning of this section. - For a description of all the available mailing lists, see the - "<ulink url='&YOCTO_DOCS_REF_URL;#resources-mailinglist'>Mailing Lists</ulink>" - section in the Yocto Project Reference Manual. - </para> - - <para> - Here is the general procedure on how to submit a patch through - email without using the scripts: - <orderedlist> - <listitem><para> - <emphasis>Make Your Changes Locally:</emphasis> - Make your changes in your local Git repository. - You should make small, controlled, isolated changes. - Keeping changes small and isolated aids review, - makes merging/rebasing easier and keeps the change - history clean should anyone need to refer to it in - future. - </para></listitem> - <listitem><para> - <emphasis>Stage Your Changes:</emphasis> - Stage your changes by using the <filename>git add</filename> - command on each file you changed. - </para></listitem> - <listitem><para> - <emphasis>Commit Your Changes:</emphasis> - Commit the change by using the - <filename>git commit --signoff</filename> command. - Using the <filename>--signoff</filename> option identifies - you as the person making the change and also satisfies - the Developer's Certificate of Origin (DCO) shown earlier. - </para> - - <para>When you form a commit, you must follow certain - standards established by the Yocto Project development - team. - See - <link linkend='making-sure-you-have-correct-commit-information'>Step 3</link> - in the previous section for information on how to - provide commit information that meets Yocto Project - commit message standards. - </para></listitem> - <listitem><para> - <emphasis>Format the Commit:</emphasis> - Format the commit into an email message. - To format commits, use the - <filename>git format-patch</filename> command. - When you provide the command, you must include a revision - list or a number of patches as part of the command. - For example, either of these two commands takes your most - recent single commit and formats it as an email message in - the current directory: - <literallayout class='monospaced'> - $ git format-patch -1 - </literallayout> - or - <literallayout class='monospaced'> - $ git format-patch HEAD~ - </literallayout></para> - - <para>After the command is run, the current directory - contains a numbered <filename>.patch</filename> file for - the commit.</para> - - <para>If you provide several commits as part of the - command, the <filename>git format-patch</filename> command - produces a series of numbered files in the current - directory – one for each commit. - If you have more than one patch, you should also use the - <filename>--cover</filename> option with the command, - which generates a cover letter as the first "patch" in - the series. - You can then edit the cover letter to provide a - description for the series of patches. - For information on the - <filename>git format-patch</filename> command, - see <filename>GIT_FORMAT_PATCH(1)</filename> displayed - using the <filename>man git-format-patch</filename> - command. - <note> - If you are or will be a frequent contributor to the - Yocto Project or to OpenEmbedded, you might consider - requesting a contrib area and the necessary associated - rights. - </note> - </para></listitem> - <listitem><para> - <emphasis>Import the Files Into Your Mail Client:</emphasis> - Import the files into your mail client by using the - <filename>git send-email</filename> command. - <note> - In order to use <filename>git send-email</filename>, - you must have the proper Git packages installed on - your host. - For Ubuntu, Debian, and Fedora the package is - <filename>git-email</filename>. - </note></para> - - <para>The <filename>git send-email</filename> command - sends email by using a local or remote Mail Transport Agent - (MTA) such as <filename>msmtp</filename>, - <filename>sendmail</filename>, or through a direct - <filename>smtp</filename> configuration in your Git - <filename>~/.gitconfig</filename> file. - If you are submitting patches through email only, it is - very important that you submit them without any whitespace - or HTML formatting that either you or your mailer - introduces. - The maintainer that receives your patches needs to be able - to save and apply them directly from your emails. - A good way to verify that what you are sending will be - applicable by the maintainer is to do a dry run and send - them to yourself and then save and apply them as the - maintainer would.</para> - - <para>The <filename>git send-email</filename> command is - the preferred method for sending your patches using - email since there is no risk of compromising whitespace - in the body of the message, which can occur when you use - your own mail client. - The command also has several options that let you - specify recipients and perform further editing of the - email message. - For information on how to use the - <filename>git send-email</filename> command, - see <filename>GIT-SEND-EMAIL(1)</filename> displayed using - the <filename>man git-send-email</filename> command. - </para></listitem> - </orderedlist> - </para> - </section> - </section> - </section> - - <section id='working-with-licenses'> - <title>Working With Licenses</title> - - <para> - As mentioned in the - "<ulink url='&YOCTO_DOCS_OM_URL;#licensing'>Licensing</ulink>" - section in the Yocto Project Overview and Concepts Manual, - open source projects are open to the public and they - consequently have different licensing structures in place. - This section describes the mechanism by which the - <ulink url='&YOCTO_DOCS_REF_URL;#build-system-term'>OpenEmbedded build system</ulink> - tracks changes to licensing text and covers how to maintain open - source license compliance during your project's lifecycle. - The section also describes how to enable commercially licensed - recipes, which by default are disabled. - </para> - - <section id="usingpoky-configuring-LIC_FILES_CHKSUM"> - <title>Tracking License Changes</title> - - <para> - The license of an upstream project might change in the future. - In order to prevent these changes going unnoticed, the - <ulink url='&YOCTO_DOCS_REF_URL;#var-LIC_FILES_CHKSUM'><filename>LIC_FILES_CHKSUM</filename></ulink> - variable tracks changes to the license text. The checksums are - validated at the end of the configure step, and if the - checksums do not match, the build will fail. - </para> - - <section id="usingpoky-specifying-LIC_FILES_CHKSUM"> - <title>Specifying the <filename>LIC_FILES_CHKSUM</filename> Variable</title> - - <para> - The <filename>LIC_FILES_CHKSUM</filename> - variable contains checksums of the license text in the - source code for the recipe. - Following is an example of how to specify - <filename>LIC_FILES_CHKSUM</filename>: - <literallayout class='monospaced'> - LIC_FILES_CHKSUM = "file://COPYING;md5=xxxx \ - file://licfile1.txt;beginline=5;endline=29;md5=yyyy \ - file://licfile2.txt;endline=50;md5=zzzz \ - ..." - </literallayout> - <note><title>Notes</title> - <itemizedlist> - <listitem><para> - When using "beginline" and "endline", realize - that line numbering begins with one and not - zero. - Also, the included lines are inclusive (i.e. - lines five through and including 29 in the - previous example for - <filename>licfile1.txt</filename>). - </para></listitem> - <listitem><para> - When a license check fails, the selected license - text is included as part of the QA message. - Using this output, you can determine the exact - start and finish for the needed license text. - </para></listitem> - </itemizedlist> - </note> - </para> - - <para> - The build system uses the - <ulink url='&YOCTO_DOCS_REF_URL;#var-S'><filename>S</filename></ulink> - variable as the default directory when searching files - listed in <filename>LIC_FILES_CHKSUM</filename>. - The previous example employs the default directory. - </para> - - <para> - Consider this next example: - <literallayout class='monospaced'> - LIC_FILES_CHKSUM = "file://src/ls.c;beginline=5;endline=16;\ - md5=bb14ed3c4cda583abc85401304b5cd4e" - LIC_FILES_CHKSUM = "file://${WORKDIR}/license.html;md5=5c94767cedb5d6987c902ac850ded2c6" - </literallayout> - </para> - - <para> - The first line locates a file in - <filename>${S}/src/ls.c</filename> and isolates lines five - through 16 as license text. - The second line refers to a file in - <ulink url='&YOCTO_DOCS_REF_URL;#var-WORKDIR'><filename>WORKDIR</filename></ulink>. - </para> - - <para> - Note that <filename>LIC_FILES_CHKSUM</filename> variable is - mandatory for all recipes, unless the - <filename>LICENSE</filename> variable is set to "CLOSED". - </para> - </section> - - <section id="usingpoky-LIC_FILES_CHKSUM-explanation-of-syntax"> - <title>Explanation of Syntax</title> - - <para> - As mentioned in the previous section, the - <filename>LIC_FILES_CHKSUM</filename> variable lists all - the important files that contain the license text for the - source code. - It is possible to specify a checksum for an entire file, - or a specific section of a file (specified by beginning and - ending line numbers with the "beginline" and "endline" - parameters, respectively). - The latter is useful for source files with a license - notice header, README documents, and so forth. - If you do not use the "beginline" parameter, then it is - assumed that the text begins on the first line of the file. - Similarly, if you do not use the "endline" parameter, - it is assumed that the license text ends with the last - line of the file. - </para> - - <para> - The "md5" parameter stores the md5 checksum of the license - text. - If the license text changes in any way as compared to - this parameter then a mismatch occurs. - This mismatch triggers a build failure and notifies - the developer. - Notification allows the developer to review and address - the license text changes. - Also note that if a mismatch occurs during the build, - the correct md5 checksum is placed in the build log and - can be easily copied to the recipe. - </para> - - <para> - There is no limit to how many files you can specify using - the <filename>LIC_FILES_CHKSUM</filename> variable. - Generally, however, every project requires a few - specifications for license tracking. - Many projects have a "COPYING" file that stores the - license information for all the source code files. - This practice allows you to just track the "COPYING" - file as long as it is kept up to date. - <note><title>Tips</title> - <itemizedlist> - <listitem><para> - If you specify an empty or invalid "md5" - parameter, - <ulink url='&YOCTO_DOCS_REF_URL;#bitbake-term'>BitBake</ulink> - returns an md5 mis-match - error and displays the correct "md5" parameter - value during the build. - The correct parameter is also captured in - the build log. - </para></listitem> - <listitem><para> - If the whole file contains only license text, - you do not need to use the "beginline" and - "endline" parameters. - </para></listitem> - </itemizedlist> - </note> - </para> - </section> - </section> - - <section id="enabling-commercially-licensed-recipes"> - <title>Enabling Commercially Licensed Recipes</title> - - <para> - By default, the OpenEmbedded build system disables - components that have commercial or other special licensing - requirements. - Such requirements are defined on a - recipe-by-recipe basis through the - <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS'><filename>LICENSE_FLAGS</filename></ulink> - variable definition in the affected recipe. - For instance, the - <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> - recipe contains the following statement: - <literallayout class='monospaced'> - LICENSE_FLAGS = "commercial" - </literallayout> - Here is a slightly more complicated example that contains both - an explicit recipe name and version (after variable expansion): - <literallayout class='monospaced'> - LICENSE_FLAGS = "license_${PN}_${PV}" - </literallayout> - In order for a component restricted by a - <filename>LICENSE_FLAGS</filename> definition to be enabled and - included in an image, it needs to have a matching entry in the - global - <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_FLAGS_WHITELIST'><filename>LICENSE_FLAGS_WHITELIST</filename></ulink> - variable, which is a variable typically defined in your - <filename>local.conf</filename> file. - For example, to enable the - <filename>poky/meta/recipes-multimedia/gstreamer/gst-plugins-ugly</filename> - package, you could add either the string - "commercial_gst-plugins-ugly" or the more general string - "commercial" to <filename>LICENSE_FLAGS_WHITELIST</filename>. - See the - "<link linkend='license-flag-matching'>License Flag Matching</link>" - section for a full - explanation of how <filename>LICENSE_FLAGS</filename> matching - works. - Here is the example: - <literallayout class='monospaced'> - LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly" - </literallayout> - Likewise, to additionally enable the package built from the - recipe containing - <filename>LICENSE_FLAGS = "license_${PN}_${PV}"</filename>, - and assuming that the actual recipe name was - <filename>emgd_1.10.bb</filename>, the following string would - enable that package as well as the original - <filename>gst-plugins-ugly</filename> package: - <literallayout class='monospaced'> - LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly license_emgd_1.10" - </literallayout> - As a convenience, you do not need to specify the complete - license string in the whitelist for every package. - You can use an abbreviated form, which consists - of just the first portion or portions of the license - string before the initial underscore character or characters. - A partial string will match any license that contains the - given string as the first portion of its license. - For example, the following whitelist string will also match - both of the packages previously mentioned as well as any other - packages that have licenses starting with "commercial" or - "license". - <literallayout class='monospaced'> - LICENSE_FLAGS_WHITELIST = "commercial license" - </literallayout> - </para> - - <section id="license-flag-matching"> - <title>License Flag Matching</title> - - <para> - License flag matching allows you to control what recipes - the OpenEmbedded build system includes in the build. - Fundamentally, the build system attempts to match - <filename>LICENSE_FLAGS</filename> strings found in recipes - against <filename>LICENSE_FLAGS_WHITELIST</filename> - strings found in the whitelist. - A match causes the build system to include a recipe in the - build, while failure to find a match causes the build - system to exclude a recipe. - </para> - - <para> - In general, license flag matching is simple. - However, understanding some concepts will help you - correctly and effectively use matching. - </para> - - <para> - Before a flag - defined by a particular recipe is tested against the - contents of the whitelist, the expanded string - <filename>_${PN}</filename> is appended to the flag. - This expansion makes each - <filename>LICENSE_FLAGS</filename> value recipe-specific. - After expansion, the string is then matched against the - whitelist. - Thus, specifying - <filename>LICENSE_FLAGS = "commercial"</filename> - in recipe "foo", for example, results in the string - <filename>"commercial_foo"</filename>. - And, to create a match, that string must appear in the - whitelist. - </para> - - <para> - Judicious use of the <filename>LICENSE_FLAGS</filename> - strings and the contents of the - <filename>LICENSE_FLAGS_WHITELIST</filename> variable - allows you a lot of flexibility for including or excluding - recipes based on licensing. - For example, you can broaden the matching capabilities by - using license flags string subsets in the whitelist. - <note> - When using a string subset, be sure to use the part of - the expanded string that precedes the appended - underscore character (e.g. - <filename>usethispart_1.3</filename>, - <filename>usethispart_1.4</filename>, and so forth). - </note> - For example, simply specifying the string "commercial" in - the whitelist matches any expanded - <filename>LICENSE_FLAGS</filename> definition that starts - with the string "commercial" such as "commercial_foo" and - "commercial_bar", which are the strings the build system - automatically generates for hypothetical recipes named - "foo" and "bar" assuming those recipes simply specify the - following: - <literallayout class='monospaced'> - LICENSE_FLAGS = "commercial" - </literallayout> - Thus, you can choose to exhaustively - enumerate each license flag in the whitelist and - allow only specific recipes into the image, or - you can use a string subset that causes a broader range of - matches to allow a range of recipes into the image. - </para> - - <para> - This scheme works even if the - <filename>LICENSE_FLAGS</filename> string already - has <filename>_${PN}</filename> appended. - For example, the build system turns the license flag - "commercial_1.2_foo" into "commercial_1.2_foo_foo" and - would match both the general "commercial" and the specific - "commercial_1.2_foo" strings found in the whitelist, as - expected. - </para> - - <para> - Here are some other scenarios: - <itemizedlist> - <listitem><para> - You can specify a versioned string in the recipe - such as "commercial_foo_1.2" in a "foo" recipe. - The build system expands this string to - "commercial_foo_1.2_foo". - Combine this license flag with a whitelist that has - the string "commercial" and you match the flag - along with any other flag that starts with the - string "commercial". - </para></listitem> - <listitem><para> - Under the same circumstances, you can use - "commercial_foo" in the whitelist and the build - system not only matches "commercial_foo_1.2" but - also matches any license flag with the string - "commercial_foo", regardless of the version. - </para></listitem> - <listitem><para> - You can be very specific and use both the - package and version parts in the whitelist (e.g. - "commercial_foo_1.2") to specifically match a - versioned recipe. - </para></listitem> - </itemizedlist> - </para> - </section> - - <section id="other-variables-related-to-commercial-licenses"> - <title>Other Variables Related to Commercial Licenses</title> - - <para> - Other helpful variables related to commercial - license handling exist and are defined in the - <filename>poky/meta/conf/distro/include/default-distrovars.inc</filename> file: - <literallayout class='monospaced'> - COMMERCIAL_AUDIO_PLUGINS ?= "" - COMMERCIAL_VIDEO_PLUGINS ?= "" - </literallayout> - If you want to enable these components, you can do so by - making sure you have statements similar to the following - in your <filename>local.conf</filename> configuration file: - <literallayout class='monospaced'> - COMMERCIAL_AUDIO_PLUGINS = "gst-plugins-ugly-mad \ - gst-plugins-ugly-mpegaudioparse" - COMMERCIAL_VIDEO_PLUGINS = "gst-plugins-ugly-mpeg2dec \ - gst-plugins-ugly-mpegstream gst-plugins-bad-mpegvideoparse" - LICENSE_FLAGS_WHITELIST = "commercial_gst-plugins-ugly commercial_gst-plugins-bad commercial_qmmp" - </literallayout> - Of course, you could also create a matching whitelist - for those components using the more general "commercial" - in the whitelist, but that would also enable all the - other packages with <filename>LICENSE_FLAGS</filename> - containing "commercial", which you may or may not want: - <literallayout class='monospaced'> - LICENSE_FLAGS_WHITELIST = "commercial" - </literallayout> - </para> - - <para> - Specifying audio and video plugins as part of the - <filename>COMMERCIAL_AUDIO_PLUGINS</filename> and - <filename>COMMERCIAL_VIDEO_PLUGINS</filename> statements - (along with the enabling - <filename>LICENSE_FLAGS_WHITELIST</filename>) includes the - plugins or components into built images, thus adding - support for media formats or components. - </para> - </section> - </section> - - <section id='maintaining-open-source-license-compliance-during-your-products-lifecycle'> - <title>Maintaining Open Source License Compliance During Your Product's Lifecycle</title> - - <para> - One of the concerns for a development organization using open source - software is how to maintain compliance with various open source - licensing during the lifecycle of the product. - While this section does not provide legal advice or - comprehensively cover all scenarios, it does - present methods that you can use to - assist you in meeting the compliance requirements during a software - release. - </para> - - <para> - With hundreds of different open source licenses that the Yocto - Project tracks, it is difficult to know the requirements of each - and every license. - However, the requirements of the major FLOSS licenses can begin - to be covered by - assuming that three main areas of concern exist: - <itemizedlist> - <listitem><para>Source code must be provided.</para></listitem> - <listitem><para>License text for the software must be - provided.</para></listitem> - <listitem><para>Compilation scripts and modifications to the - source code must be provided. - </para></listitem> - </itemizedlist> - There are other requirements beyond the scope of these - three and the methods described in this section - (e.g. the mechanism through which source code is distributed). - </para> - - <para> - As different organizations have different methods of complying with - open source licensing, this section is not meant to imply that - there is only one single way to meet your compliance obligations, - but rather to describe one method of achieving compliance. - The remainder of this section describes methods supported to meet the - previously mentioned three requirements. - Once you take steps to meet these requirements, - and prior to releasing images, sources, and the build system, - you should audit all artifacts to ensure completeness. - <note> - The Yocto Project generates a license manifest during - image creation that is located - in <filename>${DEPLOY_DIR}/licenses/<replaceable>image_name-datestamp</replaceable></filename> - to assist with any audits. - </note> - </para> - - <section id='providing-the-source-code'> - <title>Providing the Source Code</title> - - <para> - Compliance activities should begin before you generate the - final image. - The first thing you should look at is the requirement that - tops the list for most compliance groups - providing - the source. - The Yocto Project has a few ways of meeting this - requirement. - </para> - - <para> - One of the easiest ways to meet this requirement is - to provide the entire - <ulink url='&YOCTO_DOCS_REF_URL;#var-DL_DIR'><filename>DL_DIR</filename></ulink> - used by the build. - This method, however, has a few issues. - The most obvious is the size of the directory since it includes - all sources used in the build and not just the source used in - the released image. - It will include toolchain source, and other artifacts, which - you would not generally release. - However, the more serious issue for most companies is accidental - release of proprietary software. - The Yocto Project provides an - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-archiver'><filename>archiver</filename></ulink> - class to help avoid some of these concerns. - </para> - - <para> - Before you employ <filename>DL_DIR</filename> or the - <filename>archiver</filename> class, you need to decide how - you choose to provide source. - The source <filename>archiver</filename> class can generate - tarballs and SRPMs and can create them with various levels of - compliance in mind. - </para> - - <para> - One way of doing this (but certainly not the only way) is to - release just the source as a tarball. - You can do this by adding the following to the - <filename>local.conf</filename> file found in the - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>: - <literallayout class='monospaced'> - INHERIT += "archiver" - ARCHIVER_MODE[src] = "original" - </literallayout> - During the creation of your image, the source from all - recipes that deploy packages to the image is placed within - subdirectories of - <filename>DEPLOY_DIR/sources</filename> based on the - <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE'><filename>LICENSE</filename></ulink> - for each recipe. - Releasing the entire directory enables you to comply with - requirements concerning providing the unmodified source. - It is important to note that the size of the directory can - get large. - </para> - - <para> - A way to help mitigate the size issue is to only release - tarballs for licenses that require the release of - source. - Let us assume you are only concerned with GPL code as - identified by running the following script: - <literallayout class='monospaced'> - # Script to archive a subset of packages matching specific license(s) - # Source and license files are copied into sub folders of package folder - # Must be run from build folder - #!/bin/bash - src_release_dir="source-release" - mkdir -p $src_release_dir - for a in tmp/deploy/sources/*; do - for d in $a/*; do - # Get package name from path - p=`basename $d` - p=${p%-*} - p=${p%-*} - # Only archive GPL packages (update *GPL* regex for your license check) - numfiles=`ls tmp/deploy/licenses/$p/*GPL* 2> /dev/null | wc -l` - if [ $numfiles -gt 1 ]; then - echo Archiving $p - mkdir -p $src_release_dir/$p/source - cp $d/* $src_release_dir/$p/source 2> /dev/null - mkdir -p $src_release_dir/$p/license - cp tmp/deploy/licenses/$p/* $src_release_dir/$p/license 2> /dev/null - fi - done - done - </literallayout> - At this point, you could create a tarball from the - <filename>gpl_source_release</filename> directory and - provide that to the end user. - This method would be a step toward achieving compliance - with section 3a of GPLv2 and with section 6 of GPLv3. - </para> - </section> - - <section id='providing-license-text'> - <title>Providing License Text</title> - - <para> - One requirement that is often overlooked is inclusion - of license text. - This requirement also needs to be dealt with prior to - generating the final image. - Some licenses require the license text to accompany - the binary. - You can achieve this by adding the following to your - <filename>local.conf</filename> file: - <literallayout class='monospaced'> - COPY_LIC_MANIFEST = "1" - COPY_LIC_DIRS = "1" - LICENSE_CREATE_PACKAGE = "1" - </literallayout> - Adding these statements to the configuration file ensures - that the licenses collected during package generation - are included on your image. - <note> - <para>Setting all three variables to "1" results in the - image having two copies of the same license file. - One copy resides in - <filename>/usr/share/common-licenses</filename> and - the other resides in - <filename>/usr/share/license</filename>.</para> - - <para>The reason for this behavior is because - <ulink url='&YOCTO_DOCS_REF_URL;#var-COPY_LIC_DIRS'><filename>COPY_LIC_DIRS</filename></ulink> - and - <ulink url='&YOCTO_DOCS_REF_URL;#var-COPY_LIC_MANIFEST'><filename>COPY_LIC_MANIFEST</filename></ulink> - add a copy of the license when the image is built but do - not offer a path for adding licenses for newly installed - packages to an image. - <ulink url='&YOCTO_DOCS_REF_URL;#var-LICENSE_CREATE_PACKAGE'><filename>LICENSE_CREATE_PACKAGE</filename></ulink> - adds a separate package and an upgrade path for adding - licenses to an image.</para> - </note> - </para> - - <para> - As the source <filename>archiver</filename> class has already - archived the original - unmodified source that contains the license files, - you would have already met the requirements for inclusion - of the license information with source as defined by the GPL - and other open source licenses. - </para> - </section> - - <section id='providing-compilation-scripts-and-source-code-modifications'> - <title>Providing Compilation Scripts and Source Code Modifications</title> - - <para> - At this point, we have addressed all we need to - prior to generating the image. - The next two requirements are addressed during the final - packaging of the release. - </para> - - <para> - By releasing the version of the OpenEmbedded build system - and the layers used during the build, you will be providing both - compilation scripts and the source code modifications in one - step. - </para> - - <para> - If the deployment team has a - <ulink url='&YOCTO_DOCS_BSP_URL;#bsp-layers'>BSP layer</ulink> - and a distro layer, and those those layers are used to patch, - compile, package, or modify (in any way) any open source - software included in your released images, you - might be required to release those layers under section 3 of - GPLv2 or section 1 of GPLv3. - One way of doing that is with a clean - checkout of the version of the Yocto Project and layers used - during your build. - Here is an example: - <literallayout class='monospaced'> - # We built using the &DISTRO_NAME_NO_CAP; branch of the poky repo - $ git clone -b &DISTRO_NAME_NO_CAP; git://git.yoctoproject.org/poky - $ cd poky - # We built using the release_branch for our layers - $ git clone -b release_branch git://git.mycompany.com/meta-my-bsp-layer - $ git clone -b release_branch git://git.mycompany.com/meta-my-software-layer - # clean up the .git repos - $ find . -name ".git" -type d -exec rm -rf {} \; - </literallayout> - One thing a development organization might want to consider - for end-user convenience is to modify - <filename>meta-poky/conf/bblayers.conf.sample</filename> to - ensure that when the end user utilizes the released build - system to build an image, the development organization's - layers are included in the <filename>bblayers.conf</filename> - file automatically: - <literallayout class='monospaced'> - # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf - # changes incompatibly - POKY_BBLAYERS_CONF_VERSION = "2" - - BBPATH = "${TOPDIR}" - BBFILES ?= "" - - BBLAYERS ?= " \ - ##OEROOT##/meta \ - ##OEROOT##/meta-poky \ - ##OEROOT##/meta-yocto-bsp \ - ##OEROOT##/meta-mylayer \ - " - </literallayout> - Creating and providing an archive of the - <ulink url='&YOCTO_DOCS_REF_URL;#metadata'>Metadata</ulink> - layers (recipes, configuration files, and so forth) - enables you to meet your - requirements to include the scripts to control compilation - as well as any modifications to the original source. - </para> - </section> - </section> - - <section id='copying-licenses-that-do-not-exist'> - <title>Copying Licenses that Do Not Exist</title> - - <para> - Some packages, such as the linux-firmware package, have many - licenses that are not in any way common. - You can avoid adding a lot of these types of common license - files, which are only applicable to a specific package, by using - the - <ulink url='&YOCTO_DOCS_REF_URL;#var-NO_GENERIC_LICENSE'><filename>NO_GENERIC_LICENSE</filename></ulink> - variable. - Using this variable also avoids QA errors when you use a - non-common, non-CLOSED license in a recipe. - </para> - - <para> - The following is an example that uses the - <filename>LICENSE.Abilis.txt</filename> - file as the license from the fetched source: - <literallayout class='monospaced'> - NO_GENERIC_LICENSE[Firmware-Abilis] = "LICENSE.Abilis.txt" - </literallayout> - </para> - </section> - </section> - - <section id='using-the-error-reporting-tool'> - <title>Using the Error Reporting Tool</title> - - <para> - The error reporting tool allows you to - submit errors encountered during builds to a central database. - Outside of the build environment, you can use a web interface to - browse errors, view statistics, and query for errors. - The tool works using a client-server system where the client - portion is integrated with the installed Yocto Project - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink> - (e.g. <filename>poky</filename>). - The server receives the information collected and saves it in a - database. - </para> - - <para> - A live instance of the error reporting server exists at - <ulink url='http://errors.yoctoproject.org'></ulink>. - This server exists so that when you want to get help with - build failures, you can submit all of the information on the - failure easily and then point to the URL in your bug report - or send an email to the mailing list. - <note> - If you send error reports to this server, the reports become - publicly visible. - </note> - </para> - - <section id='enabling-and-using-the-tool'> - <title>Enabling and Using the Tool</title> - - <para> - By default, the error reporting tool is disabled. - You can enable it by inheriting the - <ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-report-error'><filename>report-error</filename></ulink> - class by adding the following statement to the end of - your <filename>local.conf</filename> file in your - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - <literallayout class='monospaced'> - INHERIT += "report-error" - </literallayout> - </para> - - <para> - By default, the error reporting feature stores information in - <filename>${</filename><ulink url='&YOCTO_DOCS_REF_URL;#var-LOG_DIR'><filename>LOG_DIR</filename></ulink><filename>}/error-report</filename>. - However, you can specify a directory to use by adding the following - to your <filename>local.conf</filename> file: - <literallayout class='monospaced'> - ERR_REPORT_DIR = "path" - </literallayout> - Enabling error reporting causes the build process to collect - the errors and store them in a file as previously described. - When the build system encounters an error, it includes a - command as part of the console output. - You can run the command to send the error file to the server. - For example, the following command sends the errors to an - upstream server: - <literallayout class='monospaced'> - $ send-error-report /home/brandusa/project/poky/build/tmp/log/error-report/error_report_201403141617.txt - </literallayout> - In the previous example, the errors are sent to a public - database available at - <ulink url='http://errors.yoctoproject.org'></ulink>, which is - used by the entire community. - If you specify a particular server, you can send the errors - to a different database. - Use the following command for more information on available - options: - <literallayout class='monospaced'> - $ send-error-report --help - </literallayout> - </para> - - <para> - When sending the error file, you are prompted to review the - data being sent as well as to provide a name and optional - email address. - Once you satisfy these prompts, the command returns a link - from the server that corresponds to your entry in the database. - For example, here is a typical link: - <literallayout class='monospaced'> - http://errors.yoctoproject.org/Errors/Details/9522/ - </literallayout> - Following the link takes you to a web interface where you can - browse, query the errors, and view statistics. - </para> - </section> - - <section id='disabling-the-tool'> - <title>Disabling the Tool</title> - - <para> - To disable the error reporting feature, simply remove or comment - out the following statement from the end of your - <filename>local.conf</filename> file in your - <ulink url='&YOCTO_DOCS_REF_URL;#build-directory'>Build Directory</ulink>. - <literallayout class='monospaced'> - INHERIT += "report-error" - </literallayout> - </para> - </section> - - <section id='setting-up-your-own-error-reporting-server'> - <title>Setting Up Your Own Error Reporting Server</title> - - <para> - If you want to set up your own error reporting server, you - can obtain the code from the Git repository at - <ulink url='http://git.yoctoproject.org/cgit/cgit.cgi/error-report-web/'></ulink>. - Instructions on how to set it up are in the README document. - </para> - </section> - </section> - - <section id="dev-using-wayland-and-weston"> - <title>Using Wayland and Weston</title> - - <para> - <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)'>Wayland</ulink> - is a computer display server protocol that - provides a method for compositing window managers to communicate - directly with applications and video hardware and expects them to - communicate with input hardware using other libraries. - Using Wayland with supporting targets can result in better control - over graphics frame rendering than an application might otherwise - achieve. - </para> - - <para> - The Yocto Project provides the Wayland protocol libraries and the - reference - <ulink url='http://en.wikipedia.org/wiki/Wayland_(display_server_protocol)#Weston'>Weston</ulink> - compositor as part of its release. - You can find the integrated packages in the - <filename>meta</filename> layer of the - <ulink url='&YOCTO_DOCS_REF_URL;#source-directory'>Source Directory</ulink>. - Specifically, you can find the recipes that build both Wayland - and Weston at <filename>meta/recipes-graphics/wayland</filename>. - </para> - - <para> - You can build both the Wayland and Weston packages for use only - with targets that accept the - <ulink url='https://en.wikipedia.org/wiki/Mesa_(computer_graphics)'>Mesa 3D and Direct Rendering Infrastructure</ulink>, - which is also known as Mesa DRI. - This implies that you cannot build and use the packages if your - target uses, for example, the - <trademark class='registered'>Intel</trademark> Embedded Media - and Graphics Driver - (<trademark class='registered'>Intel</trademark> EMGD) that - overrides Mesa DRI. - <note> - Due to lack of EGL support, Weston 1.0.3 will not run - directly on the emulated QEMU hardware. - However, this version of Weston will run under X emulation - without issues. - </note> - </para> - - <para> - This section describes what you need to do to implement Wayland and - use the Weston compositor when building an image for a supporting - target. - </para> - - <section id="enabling-wayland-in-an-image"> - <title>Enabling Wayland in an Image</title> - - <para> - To enable Wayland, you need to enable it to be built and enable - it to be included (installed) in the image. - </para> - - <section id="enable-building"> - <title>Building</title> - - <para> - To cause Mesa to build the <filename>wayland-egl</filename> - platform and Weston to build Wayland with Kernel Mode - Setting - (<ulink url='https://wiki.archlinux.org/index.php/Kernel_Mode_Setting'>KMS</ulink>) - support, include the "wayland" flag in the - <ulink url="&YOCTO_DOCS_REF_URL;#var-DISTRO_FEATURES"><filename>DISTRO_FEATURES</filename></ulink> - statement in your <filename>local.conf</filename> file: - <literallayout class='monospaced'> - DISTRO_FEATURES_append = " wayland" - </literallayout> - <note> - If X11 has been enabled elsewhere, Weston will build - Wayland with X11 support - </note> - </para> - </section> - - <section id="enable-installation-in-an-image"> - <title>Installing</title> - - <para> - To install the Wayland feature into an image, you must - include the following - <ulink url='&YOCTO_DOCS_REF_URL;#var-CORE_IMAGE_EXTRA_INSTALL'><filename>CORE_IMAGE_EXTRA_INSTALL</filename></ulink> - statement in your <filename>local.conf</filename> file: - <literallayout class='monospaced'> - CORE_IMAGE_EXTRA_INSTALL += "wayland weston" - </literallayout> - </para> - </section> - </section> - - <section id="running-weston"> - <title>Running Weston</title> - - <para> - To run Weston inside X11, enabling it as described earlier and - building a Sato image is sufficient. - If you are running your image under Sato, a Weston Launcher - appears in the "Utility" category. - </para> - - <para> - Alternatively, you can run Weston through the command-line - interpretor (CLI), which is better suited for development work. - To run Weston under the CLI, you need to do the following after - your image is built: - <orderedlist> - <listitem><para> - Run these commands to export - <filename>XDG_RUNTIME_DIR</filename>: - <literallayout class='monospaced'> - mkdir -p /tmp/$USER-weston - chmod 0700 /tmp/$USER-weston - export XDG_RUNTIME_DIR=/tmp/$USER-weston - </literallayout> - </para></listitem> - <listitem><para> - Launch Weston in the shell: - <literallayout class='monospaced'> - weston - </literallayout></para></listitem> - </orderedlist> - </para> - </section> - </section> -</chapter> - -<!-- -vim: expandtab tw=80 ts=4 ---> |