| Age | Commit message (Collapse) | Author |
|---|
|
Kconfiglib/* were under ISC license before they were imported
here from https://github.com/ulfalizer/Kconfiglib
Adjusting SPDX header to reflect that fact.
tools/* all have some sort of GPLv2 headers; adding SPDX header
to make it obvious.
This address bug #13334 :
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13334
Change-Id: I243f2dd266a398f982798b771e74a67be70ecb52
Signed-off-by: William Bourque <wbourque@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
The resume logic was broken if a patch was renamed from
'foo' to 'foo/base', since the correct queue and hence patch
series could not be detected.
When searching for the series, we should check foo/base and
foo.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Create a new command to merge a feature branch inside the current branch
when processing meta-series.
Signed-off-by: David Vincent <freesilicon@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
During a tree generation we must always make the branch active when we
see any kind of branch command. This ensures that any subsequent patches
are applied in the proper context.
Previously, only branch creation was changing the active branch, and
this mean that tree generation was not determinstic and relied on the
order of processing to generate a correct tree.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Now that scc/spp and the other tools are modular (and smarter),
kgit-meta can be drastically simplified. It no longer needs to
worry about git, branches, kconfigs, or anything but taking
a meta series and applying it to a tree.
Currently this means branches and patches.
Additionally, kgit-meta is only used to generate a tree, so we
don't have to worry about it resuming or incorrect apply patches
when a developer is doing a build via a build system.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Once we've found a reason to push a patch, we should never stop
patching. Since if we top patching, then changes may be hidden
on ancestor branches.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
As part of processing configuration, patches and other operations the
paths of the files are normalized so they can be relocated from where
they were originally processed.
This is done by recording the directories that contain patches and
configuration, and then removing the longest possible path from each
element. Once that is done, we can search by their unique suffix across
any number of target directory structures.
The code that generated the path names, and the code that searches for
the longest prefix do not always agree on whether a trailing slash is
present. To cover all cases, we can simply try once with the slash
and once without.
When the relocation code misses the longest element, we end up with
arcane processing errors later in the tools since configs and patches
are not where they are expected.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Andreas Fenkart <afenkart@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When the auto-resume (resume point detection) was removed from the
processing of a meta-series, it ignored the fact that a single patch
series may in fact be processed a number of times.
Two layers patching a kernel will generate two different runs on the
same branch, which always start at patch one. This will obviously
break with duplicate patches.
To avoid this, we simply track the last patch applied, and explicitly
tell the patch scripts where to start. This gets us resume
functionality, without the overhead of resume point detection.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
On some machines (and kernel trees), it has been found that the branch
count may resolve to an empty string.
With the current patch/branch tests, this causes a patch application
error due to comparisons resolving with no right or left hand side.
To protect against this, we can combine the conditions and protect the
counts with quotes. This doesn't resolve the root cause, but is a
better, defensive condition.
Signed-off-by: Marek Vasut <marex@denx.de>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Previously the patching tools would consider both system and user
supplied patches in the same manner .. they are simply a series of
patches to be applied to a branch, and that the scripts should determine
where in the series to start (based on what is already on the branch).
This detection was causing a few problems:
- time consuming
- starting in the middle of a series when intermediate patches were
merged to a branch.
To solve both the performance and start detection, we instead simply
note the transition from system (i.e. already defined features and
series) and user/recipe supplied patches. When the transition is noted,
the system will start pushing ALL patches without doing autoresume
detection.
Control in keeping the series up to date is passed to the user, and
consistent behaviour/performance is achieved.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
configme relies on kgit-meta to gather fragments for processing.
To allow configuration of an external meta-data repository and non
build system use cases, we need a single script with no dependencies
or complex command lines.
By moving the fragment gathering routines into configme, we have the
first step in this process complte.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
The various scripts in the kern-tools have grown slightly different
meta directory detection routines over time. This means that we are
not always consistent, and the checks are not as robust as possible.
As part of this change, we have a single (which to avoid including
a common kgit script and coupling the tools .. is repeated) routine
for detecting the meta branch.
This detection logs the result in a new file ".metadir", which
subsequent scripts can find, and use, thereby avoid repeating the same
check many times.
We also enhance the check to look for a sentinel file in a proper meta
directory, to avoid false positives when an unexpected kernel process
leaves an uncommitted directory in the kernel dir.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In a similar way as commit 0768d697 [kgit-meta: dont run kgit-s2q for
non-leaf nodes], we can save even more processing time by not even
analysing and linking patches if we aren't on the leaf node of the
tree.
This early exit can save nearly 95% of the time required to "patch"
a tree when no changes are actually applied.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When changing branches it used to be a purely addititive operation
if a branch existed. This is no longer the case, so we shouldn't
preserve the existig series file, but should instead clear the series
file, and let the patch routines add to it if required.
This was triggered by the patch optimization changes, that no longer
run do_patch if a leaf/final branch is not being processed.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To speed up the processing of a meta-series, we can skip patch
validation on anything but the leaf branch, since we do not push
and propagate changes from sub branches to the final/BSP branch.
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In order to limit the matching of path relocation and sanity checking to
actual directory divisions, we need to retain the trailing / on paths. We
really only care about // and other invalid characters.
Once we are sure the trailing / is present, we'll no longer match on
partial directory names when multiple directories contain a common
payth and final directory name.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
With this change it is now possible to have a meta branch with meta data
in a directory that is not the same name as the branch.
The changes to three parts of the build are required to discover the name
of the meta directory by relying on the fact that in a clean/proper build
the meta directory is the only untracked, top level directory in the build.
As such, we can restore a checkpoint and then examine the build directory
to determine the meta directory name .. avoiding any new variables to
indicate this to the scripts and build system.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Previously the meta directory was added to all .gitignore files, otherwise
all users of the kernel would see an untracked directory "meta". This
caused workflow issues, since the same .gitignore was shared on the meta
branch itself. Which meant that changes were not shown and files needed to
be added with the -f flag.
To fix this, we teach the tools to look for .$META as the meta data
container, and place the meta branches directory under that same name.
The result is that the .meta directory is ignored by git automatically, but
is available to all branches. And switching to the meta branch no longer
warns of clobbered files, and once you arrive, files are tracked and
manipulated normally.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
kgit-meta knows if it just added a patch to the series file, so we
can pass it to kgit-s2q as the starting point. This avoid auto resume
detection and speeds patch application.
Also as part of this change, the variable naming is cleaned up to
differentiate between the resume fencepost detection, and the command
line option, which is the exact patch to start with.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Use new command line option and existing KMETA env var where
appropriate.
Also disable autoresume in order to speed up treegen patch
pushes. ($sname vs. "-a" flag).
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
|
|
The kern-tools use guilt as a patch management system, but with the
ability to merge branches, and do iterative development most of the
guilt safe guards need to be disabled. The resulting complexity to
both guilt and the kern-tools outweighs the benifits.
This change simplifies kgit-meta and introduces a new script called
kgit-s2q (series-to-queue), which manages the series file that was
previously processed by guilt.
This script can deal with autoresume, validates patches against
the tree (instead of against a status file), can deal with merges
and locally applied changes.
No user visible functionality is impacted by this change.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When patches are prep'd for tree application they are symlinked
from their original location into a branch named directory. The symlink
assumed the patch would be accesible via the "meta" directory, but meta
can be different per META branch.
Changing the symlink code to use the proper variable, versus the hardcoded
name allows patching to work on a non-meta named directory.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If a configuration fragment was missing, the previous error output
was not clear about the error:
| [INFO] doing kernel configme
| [INFO] Configuring target/machine combo: "standard/atom-pc"
| [INFO] collecting configs in ./meta/meta-series
| ERROR: could not sanitize configuration fragments
| errors are logged in ... linux/meta/cfg/standard/atom-pc/config.log
but we know the name of the missing fragment and can improve the error
message to be this:
| [ERROR] kernel configuration fragment fragment 'virto.cfg' cannot be found
| ERROR. A meta series could not be created for branch yocto/standard/common-pc/atom-pc
| ERROR. Could not locate meta series for atom-pc
| ERROR. Could not apply patches for atom-pc.
| Patch failures can be resolved in the devshell (bitbake -c devshell linux-yocto)
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
It is common that a feature fragment is included multiple times,
and these double inclusions can cause problems if a fragment includes
both patches and configuration.
This was previously dealt with by splitting the feature into an
'enable' configuration fragment and a 'feature' fragment that applies
the patches. This is effective, but requires that knowledge of the
entire include structure be known when creating a new feature.
We can detect when a patch has already been applied, or when a
feature fragment is included multiple times and do two things:
- emit a clear message about the multiple includes
- skip the double application of the patches via tracking a global
series file
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To allow a BSP to verify that critical options have made it to the
final .config (and to throw an error if they are not present), a
new "verify" kernel config fragment type is introduced.
The format of this fragment is identical to any other fragment,
except that its contents will not be used to directly configure
the kernel, but will be checked against the final .config any
missing or changed options will result in a message and error exit
code.
This allows the BSP to enforce critical options, but not have to
override policy or declare non-hardware options as hardware.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
kernel branches are constructed during patching of the tree by
constructing a '/' based hierarchy of names as each branch
directive is encountered.
But if a "branch $name $branchpoint" is used, the entire branch
name is supplied so no additions to the hierarchy should
happen. As such, that type of branch command should not be part
of branch name calculation and preparation.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When a series of patches fail to apply the pending patches are dumped
to the user. But if the GUILT_BASE directory is not set, you simply
get something that looks like an internal error:
| To force apply this patch, use 'guilt push -f'
| [ERROR] unable to complete push
| pending patches are:
| Patches directory doesn't exist, try guilt-init
If we export GUILT_BASE, the output is much better:
| [ERROR] unable to complete push
| pending patches are:
links/files/0002-makefile-patch.patch
| ERROR. Could not update git tree
| ERROR. Could not apply updates for qemux86.
| Patch failures can be resolved in the devshell:
| % bitbake -c devshell linux-yocto
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In the case of a custom repository, we may not enounter ANY explicit
branch points. This means that the first patch push will fail, since
the series file and other infrastructure has not been set up. To
ensure that we can always push patches, the starting branch is now
prepped for modifications.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
When an explicit branch point is specified:
branch $new_branch $branch_point
it means that the name of the new branch is explicit (and
known), and the branch name shouldn't inherit the names of
the branches that have proceeded it.
So if this is detected, branch name generation is skipped.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Not all features will create branches with the same origin, to support
these features, we need to do several things:
- condition the branch name to be the first argument in "branch <foo> <bar>"
since <foo> is the name and <bar> is the branch point
- during tree generation the branch should be reset to a known starting
point between features.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
createme and kgit-meta had two remaining assumptions about the
meta branch name and directory. With these fixes, all tools will now
respect and use a meta branch name/directory that is set via KMETA
in the environment.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
We no longer need kgit-meta to condition the tree when gathering
configuration fragments for config processing, since config
processing is no longer triggered by the branch name (as it
used to be). So we can safely remove the branch manipulation
routines from kgit-meta's config processing.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Largely due to historical reasons, there used to be a distinction
between the top of a "project" and the location of the kernel tree.
However, we currently were just passing in /dev/null to pad the args
and doing a pointless "if [ -d ... ]" on it.
Since the current instance ends up having `pwd` for both LINUX_SRC
and TOP_BUILD_DIR, just replace all instances of the latter with
the former.
With respect to the /dev/null, replace it with the $META file name.
We want this in kconf_check, since it was storing the audit data
on a per branch basis, and that is no longer unique now that several
BSPs re-use the same branch, resulting in overwritten audit data
when multiple builds were done in a single tree (i.e. buildall).
The same change is made in configme to KTGT; it was set to the
branch, and used in the path of audit data, but it is not unique.
As we aren't using the branch name at all anymore, drop it from
the args entirely. This allows deleting the get_branch function,
and we assign KTGT=$KTYPE/$KMACHINE which should hopefully be
unique per build.
For BSPs that have their own unique branch, the above is consistent
with the current kgit-meta path creation naming behaviour, i.e.:
linux$ls meta/cfg/standard/emenlow/
ls: cannot access meta/cfg/standard/emenlow/: No such file or directory
linux$kgit-meta -v -k meta/cfg/meta/emenlow-standard-meta
[INFO] collecting configs in ./meta/meta-series
[##################################################] (completed in 1 seconds)
linux$ls meta/cfg/standard/emenlow/
config_frag.txt hdw_frags.txt non_hdw_frags.txt
linux$
We see a slightly different behaviour from kgit-meta though,
when the branch is shared. It had similar code that, on _kconf
processing, would use the non-unique branch name for placement
of audit data. It also has been updated to use instead a KTGT
that is set from $KTYPE/$KMACHINE
Signed-off-by: Paul Gortmaker <paul.gortmaker@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
The period of supporting old trees with a different meta
branch name and directory structure are gone. So the cleanup
and removal of the old structure can be completed.
The meta branch and directory are now controlled via command line,
or via the KMETA environment variable. No testing and conditional
processing of the tree are required.
Additionally, the generate_cfg script is no longer used, or is the
branch conditing code in createme. So they can be safely removed
from the tools and repository.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Adding quoting to protect multiline defines.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
In addition to the hardware/non-hardware classification of kernel config
options, with this commit is it possible to specify 'required' and
'optional' kernel configuration values in feature descriptions. This allows
kernel configuration policy to be more than just BSP and "non-BSP".
If required options are missing, or defined and changed a WARNING during
the kernel config validation phase will be displayed. This warning can
be optionally turned into a hard error.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Multiple tools process the meta series to determine the final
branch name that a given series would create. This means that
tweaking in branch naming logic ripples through many tools and
scripts. Unifying the branch logic in kgit-meta, and providing
the ability to simply execute branching commands allows other
tools to assume that the current branch is always the correct
branch to process.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
To support feature descriptions that unconditionally create branches
via embedded git commands two constructs are required:
- a flag that indicates non-zero return codes are warnings only
- the ability to pass a source branch when creating a new branch
Both are added here, and when combined allow feature descriptions to
create a branch from another non-current branch, and to make that
creation multiple times if the feature is included from multiple
sources.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Fixes [YOCTO #1838]
When git commands are embedded into feature descriptions, they can
fail for a variety of reasons. The current meta series processing does
not trap errors, and will continue to process potentially failing
commands. All git commands are now checked for their return code, and
in case of error, the error code is passed up via early exit.
A second common git issue is on a new machine that has not been
configured with a username/email for commits. This is now detected
by checking for a global user.name. If one is not found, a local
user name of "Auto Configured" and email of "auto.configured" is
created to allow git operations to succeed.
Signed-off-by: Bruce Ashfield <bruce@windriver.com>
|
|
The wrong variable was being used when reporting the meta runtime
which results in "" always being the runtime.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If a branch is renamed *after* the system has detected whether or
not it was supposed to be created, we need to clear the 'create'
flag or will be checked out and processing will fail.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Remove duplicated and unnecessary code from kgit-meta. This is
a maintenance update to reduce the complexity of the scripts.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
After executing kgit-meta the checked out branch is the one that
the feature descriptions dictate. This should be the branch we
continue processing, so checking out the original branch is
counter productive.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
There are several ways that branch naming can fail:
- if a wrong name is picked
- if we don't check the error code
- if we unconditionally create a conditional branch
This commit fixes all these cases by only use /base if branch
inheritance is enabled, and by moving an error code check to
the right location to pickup branch rename issues. And finally
it adds the ability to pass -D<foo> for conditionally branching.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
William Bourque
Bruce Ashfield
David Vincent
Andreas Fenkart
Paul Gortmaker
Bruce Ashfield