| Age | Commit message (Collapse) | Author |
|---|
|
To make sure the kern-tools are compliant with the latest
security processes and awareness, we add a security file
with instructions on how to handle issues.
Also renaming the README file to a more common name.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
When we are commiting to the tree avoid running any locally
configured hooks, as they may be broken and prevent us from applying
changes.
If hooks are required a new flag can be introduced to indicate
hooks should be run.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
The way that kgit-s2q was previously used would always skip a
failed patch if you reran the series.
That is not always desireable if you fix a patch, and want to
have it retried on the next run.
Adding a --prev flag to the script which will roll back the
fence post files to a state where the last patch will be
retried versus skipped.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
"--committer-date-is-author-date" makes little sense for git-am --abort,
but a lot of sense for the seconds git-am attempt (with -C1, reduced
context), because poorly-formatted patches still cause non-reproducible
builds in this case. Add them to second attempt.
Cc: Bruce Ashfield <bruce.ashfield@gmail.com>
Fixes: dbf6fc9a0191 ("kgit-s2q: allow author date to be used instead of current time/date")
Co-developed-by: Michal Seben <michal.seben@siemens.com>
Signed-off-by: Alexander Sverdlin <alexander.sverdlin@siemens.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Add a python tool to generate a config fragment from a unified diff of two
kernel configurations. The diff is read from stdin and the fragment is printed
to stdout.
Usage:
$ diff -u config1 config2 | kconfig-diff2frag.py
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Some file systems, like ZFS, are very slow at appending to existing
files. Due to Copy-On-Write nature, they create a new copy of a file
each time we do ">>" in a shell script. This becomes very noticeable
if shell script does lots and lots of appends, like sanitize_fragment()
function in kconf_check. On my setup, do_kernel_configcheck task takes
literally hours to complete.
To fix this issue, we can store sanitized_list and fragment_errors.txt
files on tmpfs, which is extremely fast at writing. As most distros
use tmpfs for /tmp, logical step is to use `mktemp` to create
temporary files.
After completing writing to temporary locations, we can move those two
files back to ${LOGDIR}.
Also, function 'cleanup' was added to remove temporary files in case
of abnormal exit.
With this patch, do_kernel_configcheck task completes in ~2 minutes on
my setup, which is a great improvement.
Signed-off-by: Volodymyr Babchuk <volodymyr_babchuk@epam.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Some configuration fragments are included via a relative path, i.e.
include ../../feature2/feature.cfg
This breaks the relocation code, as the name of the fragment
will never match any of the path prefixs.
We solve that by using the kconfig name as located by searching the
includes. Those are always absolute and will match the prefix, allowing
us to figure out the relocation directory.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
When process_file function fails the output of the processed script is show to
the user, some parsing is performed as well to look for common errors so we
can point to the right input file.
This can only be done when the scc_output_file have some valid content
otherwise it will show invalid messages to the user.
Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
The previous fix for this issue was too broad, and impacted
all calls to the prefix check and removal. With this change,
we only expand the input on scc/spp operations that may
execute with relative paths.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
This reverts commit 9320c2a1aaa085e94abd15ede0d93ffdab578e9d.
|
|
Some options that are hardware related, can either be built in or
modules, and meet the criteria of being acceptable for a BSP.
To allow these options to not throw warnings, we create a new
configuration fragment to go along with the 'hardware.cfg' and
'nonhardware.cfg' special fragments.
This new fragment is called 'y_or_m_enabled.cfg' and is effective
for the fragments in its directory. The format is the same as
hardware/nonhardware.cfg, is simply a list of CONFIG_ options.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
If include directories (locations of layers) have relative
components, and the path to a patch doesn't share that relative
location, the detection of a common prefix can fail.
When the common prefix is incorrectly specified, it can be the
entire path to a patch. This results in everything being removed
and subtle errors (as this shouldn't happen).
By ensuring that both the relocation directories and patch path
are absolute, we can avoid this issue.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
We rename the commit date to be "applied" and clarify the help
message.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
In situations where a buid needs to be reproducible, it is
sometimes desireable to use a patches author date, versus the
time when it is applied. This generates a consistent hash
between different patch applications.
We introduce a new option to enable this functionality, but
don't change the default (the application date).
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
FileNotFoundError: [Errno 2] No such file or directory: 'kgit-create-buckets'
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
In the query path the error message should mention 'query' and not
'creation'.
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
If the kernel config option being queried is not found, we get the following
output:
[INFO]: looking for ['CONFIG_GPIO_SYSCON']
Traceback (most recent call last):
File "/path/to/kgit-config", line 671, in <module>
print( "no matching cfg files with option %s found" % regex )
NameError: name 'regex' is not defined
Update the queried-for variable (regex) so that it contains the option the
user is querying (oregex):
[INFO]: looking for ['CONFIG_GPIO_SYSCON']
no matching cfg files with option CONFIG_GPIO_SYSCON found
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Trevor Woerner <twoerner@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
With this version of kgit-config, we can run without an existing
meta-data repository and create a rough guideline of hardware
versus policy options using existing kgit-tooling. This isn't
as good as the yocto kernel-cache, but it is a starting point.
We can also now write BSP .scc files and hardware/policy fragments.
An example run follows:
% kgit-config create --bsp mybsp --savetemps --ksrc \
~/poky-kernel/linux-yocto.git/ --defconfig \
~/poky-kernel/linux-yocto.git/arch/x86/configs/x86_64_defconfig --outdir .
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
From the tools help:
usage: kgit-config [-h] [-v] [-s KSRC] [--create] [--query] [-k KMETA]
[-d DEFCONFIG] [-e ENTRYPOINT] [--savetemps]
...
kernel fragment generation/manipulation/query
positional arguments:
args <command> (create or query)
optional arguments:
-h, --help show this help message and exit
-v verbose
-s KSRC, --ksrc KSRC path to the kernel source
--create Create a minimal BSP hardware config
--query run a query against kernel meta data
-k KMETA, --kmeta KMETA
path to the kernel meta data
-d DEFCONFIG, --defconfig DEFCONFIG
path to the kernel defconfig
-e ENTRYPOINT, --entrypoint ENTRYPOINT
fragment to use as an entrypoint for
classification
(default is ktypes/standard/standard.scc
--savetemps don't delete temporary working directory
Overview:
kgit-config can be used to perform initial processing on a defconfig
to determine what options are "hardware" and are hence suitable to be
placed in a BSP, while leaving non-hardware/policy options to the
fragments contained within a kernel-cache repository.
It can also query a kernel-cache directory for details about which
fragments provide an option, and where they are included (which
is helpful when completing initial BSP fragments.
Examples:
# process a defconfig and output hardware options suitable for a BSP
% kgit-config create --ksrc ~/poky-kernel/linux-yocto.git/ --defconfig ~/poky-kernel/linux-yocto.git/arch/x86/configs/x86_64_defconfig --kmeta ~/poky-kernel/kernel-cache
# query a set of fragments for details on options
% kgit-config query --kmeta ~/poky-kernel/kernel-cache CONFIG_DEBUG_FS
[INFO]: looking for ['CONFIG_DEBUG_FS']
[INFO]: cfg files that have options matching regex: CONFIG_DEBUG_FS
[INFO]: 7 cfg files have option: CONFIG_DEBUG_FS
cfg file: cfg/fs/debugfs.cfg
included by: /home/bruce/poky-kernel/kernel-cache/cfg/fs/debugfs.scc
cfg file: cfg/debug/irq/debug-generic-irq-debugfs.cfg
included by: /home/bruce/poky-kernel/kernel-cache/cfg/debug/irq/debug-generic-irq-debugfs.scc
cfg file: cfg/debug/irq/debug-irq-domain.cfg
included by: /home/bruce/poky-kernel/kernel-cache/cfg/debug/irq/debug-irq-domain.scc
cfg file: cfg/debug/printk/debug-dynamic-debug.cfg
included by: /home/bruce/poky-kernel/kernel-cache/cfg/debug/printk/debug-dynamic-debug.scc
cfg file: features/systemtap/systemtap.cfg
included by: /home/bruce/poky-kernel/kernel-cache/features/systemtap/systemtap.scc
cfg file: bsp/beaglebone/beaglebone-non_hardware.cfg
included by: /home/bruce/poky-kernel/kernel-cache/bsp/beaglebone/beaglebone.scc
cfg file: bsp/intel-x86/intel-x86-acpi.cfg
included by: /home/bruce/poky-kernel/kernel-cache/bsp/intel-x86/intel-x86.scc
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
commit 6dd85ff178cd76851e2184b13e545f5a88d1be30 [kconfig: change
"modules" from sub-option to first-level attribute] broke parsing
in 5.13+ trees.
We add the new location to the parser so we can support both
types of module specifications.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
The classification of fragments should go in the following order:
- kconf include <type>
- general .kcf group classification
- specific hardware.cfg/non-hardware.cfg
With our previous singe pass through the queue, it wasn't possible
to give the specific values the last say.
Doing two passes allows us to give the final say to a specific
classifcation.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
The config.queue has typing hints inline with each fragment,
we should be using them to further classify the options, and
not only relying on the special hardware.cfg, etc, files that
are part of the meta data
We also should be checking for options that were set to a
non 'no' value, and that don't make it into the final .config,
since without that check it means we are missing some warnings.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
sys.exit(1) belongs to the error message above but has the wrong
indentation.
Signed-off-by: Sam Van Den Berge <sam.van.den.berge@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Being in both h/w and non-hardware leads us to results where we
can't easily silence a warning. So if we've added it to h/w we need
remove it from non-hardware. If we do it in both the hardware and
non hardware cases, the end result is that the last classification
wins, which is what we want.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Pass certain environment variables, meant to convey information about
the toolchain to use, as make variables when creating a kernel config
file. The kernel build system checks for toolchain capabilities when
generating the merged config file and a valid toolchain is required to
enable some options.
OpenEmbedded has attempted, for a couple years now, to specify the
compiler by setting the CC environment variable:
https://git.openembedded.org/openembedded-core/commit/?id=ff1bdd75d50f0ebac3d599e461685ace29559a82
This was unsuccessful due to the kernel's top-level Makefile explicitly
assigning variables, such as CC, and ignoring what's present in the
environment. It is possible to override this behavior by invoking make
with the '-e' option but the make manual discourages this:
Every environment variable that make sees when it starts up is
transformed into a make variable with the same name and value.
However, an explicit assignment in the makefile, or with a command
argument, overrides the environment. (If the ‘-e’ flag is specified,
then values from the environment override assignments in the makefile.
See Summary of Options. But this is not recommended practice.)
A simple test case is to create a defconfig while specifying clang as
the compiler to use:
$ make CC=clang-10 defconfig
...
Create a config fragment that attempts to enable
CONFIG_INIT_STACK_ALL_PATTERN which is currently only possible to enable
when building with clang:
$ echo "CONFIG_INIT_STACK_ALL_PATTERN=y" > init-stack-all.cfg
Specify clang in the environment when running merge_config.sh and see
that the option couldn't be enabled:
$ CC=clang-10 merge_config.sh .config init-stack-all.cfg
...
Value requested for CONFIG_INIT_STACK_ALL_PATTERN not in final .config
Requested value: CONFIG_INIT_STACK_ALL_PATTERN=y
Actual value:
Note that the less secure CONFIG_INIT_STACK_NONE option is enabled,
instead:
$ grep CONFIG_INIT_STACK .config
CONFIG_INIT_STACK_NONE=y
Signed-off-by: Tyler Hicks <tyhicks@linux.microsoft.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
If there's a mismatch, and an option is in non-hardware AND hardware,
then it means that it was globally classified as non-hardware, but
had a hardware.cfg that re-classifed it.
We should report these errors with --classify is passed.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Commented lines that contained CONFIG_FOO ... were being picked up
as option lines and reported as invalid.
We make the regex more explicit and only match on spaces before
CONFIG_ to declare if a 'is not set' line is invalid.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
This large commit is a re-work of symbol why to not only explain
why a symbol is set to a value, but to be able to process and
entire configuration queue and fragments to perform extended
analsysis and configuration tracking.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
LD often has parametes (like the sysroot), so we need to ensure it
is quoted, otherwise the parameters will be passed to make, triggering
an error.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
* since 5.4 kernel Kconfig will fail immediately when it detects
that LD points to gold linker:
scripts/Kconfig.include:39: gold linker 'i686-oe-linux-ld' not supported
* in OE we already pass bfd linker in KERNEL_LD variable to merge_config.sh
but we need to pass it also into the make call here
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
When spp is including a file, read returns a failure result if the
last line has no ending. While this is posix behaviour, we do want that
line.
As such, we both check the read return code AND a variable for non
zero content to ensure that we have the entire file included.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
strip_common_prefix() is the hot path that executes on every input file.
This patch sorts and uniqs the $include_paths by length descreasingly.
And with the sorted $include_paths, the for-loop inside strip_common_prefix
can break earlier, thus kernel_metadata task can be sped up by multiple times.
Signed-off-by: Zhaolong Zhang <zhangzl2013@126.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
As reported in https://bugzilla.yoctoproject.org/show_bug.cgi?id=12563,
the regex that matches valid CONFIG_ options was missing some of the
ones in net/netfilter/ipvs/Kconfig, and hence triggering invalid option
warnings.
By dropping the trailing space on the regex, we'll cover all the cases
for valid option.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Kconfiglib/* were under ISC license before they were imported
here from https://github.com/ulfalizer/Kconfiglib
Adjusting SPDX header to reflect that fact.
tools/* all have some sort of GPLv2 headers; adding SPDX header
to make it obvious.
This address bug #13334 :
https://bugzilla.yoctoproject.org/show_bug.cgi?id=13334
Change-Id: I243f2dd266a398f982798b771e74a67be70ecb52
Signed-off-by: William Bourque <wbourque@gmail.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
Modern git versions required an authot-script to be present
in the rebase-apply directory for git am to complete. So we
write out our values for use by git am.
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com>
|
|
The cmd line, <grep '^[ ]*\(menu\)*config '>, can't
match all expect config options.
This is because that it is not always a single space after 'config' in
kernel-source/*/.../Kconfig. e.g. "config IP_VS_IPV6" in
net/netfilter/ipvs/Kconfig
So we should change the cmd to grep '^[ ]*\(menu\)*config\s'.
Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com>
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
If a fragment has requested "# CONFIG_FOO is not set" and the
config does not appear in the final .config at all, this is not
a mismatch, it is intended.
To fix this, we modify the mismatch code to allow this combo.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Rather than forcing a set of non-hardware specified options to have
values set, we should also allow just a list of options. To do this,
we make the grep expression more broad and accept "CONFIG_FOO" rather
than requiring "CONFIG_FOO = y", etc.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
|
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com>
|
Bruce Ashfield
Alexander Sverdlin
Trevor Woerner
Volodymyr Babchuk
Jose Quaresma
Armin Kuster
svdb
Tyler Hicks
Martin Jansa
Zhaolong Zhang
William Bourque
Hongzhi.Song
Bruce Ashfield