| Age | Commit message (Collapse) | Author |
|---|
|
Fix warnings from oe-selftest -j:
/usr/lib/python3.10/os.py:1030: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
return io.open(fd, mode, buffering, encoding, *args, **kwargs)
Remove the option since it clearly doesn't do much.
(From OE-Core rev: 6b872ee72942951fd464c4c6cb9eadcb9b4749c1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
"bitbake -e" executions from get_bb_var calls are slow and slow down oe-selftest
startup. Rationalise the code to avoid them and minimise the number of "parsing"
locations we use by caching key variables and passing them around more.
This was particularly problematic with oe-selftest -j usage since it would
have multiple bitbake -e executions per process making parallel usage
particularly slow.
(From OE-Core rev: 3689cadeb07d76e66f97d890e844f899f69666fe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When running oe-selftest and seeing the end of a running log, it is
extremely helpful to know if there have been failures or not to save
looking at the rest of the log. Add the number of failures to the summary
line so that people monitoring builds have an easier time before the end
totals are printed.
(From OE-Core rev: 6b23996911d91f7f99774646c6db9f3490b4cb62)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Where there isn't a copyright statement, add one to make it explicit.
Also add license identifiers as MIT if there isn't one.
(From OE-Core rev: bb731d1f3d2a1d50ec0aed864dbca54cf795b040)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This file dates back to 2016. Half of the functions have never been used,
the rest are used in one place and have now been replaced.
(From OE-Core rev: 5a053b2a84e7a671925fb4a78005145786c57f6a)
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If oe-selftest is run with -j, the output to stdout/stderr is being
lost at present. Capture this and display it upon test failure. We
have code that previously tried to enable this but it wasn't functioning
correctly. This should give more usable error reports on the autobuilder.
This code will mix stdout and stderr as the output is streamed from the test
server without markup. This is most in keeping with subunit/testools though
and the easiest way to handle the various challenges here as far as I can
see.
(From OE-Core rev: 6a954ce5834c8026adecff8478c3d827640bc647)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Each time I look at this code I get confused about what the different
variables represent. Rename a few of them to better indicate what they
represent.
(From OE-Core rev: e39d97c0b191add9281bac463ca059685288c81a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
If oe-selftest is run without -j, the separate build directory "build-st"
isn't cleaned up afterwards. Mirror the behaviour of the -j option to
handle this the same way, only preserve upon failure.
To do this, the remove function needs to be moved to the selftest
context module so that it can be accessed without requiring the
testtools and subunit modules the -j option requires.
A dummy wrapper class is used to wrap the tests and clean up afterwards.
[YOCTO #13953]
(From OE-Core rev: 1b376ade430d40d3cfe9c18f200c764d622710e5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
(From OE-Core rev: 3d5aa170d2e88b852bd2a4452aab9311a24badef)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
non-concurrent selftest
Currently oe-selftest reuses the current build directory and the concurrent
version run with -j does not.
Standardise and use a separate new build directory in both cases. This will lead
to simpler code and more reliable user run tests.
(From OE-Core rev: 50238b0717b04e0a1fa69d618e8c8aa8445a80b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
see: https://bugzilla.yoctoproject.org/show_bug.cgi?id=13663
When running oe-selftest in concurrency mode (e.g. with oe-selftest
--run-tests oescripts.OEPybootchartguyTests -j 4), if a skip occurred
during setUpClass() rather than within individual tests, the entire
suite would show "UNKNOWN" as each test's result. This is because
subunit doesn't know how to handle skips outside of individual tests.
An example of where this occurs is when running the above call to
oe-selftest in concurrency mode on a host machine that does not have
python3-cairo installed.
Patch subunit inside concurrencytest.py to provide a method called
outSideTestaddSkip, which will allow subunit to correctly detect the
skip in setUpClass().
(From OE-Core rev: 9b8734b584d6e8d9c32ff2a721b29f3f3e61cca7)
Signed-off-by: Trevor Gamblin <trevor.gamblin@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're seeing huge delays on the autobuilder during oe-selftest builddir
deletion. For example there is a currently running selftest we could do
with the results from and its been going 13 hours, at least 8 of which
was in deletion of the builddirs.
There are a variety of ways we could solve this problem however the
autobuilder has a mechanism for deferred deletion, "clobberdir" which
it already uses for this kind of work.
Whilst in general hardcoding things like this is horrible, I believe
in this case the benefits (and resulting improvements on my sanity
if nothing else) mean this is a case where we should do it.
If/as/when someone can come up with a better solution that is fine
and this can be replaced.
(From OE-Core rev: 51a84937d32a85cbdb7d5b1d1ec69a290d0c303b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Handle the streaming of exception content with details data. The
testtools package allows both 'err' and 'details' kwargs but can only
pass one of them to the parent.
To handle the passing of exception traceback and details data at the
same time, encode the traceback into the details object and remove the
'err' arg from the add* result call. This encodes the traceback similar
to how 'err' is handled without any details object. Decoding is already
done by testtools when the traceback is encoded in the details object.
(From OE-Core rev: 3613451825b251784b7673d89db465b9782c3a31)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Autobuilder type infrastructure can benefit from deletion of certain files as
background IO due to the way Linux filesystem priority works.
We have problems where build directories as part of oe-selftest being
delete starves the running tasks of IO to the point builds take much
longer to compelte.
Having this option of running the deletion at "idle" helps a lot with
that. Use the new option added to bb.utils.prunedir().
(From OE-Core rev: d41e7018be56902b7a1be4590e468cd15e02a3b5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Implement handling of extra result (e.g. ptestresult) collection with
the addition of a "extraresults" extraction function in OETestResult. In
order to be able to serialize and deserialize the extraresults data,
allow OETestResult add* calls to take a details kwarg. The subunit
module can handle cross-process transfer of binary data for the details
kwarg. With a TestResult proxy class to sit inbetween to encode and
decode to and from json.
(From OE-Core rev: b0831d43606415807af80e2aa1d0566d0b8c209c)
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently setupClass errors were not being mapped back to the failing tests
and they were hence being marked as UNKNOWN and the test statistics were
inaccurate.
This is because whilst the errors were being encoded into the test results
stream, the decoder doesn't cope with an error outside a testStart event.
We patch in an addError handler to the outsideTest parser so that this
does get handled in a way similar to the non-concurrent case.
It would be nice if we didn't have to do this but there doesn't seem
to be any other way to fix this other than forking subunit.
We also make a minor change so another of our changes can cope with
tests without a start time.
(From OE-Core rev: 8f7352ed9c1a3e82689b842b7f87e469ebf2e48f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This adds SPDX license headers in place of the wide assortment of things
currently in our script headers. We default to GPL-2.0-only except for the
oeqa code where it was clearly submitted and marked as MIT on the most part
or some scripts which had the "or later" GPL versioning.
The patch also drops other obsolete bits of file headers where they were
encoountered such as editor modelines, obsolete maintainer information or
the phrase "All rights reserved" which is now obsolete and not required in
copyright headers (in this case its actually confusing for licensing as all
rights were not reserved).
More work is needed for OE-Core but this takes care of the bulk of the scripts
and meta/lib directories.
The top level LICENSE files are tweaked to match the new structure and the
SPDX naming.
(From OE-Core rev: f8c9c511b5f1b7dbd45b77f345cb6c048ae6763e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The previous code assumed builddir and meta-selftest are in the same dir, but
this isn't always true, builddir can be anywhere, use get_test_layer() to
locate meta-selftest can fix the problem.
(From OE-Core rev: 56d2493a9adfcc47ae7e265439e05ff42cdbbbbf)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Avoid an unclosed file per thread warning when running selftests concurrently
by closing the result stream.
(From OE-Core rev: 33a4a076e8aa72a872807332501e7f5ae1cee0e2)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Without this, error output such as that in the teardown can be lost
and processes may recieve signals they're not expecting causing other
strange errors.
(From OE-Core rev: 1e3f44737a15feb3128ba7fc0dbe896dd8782e07)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows oe-selftest to take a -j option which specifies how much test
parallelisation to use. Currently this is "module" based with each module
being split and run in a separate build directory. Further splitting could
be done but this seems a good compromise between test setup and parallelism.
You need python-testtools and python-subunit installed to use this but only
when the -j option is specified.
See notes posted to the openedmbedded-architecture list for more details
about the design choices here.
Some of this functionality may make more sense in the oeqa core ultimately.
(From OE-Core rev: 326ababfd620ae5ea29bf486b9d68ba3d60cad30)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This is a complementary fix to commit 7c552996:
[ meta: remove True option to getVar calls ]
it intended to remove all True option to getVar calls, but there are
still some remaining.
Search made with the following regex: getVar ?\((.*), True\)
(From OE-Core rev: 87d03ffe03d6f01e360bfd51714be96e62506e0a)
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
With the change to the new framework data store dependecy was
removed, instead a new file is generated and used in testimage.
When testing builds from the autobuilders the test data values
are from the autobuilder, including the paths.
Some tests require paths to current environment in order to run,
this commit will update such paths and fix the error of running
images donwloaded from autobuilders.
[YOCTO #10964]
(From OE-Core rev: 26ad5105fc2ce03b7ee8ecc6911fd40a52bd573a)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This allows to use the package install feature with
the new OEQA framework.
[YOCTO #10234]
(From OE-Core rev: 077dc19445574457769eb4f231de97e8059cb75e)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This will add functions to get module file path from a test
case or a complete suite.
[YOCTO #10234]
(From OE-Core rev: 11dd49cb9faaf6588f045083ab5bd6679e2eafc1)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
misc: Functions for transform object to other types.
path: Functions for path handling.
test: Functions for operations related to test cases and suites.
[YOCTO #10232]
(From OE-Core rev: 102d04ccca3ca89d41b76a8c44e0ca0f436b7004)
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
Richard Purdie
Ross Burton
Steve Sakoman
Trevor Gamblin
Nathan Rossi
Robert Yang
Ming Liu
Mariano Lopez