aboutsummaryrefslogtreecommitdiffstats
path: root/meta/lib/oeqa
AgeCommit message (Collapse)Author
2017-06-28runcmd.py: unit testing for runCmd()Patrick Ohly
This covers the traditional API as well as the new output_log feature. While testing, it was noticed that killing hanging commands does not work when a shell is used to run the command(s). This might be worth fixing. (From OE-Core rev: 62489e58ca9975f58b48fc2bd8cf27fd22e25564) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-28commands.py: live output logging + result.error encoding fixPatrick Ohly
Tests that use bitbake("my-test-image") can run for a long time without any indication to the user of oe-selftest about what's going on. The test author has to log the bitbake output explicitly, otherwise it is lost in case of test failures. Now it is possible to use bitbake("my-test-image", output_log=self.logger) to get more output both on the console and in the XML output (when xmlrunner is installed). Example output: 2017-06-23 12:23:14,144 - oe-selftest - INFO - Running tests... 2017-06-23 12:23:14,145 - oe-selftest - INFO - ---------------------------------------------------------------------- 2017-06-23 12:23:14,151 - oe-selftest - INFO - Running: bitbake my-test-image 2017-06-23 12:23:16,363 - oe-selftest - INFO - Loading cache...done. 2017-06-23 12:23:17,575 - oe-selftest - INFO - Loaded 3529 entries from dependency cache. 2017-06-23 12:23:18,811 - oe-selftest - INFO - Parsing recipes...done. 2017-06-23 12:23:19,659 - oe-selftest - INFO - Parsing of 2617 .bb files complete (2612 cached, 5 parsed). 3533 targets, 460 skipped, 0 masked, 0 errors. 2017-06-23 12:23:19,659 - oe-selftest - INFO - NOTE: Resolving any missing task queue dependencies Because the implementation was already using threading, the same is done to decouple reading and writing the different pipes instead of trying to multiplex IO in a single thread. Previously the helper thread waited for command completion, now that is done in the main thread. The most common case (no input data, joined stdout/stderr) still uses one extra thread and a single read(), so performance should be roughly the same as before. Probably unintentionally, result.error was left as byte string when migrating to Python3. OE-core doesn't seem to use runCmd() with split output at the moment, so changing result.error to be treated the same as result.output (i.e. decoded to a normal strings) seems like a relatively safe API change (or rather, implementation fix). (From OE-Core rev: 00b8c7ff17cd8f1920728fdc2653068e63d71724) Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> merge: wait() Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-28selftest/archiver: only execute deploy_archives taskAndré Draszik
There should be no reason to execute a full build, as we're just interested in the deployment of the archives. The newly added tests already do the same. (From OE-Core rev: 78c2897dff7cd9fe2cab511549cb146d5231e573) Signed-off-by: André Draszik <adraszik@tycoint.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-28selftest/archiver: add tests for recipe type filteringAndré Draszik
The archiver used to be able to filter based on COPYLEFT_RECIPE_TYPES. Unfortunately, this got broken with the fix for https://bugzilla.yoctoproject.org/show_bug.cgi?id=6929 in commit ae9102bda398 ("copyleft_filter.bbclass: Allow to filter on name") Add two tests to prevent that from happening again. (From OE-Core rev: 709f02c5cb25983090251c6237bac4fc0a295c4f) Signed-off-by: André Draszik <adraszik@tycoint.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-28selftest/seltest.py: Add test to check imports from other layersJose Perez Carranza
This tests adds a check of selftest itself to verify if can add test from other layers. [YOCTO #9770] (From OE-Core rev: 4fe4c408246b9a4a563106d097876e6caefca694) Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-28meta: Fix return value checks from subprocess.call()'sMikko Rapeli
Python function subprocess.call() returns the return value of the executed process. If return values are not checked, errors may go unnoticed and bad things can happen. Change all callers of subprocess.call() which do not check for the return value to use subprocess.check_call() which raises CalledProcessError if the subprocess returns with non-zero value. https://docs.python.org/2/library/subprocess.html#using-the-subprocess-module All users of the function were found with: $ git grep "subprocess\.call" | \ egrep -v 'if.*subprocess\.call|=\ +subprocess\.call|return.*subprocess\.call' Tested similar patch on top of yocto jethro. Only compile tested core-image-minimal on poky master branch. (From OE-Core rev: 578c8205fd14c48c6d30ef2889d86f1b4aee060a) Signed-off-by: Mikko Rapeli <mikko.rapeli@bmw.de> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-28selftest/devtool: Don't use removed remake as test caseJussi Kukkonen
remake was removed from oe-core: use another recipe in the devtool extract test. (From OE-Core rev: dbf680ce09ce54f97c781fc7a8e5e05ed4706073) Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-28oe-selftest: add a test for upstream version check regressionsAlexander Kanavin
The test runs an upstream version check, and then compares the list of recipes that failed the check (i.e. those where latest upstream version could not be established) against the list of known-broken upstreams. Mismatches either way (upstream check failed, recipe not in the exception list, or upstream check worked, recipe in exception list) fail the test. [YOCTO #11031] (From OE-Core rev: 544e04a11ab6c26a367ac0b67774bf7ffdf43c88) Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-23oeqa/selftest/context: Reload testlayer_path when meta-selftest isn't addedAníbal Limón
When add meta-selftest by the script the testlayer_path needs to be reloaded to avoid None value. (From OE-Core rev: d1120849e12c46aa46b02905a3f43c0853bd11a0) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-23oeqa/core/loader.py: Fix _make_failed_test for python >= 3.4.4Aníbal Limón
Python unittest change the signature of the _make_failed_test after python 3.4.4 don't pass the method name. (From OE-Core rev: 767b68e6ca22512ff80e6fbc42154f3f0c2206c0) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-23oeqa/core/threaded: Don't assume that results exists on logDetailsAníbal Limón
(From OE-Core rev: 7beed75c97a78e945e44a55b28f0f463cd6c8dcd) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-23selftest: Add Testopia ID to test casesJose Perez Carranza
Add decorator @OETestID() with proper Tesopia TC ID to the test cases that did not have it set. (From OE-Core rev: d7bc697534db911a3ce98537d772d87482a0f702) Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-23selftest/devtool: replace file assertTrue/False with assertExists/NotExistsYeoh Ee Peng
Current osselftest print confusing assertion message when using self.assertTrue(os.path.exists(filepath)) to test file path, example of confusing assertion message: AssertionError: False is not true Replce assertTrue/assertFalse with assertExists/assertNotExists to test file path, this will improve assertion message and simplify coding, self.assertExists(filepath) will print below AssertionError: <filepath> does not exist [YOCTO #11356] (From OE-Core rev: 08b0702492536d41d3cf2c9c05d1fab36d32a566) Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14selftest/sstatetests: Fix potential failure on uniprocessor machinesRichard Purdie
It was pointed out +1 is safer than -1 for systems with one processor. (From OE-Core rev: 78041e20e43d9583448ff31f8b9b1c6157da8625) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14selftest: add test_wic_rm test caseEd Bartosh
Added test case for "wic rm" functionality. - remove file from vfat partition - remove directory from vfat partition (From OE-Core rev: fc42914d426ad2dad8c3026668726c003b3ab10b) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14selftest: add test_wic_cp test caseEd Bartosh
Added test case for "wic cp" functionality. - copy file to vfat partition - copy directory to vfat partition (From OE-Core rev: cb416d1feea042bcdedc9f522d588fef2c4929bc) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14selftest: add new test case test_wic_lsEd Bartosh
Tested 'wic ls' functionality: - list of image partitions - list of directory content of vfat partition (From OE-Core rev: 3b271f6a40d7ef93fff9700dd8ac98bddd92d581) Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-14sstatetests: Use higher parallelism valueRichard Purdie
Since the processing code for signature generation is now threaded, use higher thread values as examples in this code for better performance. (From OE-Core rev: f68ec7191546474f0bd688e57d2381a8e92be617) Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-13oeqa/core/loader: Use full and small module name on filteringAníbal Limón
The small module name was added to support run a whole suite that has more that 3 levels in the test case name, but this broke the behaviour for use a full test case name. [YOCTO #11632] (From OE-Core rev: 9ab20ceb5801bee8dd8b218b3928720da5e1d403) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-13oeqa/core/runner: Don't log details twice if test failsAníbal Limón
The details of a test failure is upper on the unittest output so don't log twice the actual failure. [YOCTO #11622] (From OE-Core rev: 0f2e81c2a4458ad0ec6bab2710952ac2c2bbf1af) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12oeqa/sdk/cases: Fix skip of buildgcalculator testAníbal Limón
The tc.hasTargetPackage uses a re.search to see if gtk+3 is on the manifest but + in regex means 1 or more causing the test to be skipped. (From OE-Core rev: 271cd99d00bde0b9f2aa27141acbe06812f34638) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12oeqa/sdk/cases: Added validation for SDK compatibility tests with eSDKFrancisco Pedraza
The manifests for eSDK are generated using shared states so there is a need to validate to different "packages names" into the test cases. For example for perl: SDK provides nativesdk-perl eSDK provides perl-native [YOCTO #9090] (From OE-Core rev: 8db06dd1290dd53d626050879c9c306f95d76ac2) Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12oeqa/core/loader: Allow unittest.TestCase's to be executedAníbal Limón
Currently there was a restriction to only execute tests that's inherits from OETestCase but in some circunstancies the features from the OEQA framework isn't needed so we need to support basic unittests. [YOCTO #10828] (From OE-Core rev: baac26f1b36e89e07637b738dd31ec7356f05a02) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12oeqa/cases/oelib: Change default case class to unittest.case.TestCaseAníbal Limón
Some tests doesn't need call bitbake so it is better to use the basic unittest case class. [YOCTO #10828] (From OE-Core rev: 4d01610f36eaee8da3126bb5045856279371fd17) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-12oeqa/core/loader: Fix filtering on test modules with submodulesAníbal Limón
Our filtering allows to specify which tests to run using, <module_name>.[test_class].[test_name] But the module name logic was restricted to only accept one level, for example: runtime_test vs oelib.types, to support multiple submodules use only the first part for filtering. This allows to run the whole tests in a module with more than tree levels. Due to the ambiguity on the test filtering options with test cases with more than tree levels the supported sytnax is, <module> or <module>.[submoduleN].[test_class].[test_name] [YOCTO #11632] (From OE-Core rev: cf2ee12b007e5570959ccfbb643159b21d90426e) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-09oeqa: Change the order to logDetails and logSummaryAníbal Limón
Is better to log the summary at end to see in an easy way the actual result of the test run. [YOCTO #11622] (From OE-Core rev: 4e3ab36e8c90abc740cce1ba31faf6595116e1e2) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-09selftest/devtool: Modify test to use default configJose Perez Carranza
Modify “test_devtool_virtual_kernel_modify” to be executed with default configuration defined on oe-selftest test suite. A shorter string was added to the the header file to avoid overlapped sections when building kernel for qemux86-64. [Yocto #11300] (From OE-Core rev: a8bda790fded43eac72b2ad76d47c27363f02070) Signed-off-by: Jose Perez Carranza <jose.perez.carranza@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-09runtime-test.py: Split different tests inside selftest.pyHumberto Ibarra
There were two completely different tests inside selftest.py, and the only reason for them to be together was that both needed the same mechanism to execute (run testimage from within selftest) This fixes the design issue and error-prone situation by separating these tests. In add a new module for dnf-runtime and also has an extra time added to running the tests, but it is minimal since the builds reuse data from previuous build. [YOCTO #11436] (From OE-Core rev: 2f6a506018973f0eaf1306555df3e74a3ab25680) Signed-off-by: Humberto Ibarra <humberto.ibarra.lopez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-07buildhistory: skip tests if GitPython module is missingLeonardo Sandoval
Catching a possible exception when importing the git module, allows us to skip test if the latter is missing on the host. Also, import oe.buildhistory_analysis inside fuctions because this module also needs git to work correctly. [YOCTO #11620] (From OE-Core rev: 186882ca62bf683b93cd7a250963921b89ba071f) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06oeqa/core/loader: Call parent init method in OETestLoaderAnibal Limon
After Python 3.5 the init method contains initialization of _loading_packages set to avoid infinite loops in recursive loading. Fix, Traceback (most recent call last): File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/oe-selftest", line 70, in <module> ret = main() File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/oe-selftest", line 57, in main results = args.func(logger, args) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/context.py", line 214, in run rc = self._internal_run(logger, args) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/context.py", line 175, in _internal_run self.tc.loadTests(self.module_paths, **self.tc_kwargs['load']) File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/context.py", line 51, in loadTests self.suites = self.loader.discover() File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/loader.py", line 268, in discover pattern='*.py', top_level_dir=path) File "/usr/lib/python3.5/unittest/loader.py", line 341, in discover tests = list(self._find_tests(start_dir, pattern)) File "/usr/lib/python3.5/unittest/loader.py", line 398, in _find_tests full_path, pattern, namespace) File "/usr/lib/python3.5/unittest/loader.py", line 473, in _find_test_path self._loading_packages.add(name) AttributeError: 'OETestLoader' object has no attribute '_loading_packages' (From OE-Core rev: 17b832ba05d640570a773adbff2fa4e9ff83fce3) Signed-off-by: Anibal Limon <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06oeqa/runtime/context.py: Add support to specify port in target_ipAníbal Limón
(From OE-Core rev: 7fe4c074611eb21abdf811d8f216d4df7ab6a3ea) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06oeqa/selftest/cases: Migrate test cases into the new oe-qa frameworkLeonardo Sandoval
New framework has different classes/decorators so adapt current test cases to support these. Changes include changes on base classes and decorators. Also include paths in selftest/__init__.py isn't needed because the loader is the standard unittest one. (From OE-Core rev: ddbbefdd124604d10bd47dd0266b55a764fcc0ab) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06scripts/oe-selftest: Migrate to new framework into oeqa.selftest.contextLeonardo Sandoval
The new OEQA framework aims to re-use code into the different Test components. The previous oe-selftest implements it-self loading, run, and list test cases in a non-standard way (unittest base) and other functionalities like logging that is now on oeqa core. This ends on a compact oe-selftest script. All needed command line options was migrated but there are some of them pending of implementation and others deprecated. Deprecated options: list-tags: The tag functionality into the old oeqa framework isn't work, the selftest doesn't has tag decorators. {run, list}-tests-by: Ambiguos options it accepts all the posibilites module, class, name, id or tag. Remaining to implement: coverage: It enables covrage reports over a test run, currently isn't on on use and some bugs [1], i filed a bug to add support to OEQA core module in this way other Test components could enable it. repository: It push XML results into a git repository and isn't in use, i filed a bug to implement this into OEQA core module. [2] [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11582#c0 [2] https://bugzilla.yoctoproject.org/show_bug.cgi?id=11583#c0 (From OE-Core rev: 3b2a20eee4a39f40287bf67545839eaa09fc892d) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06oeqa/selftest/case: Don't figure out the testlayer insideAníbal Limón
The test layer is added at init of selftest and is the same across test runs so pass it as a attr in the context. (From OE-Core rev: 8441da8fbc7ff2237d9ccd3a5c5117345f5c87b2) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06oeqa/selftest/case.py: Remove machine selection logicLeonardo Sandoval
The machine selection is an operation that needs to be made in every test run, the best place to it is on the context module. Use self.tc.custommachine variable instead of use environment. SIgned-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> (From OE-Core rev: f295b70ace0ffc28256140a21af5c3a8903297cb) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06scripts/oe-selftest: Move {add,remove}_include files to caseLeonardo Sandoval
The oe-selftest creates include files to store custom configuration to make specific tests, every class executes a different test and may be uses custom configuration. So move to case class in order to simplify oe-selftest script and later implement later a build folder per class. (From OE-Core rev: 1130b40c3dfa65e7ece08a95b3941e4d1d20bcf0) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06oeqa/selftest/case: Migrate case class to the new OEQA frameworkLeonardo Sandoval
Summary of the changes: - Use OETestCase as base class instead of unittest.TestCase - Remove LogResults decorator the new framework provides logging into the core functionality. - Logger is now self.logger instead of self.log - Move comments into docstrings in several help methods - Use get_test_layer() method instead of access monkey patched variable in old oeSelfTest case class. (From OE-Core rev: c38cab77893f9d8fd505f050cc880a15677b73db) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06oeqa/selftest: Move base class to case moduleAníbal Limón
To match the new structure of the OEQA framework. (From OE-Core rev: 0d3d97414c8166d09065f6f7f45e3260ce405692) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-06scripts/oe-test: Move load_test_components to oeqa.utilsAníbal Limón
In order to maintain compatibility with oe-selftest, the load_test_components needs to be re-used, so the script executor needs to pass to only load components supported by certain script (oe-test, oe-selftest). (From OE-Core rev: d6b78ae711b93b4059690320cb8d821aaadd1684) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-05oeqa/selftest: lock down Meson git revision for reliabilityRoss Burton
The test_recipetool_create_github test fetches HEAD of the repository so upstream changes can (and do) break the test. Avoid these problems by passing the rev= argument in the URL to lock the checkout to the same version that is fetched in the github_tarball test. Also pass the commands to runCmd() as a list instead of a string, the semicolon in the URL needs more quotes if the shell is involved and passing a list bypasses the shell entirely. (From OE-Core rev: b35bedd209092432c560e998043b6a8c5c2e4d34) Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-05oeqa/core: Improve grammar in README and loader commentsRandy MacLeod
(From OE-Core rev: 48cd7ff7ae61ca5772a42ece9fce51976dd0d790) Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/core/context: Include a _pre_run methodLeonardo Sandoval
This pre runner will serve to allow Test components executes code previously of the run a suite. (From OE-Core rev: bafb7e221d40d7a87a02cec8a97d98eec7c23438) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/core/context: Omit docstring output on XMLResultLeonardo Sandoval
By default, the xml runner class prints out the docstring for every unit test but it order to keep the same format as the standard runner, avoid docstring output setting descriptions to False. (From OE-Core rev: 05b16219698fbd30ec76697e1b3be6d31e8f8878) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/core/context: Use a default iterable modules objectLeonardo Sandoval
The OETestLoader.modules must be an iterable object, otherwise checking presence of specific modules with the 'in' operator fails with the following command/error: % oe-test core .. - core - INFO - Running bitbake -p Traceback (most recent call last): File "..poky2/scripts/oe-test", line 108, in <module> ret = main() File "..poky2/scripts/oe-test", line 93, in main results = args.func(logger, args) File "..poky2/meta/lib/oeqa/core/context.py", line 235, in run self.tc.loadTests(self.module_paths, **self.tc_kwargs['load']) File "..poky2/meta/lib/oeqa/core/context.py", line 53, in loadTests self.suites = self.loader.discover() File "..poky2/meta/lib/oeqa/core/loader.py", line 204, in discover pattern='*.py', top_level_dir=path) File "/usr/lib64/python3.4/unittest/loader.py", line 275, in discover tests = list(self._find_tests(start_dir, pattern)) File "/usr/lib64/python3.4/unittest/loader.py", line 339, in _find_tests yield self.loadTestsFromModule(module) File "..poky2/meta/lib/oeqa/core/loader.py", line 266, in loadTestsFromModule and module.__name__ in self.modules \ TypeError: argument of type 'NoneType' is not iterable (From OE-Core rev: 2bb02cd3d6ec6c2c22ab6b567e2985015655fae6) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/core/context: Raise exception when a manifest is specified but missingLeonardo Sandoval
(From OE-Core rev: e7458dd24cb7464852fb7f5357d9108d5c052fa6) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/core/case: fix typo on comment about exception nameLeonardo Sandoval
(From OE-Core rev: 943a2a30e84660b412df4f1d60fb7e2e46764c50) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/core/README: Improve documentationLeonardo Sandoval
Most of the text include on this text refactor came from [1]. [1] http://lists.openembedded.org/pipermail/openembedded-architecture/2016-December/000351.html (From OE-Core rev: 77635ffa3b621add9894ed247dafe146f4af1e84) Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/runtime/case: Don't use OEQA framework internal methodsAníbal Limón
The OEQA framework has internal methods for provide functionality in decorators so Test components aren't expected to override it. Use the base unittest methods for setUp and tearDown. (From OE-Core rev: 21df9f0f6d6272adc6131cdc113000a5e6ac9d46) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/core: Add support for run tests by module, class and nameAníbal Limón
This will enable only run certain module tests and filter by class and test name. (From OE-Core rev: 0f7531803aa6ffef2b7c5be9acee3efe7d2974ef) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-06-02oeqa/core: Add list tests support in context and runnerAníbal Limón
A common operation is to list tests, currently only selftest support it, this changes enables this functionality into the core framework. (From OE-Core rev: 7e803f1a855d3091a772b13efd3cc8e9c0c766e9) Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-06-30 14:03:43 +0000