| Age | Commit message (Collapse) | Author |
|---|
|
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
LicFilesChkSum.test_lic_files_chksum_modified_not_mentioned.fail's
commit message suggests it will pass, so fix that. Also rename other
tests to be consistent with the file numbering scheme when more than one
test case is available.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Use selftest-hello as the target for test_src_uri_left_files pass and
fail cases, so that the tests work properly.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Update the existing lic_files_chksum_presence patch files so that they
use the MIT license and (in the case of the pass file) actually include
a proper LIC_FILES_CHKSUM.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The actual test for these files was removed in fab5c3ad7, but the files
were left around. It could be added back in the future, but since
bitbake already checks this it may end up being an extra point of
failure that isn't worth maintaining.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The @classmethod decorator is unnecessary for the test and actually
causes positional argument errors when running patchtest:
|Traceback (most recent call last):
| File "/usr/lib64/python3.11/unittest/case.py", line 57, in testPartExecutor
| yield
| File "/usr/lib64/python3.11/unittest/case.py", line 619, in run
| self._callSetUp()
| File "/usr/lib64/python3.11/unittest/case.py", line 576, in _callSetUp
| self.setUp()
| File "/workspace/yocto/patchtest-oe/tests/test_mbox_cve.py", line 40, in setUp
| self.skip('No new CVE patches introduced')
|TypeError: Base.skip() missing 1 required positional argument: 'issue'
|None
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Include XPASS counts in the total number of tests in the selftest
results.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Add a test for ensuring there is a CVE tag in the mbox commit message
(in addition to the carried patch) by doing the following:
- Create parse_cve_tags.py with pyparsing patterns similar to
parse_signed_off_by
- Add test_mbox_cve.py to check for CVE tags in the mbox commit message
- Modify the CVE.test_cve_presence_in_commit_message files to be
consistent with other test files, i.e. by testing modifications
against selftest-hello in oe-core
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Add XPASS as a test result distinct from PASS so that false passes can
be identified.
Also reorganize the result output so that the XPASS and XFAIL totals are
printed before any others:
============================================================================
Testsuite summary for patchtest-oe
============================================================================
\# TOTAL: 17
\# XPASS: 14
\# XFAIL: 15
\# PASS: 0
\# FAIL: 0
\# SKIP: 2
\# ERROR: 0
============================================================================
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The current selftest output is not very readable and isn't consistent
with the way other project test standards (e.g. ptest) provide results.
Refactor the selftest script so that it prints results like so:
XFAIL: Summary.test_summary_presence (Summary.test_summary_presence.fail)
PASS: Author.test_author_valid (Author.test_author_valid.2.pass)
PASS: Bugzilla.test_bugzilla_entry_format (Bugzilla.test_bugzilla_entry_format.pass)
No test for=SrcUri.test_src_uri_checksums_not_changed.fail
No test for=CVE.test_cve_presence_in_commit_message.fail
PASS: SignedOffBy.test_signed_off_by_presence (SignedOffBy.test_signed_off_by_presence.pass)
And add a summary like that found in ptests:
============================================================================
Testsuite summary for patchtest-oe
============================================================================
\# TOTAL: 31
\# PASS: 14
\# SKIP: 2
\# XFAIL: 15
\# FAIL: 0
\# ERROR: 0
============================================================================
Note that tests where the result is "No test for X" are not yet counted,
and will be handled in a future patch.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Similar to the PatchSignedOffBy test patches, these had an incorrect
path listed for the actual patch file added. Fix that and regenerate
them so the tests pass again.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The previously-added files were actually failing due to a mismatch in
the carried patch path. Fix this and regenerate them so that they pass
with the patchtest command.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Generate two new pass/fail patch files for the
lic_files_chksum_modified_not_mentioned test and remove the extra pass
file, so that the files no longer fail the merge_on_head test. Note that
these tests actually require the meta-selftest layer to be added to
bblayers.conf for the tests to work correctly.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Make selftest output more informative by doing the following:
- For successes, list the test run and the patch file being tested
- Warn if there is a mismatch between expected and actual results
- Print the patchtest invocation used on a mismatched result
- Use upper-case indicators for PASS, FAIL, SKIP (matching other parts
of the project)
Old output style (only printed result on mismatch):
patch=MboxFormat.test_mbox_format.fail expected=fail actual=pass cmd=patchtest /workspace/yocto/patchtest-oe/selftest/files
New output style:
Running 'CommitMessage.test_commit_message_presence' against file '/workspace/yocto/patchtest-oe/selftest/files/CommitMessage.test_commit_message_presence.fail': FAIL
Mismatched result for test 'SrcUri.test_src_uri_left_files' against patch 'SrcUri.test_src_uri_left_files.fail' - expected FAIL, got SKIP
The test command was: 'patchtest /workspace/yocto/patchtest-oe/selftest/files/SrcUri.test_src_uri_left_files.fail /workspace/yocto/openembedded-core /workspace/yocto/patchtest-oe/tests'
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Enable this test so that patchtest can warn if a shortlog is longer than
200 characters.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The "pass" file for test_series_merge_on_head was failing because it
lacked a commit message, now that the test for commit message presence
is enabled. Add one so that it passes again.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The "fail" test file was actually passing on tests because of a mistake
when the new version was created. Modify the file so it fails the
test_mbox_format test, with the caveat that it will also cause the
merge_on_head test to fail.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
This has been marked as auto-skip previously, but since patch
submissions ideally come with some sort of description in the commit
log, the test should be enabled.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Add new patch files that will pass the merge on head test.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Make the tests files for signoffs in mboxes (not carried patches) use
selftest-hello instead of another recipe that is more likely to change.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
This was added previously but it makes no difference for the output of
results, so remove it to avoid later confusion.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Modify the shortlog test files to use selftest-hello from oe-core,
similar to other revised tests.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Copy the previously-added CVE check files and modify them to fail/pass
the patchtest signed-off-by presence check for carried patches.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Add a new patch file that should always merge for the pass case because
it modifies selftest-hello (which shouldn't change), while replacing the
failing case with a new file that should *only* fail the merge case.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Replace the existing test patch files with new ones that change
selftest-hello and therefore aren't reliant on recipes that will change
often.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Copy and modify the new patch files from the CVE tag presence test so
that they validate tag formats instead.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Add new selftests for the presence of CVE tags that do not rely on a
real recipe whose version and/or configuration may change over time.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The old files are unclear on whether they should actually pass/fail and
why, so create some new ones using oe-core's
meta-selftest/recipes-test/selftest-hello as a baseline.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The selftests invoke patchtest in a way that doesn't match the current
usage, and therefore do not operate as expected. Until the tests are
merged into openembedded-core, allow selftest to take a single argument
(the target directory) to pass to patchtest so that selftest can run.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
This was missed in 9abbb0e8, and without it the test doesn't work.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
The license is clearly GPL 2.0 and there is no "or later" option, the patch
adding the license identifiers was incorrect. Fix the license idetifers
and clean up the boilerplate whilst we're here.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The patchtest repo is being refactored to improve maintainability, so
modify the test cases in oe-core to match.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
We had both a COPYING and a LICENSE file, but the latter simply pointed
to the former, so just remove the redundancy.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Similar to test_mbox_signed_off_by and test_patch_signed_off_by, replace
the regex object with the use of pyparsing's AtLineStart to search for
the Upstream-Status tag.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
Instead of using a regex search, use pyparsing's search_string
functionality and define a new patch_signed_off_by pattern to search the
patch file with, like previous the test_mbox_signed_off_by fix.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
This fixes a consistent issue with false failures being reported when
testing for authors' signed-off-bys in mboxes. Instead of manually
compiling a regex pattern, use pyparsing's AtLineStart to check for the
signature's presence at the beginning of the line.
Signed-off-by: Trevor Gamblin <tgamblin@baylibre.com>
|
|
[YOCTO #13532]
Signed-off-by: Changqing Li <changqing.li@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
* Switch http links to https.
* Update the yocto mailing list address after the move to groups.io.
* Minor style fixes.
Signed-off-by: Paul Barker <pbarker@konsulko.com>
|
|
Signed-off-by: Paul Barker <pbarker@konsulko.com>
|
|
The current logic for checking cve tag is not correct. It errors
out if and only if the patch contains a line which begins with
CVE-YYYY-XXXX and contains nothing else.
It will not error out if the patch contains no CVE information, nor
will it error out if the patch contains line like below.
'Fix CVE-YYYY-XXXX'
I can see that the cve tag checking logic tries to ensure the patch
contains something like 'CVE: CVE-YYYY-XXXX'. So fix to implement such
logic.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com>
|
|
Renames patches, usually from recipes upgrades, need extra care becase the
unidiff library only provides the old name (through the is_modified_file predicate)
and patchtest, on the second run (when patch is merged), the datastore contains
the new filename so we cannot compare directly. To overcome this problem, use
'basenames', which is filename without the PV part.
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
|
|
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
|
|
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
|
|
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
|
|
This check does not really makes sense because bitbake itself does this
test (the checksums from the downloaded tarball are check againts those
define on recipe).
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
|
|
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
|
Trevor Gamblin
Richard Purdie
Changqing Li
Paul Barker
Chen Qi
Alexander Kanavin
Leonardo Sandoval