Age | Commit message (Collapse) | Author |
|
Commit 56ed1082c4914c0ef3c72bfd609d68cc850557f1 moved
the pipe-creation into the loop and resetted rc before each test,
such that only the result code of the last test was taken into
account for the final verdict.
Only set rc = -1 if pipe() fails to keep collected test-errors.
Signed-off-by: Christian Hohnstaedt <christian@hohnstaedt.de>
Signed-off-by: Anibal Limon <anibal@limonsoftware.com>
|
|
Move chdir outside run_child to better handle when fails output an error
message plus remove the need to strdup inside run_child (already have
ptest_dir).
Signed-off-by: Anibal Limon <anibal@limonsoftware.com>
|
|
When get_available_ptests uses realpath to support symlinks, this is
expected to not fail because usage of stack memory, if fails prints the
error and exit.
Signed-off-by: Anibal Limon <anibal@limonsoftware.com>
|
|
When a test times out, it should still print an ERROR message in the log
for parsing. Modify the timeout test suite to ensure this is done.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Anibal Limon <anibal@limonsoftware.com>
|
|
Add a SECURITY.md file with hints for security researchers and other
parties who might report potential security vulnerabilities.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
After reporting test results, flush the output buffers to ensure the
files are written out. Also flush again at the end of running all tests
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Instead of using the _child_reader singleton to track the child process,
use variables on the stack. Also, limit the variable scope as much as
possible and used named constants for the pipe indices.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Even when a test times out, an "ERROR:" message should be printed so
that the OE selftest parser knows the test has failed.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
To match the behavior of the previous ptest-runner, only collect system
state when a test has timed out
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit e50f2175d9c6b8aeb8b0bf687e5cca64a0f6e61a.
The timeout is actually the amount of time to wait until there is no
output from the test, not the total test time.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Changes the way that tests time out to be the total elapsed time for the
test, not just the time between receiving output from the test. This
matches the implementation before 8259375 ("runner: Remove threads and
mutexes").
Also update the timeout test case to test for this correctly.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Tests can exit due to a signal, which should also be reported in the
test output.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The write end of the pipe has to be closed by ptest-runner to make sure
that it will get EOF when the child process is done with it. This means
that a new pipe needs to be opened for each child so that the write ends
can be passed to it.
Fixes the problem where tests would be reported with no output when
ptest-runner was told to run multiple tests because they were passed
invalid stdin/stderr pipes.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 07d8a676aa962ecc5ec264ec33b0074adf2a8733.
This commit incorrectly assumed that ptest ran tests in parallel, which
is not true. Doing so interleaves the test output in the log file, which
the OE selftest parser cannot handle and thus breaks the test cases.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
8259375 ("runner: Remove threads and mutexes") did not correctly account
for the case where multiple tests are being run in parallel by
ptest-runner. Fix the code to track all of the child processes (but
still have them share the same stdout/stderr pipe) and wait for all of
them to finish before exiting.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Re-works the way that ptest-runner waits for processes to exit to make
it simpler and eliminate the need for a thread. In the new system, the
runner will not wait() for the process to exit until both the stdout and
stderr pipes have gotten an EOF. This works because when a process
exits, the pipes will be closed. This also ensures that the runner reads
all available output from the child process before moving on. After
reading all the data, then ptest runner will wait() on the process,
which should never block (unless a process does something strange like
close its stdout and stderr without exiting, which is handled with an
extra SIGKILL to prevent deadlock). Test timeouts are handled by sending
the child process SIGKILL if no output is detected for the timeout, but
the loop still waits for the file descriptors to reach EOF before
reaping the child.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
[YOCTO #15154]
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently the code can race as there is a read/write thread handling the stdio but
there is no guarantee that when the process exits, the thread has handled all the
data. This results in output where "END:" isn't actually at the end of the logs
but somewhere in the middle of the output.
Synchronisation is hard. The easiest way I can see to fix this is to have a mutex
for the output and then in the main thread, after the child exits, read any remaining
data. This avoids concurrent writes corrupting the output and ensures END: is
actually at the end of the test data.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We currently wait for the child to exit but we don't flush the buffers.
This can mean the output ends up out of sync and the END: line isn't at
the end of the logs.
We've recently seen a lot of issues related to this on the autobuilder.
Add in a flush call for all fds to ensure buffers are in sync. This
does appear to improve warnings on the autobuilder now we started detecting
the issue.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Anibal is no longer at Linaro and the yocto list address needs updating,
fix these.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Currently, ptest-runner simply kills the offending test without further ado,
which is not at all helpful when trying to figure out why it happens
(especially if such hangs are intermittent and rare). There's now a script
that gets executed before killing the test, so ideas on what to have in it
are welcome.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
if poll()'s timeout expires that means the test did not
produce any output, which is exactly what we need to catch.
So there's no need to set up separate timeouts with signals
and alarms, and this greatly simplifies more sophisticated
processing of hanging tests (such as collecting overall system data).
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
I also took the opportunity to correct a weird API that
returns a result (or not), depending on some internal condition.
Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Up till now ptest-runner2 returns number of failed tests with its
exit status code. Such use case is not recommended [1] and may cause
issues when there are more than 256 tests to be executed.
To alleviate this issue the number of total tests with number of failed
ones is printed before exit. To be more specific - failure of tests (one
or more) causes ptest-runner to provide exit code of 1.
One can test this change with executing:
./ptest-runner -d tests/data fail
Links:
[1] - https://www.gnu.org/software/libc/manual/html_node/Exit-Status.html
Signed-off-by: Lukasz Majewski <lukma@denx.de>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Add missing free() to eliminate valgrind warning when running:
$ make && valgrind -s --leak-check=full ./ptest-runner -d tests/data2
==174152== HEAP SUMMARY:
==174152== in use at exit: 145 bytes in 2 blocks
==174152== total heap usage: 42 allocs, 40 frees, 43,549 bytes allocated
==174152==
==174152== 145 bytes in 2 blocks are definitely lost in loss record 1 of 1
==174152== at 0x4842839: malloc (in /usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==174152== by 0x493414E: strdup (strdup.c:42)
==174152== by 0x10BFAA: run_ptests (utils.c:476)
==174152== by 0x10B025: main (main.c:221)
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
The function cleanup_ptest_opts() is used at file scope so make it static to
avoid a warning about it not having a prototype.
Add padding variables to ensure that struct elements are word aligned
on 64 bit machines to avoid clang warnings such as:
padding struct 'struct ptest_options' with 4 bytes to align 'exclude' [-Wpadded]
Make the 'timeout' variable types be 'unsigned int' since they are used with
alarm() and will otherwise generate a warning.
Explicitly avoid warnings such as:
warning: unused parameter 'arg' [-Wunused-parameter]
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
You can now just run:
$ CC=clang make && CC=clang make check
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Removes the following warnings thrown by
make && valgrind -s --leak-check=full ./ptest-runner -d tests/data2
==4154390== Invalid write of size 8
==4154390== at 0x40360D: run_child (utils.c:357)
==4154390== by 0x403C5B: run_ptests (utils.c:534)
==4154390== by 0x402C4D: main (main.c:223)
==4154390== Address 0x4a66440 is 0 bytes inside a block of size 2 alloc'd
==4154390== at 0x4839809: malloc (vg_replace_malloc.c:307)
==4154390== by 0x4035E4: run_child (utils.c:354)
==4154390== by 0x403C5B: run_ptests (utils.c:534)
==4154390== by 0x402C4D: main (main.c:223)
==4154390==
==4154390== Invalid write of size 8
==4154390== at 0x403618: run_child (utils.c:358)
==4154390== by 0x403C5B: run_ptests (utils.c:534)
==4154390== by 0x402C4D: main (main.c:223)
==4154390== Address 0x4a66448 is 6 bytes after a block of size 2 alloc'd
==4154390== at 0x4839809: malloc (vg_replace_malloc.c:307)
==4154390== by 0x4035E4: run_child (utils.c:354)
==4154390== by 0x403C5B: run_ptests (utils.c:534)
==4154390== by 0x402C4D: main (main.c:223)
==4154390==
==4154390== Syscall param execve(argv) points to unaddressable byte(s)
==4154390== at 0x4955C2B: execve (in /usr/lib64/libc-2.32.so)
==4154390== by 0x40365E: run_child (utils.c:368)
==4154390== by 0x403C5B: run_ptests (utils.c:534)
==4154390== by 0x402C4D: main (main.c:223)
==4154390== Address 0x4a66442 is 0 bytes after a block of size 2 alloc'd
==4154390== at 0x4839809: malloc (vg_replace_malloc.c:307)
==4154390== by 0x4035E4: run_child (utils.c:354)
==4154390== by 0x403C5B: run_ptests (utils.c:534)
==4154390== by 0x402C4D: main (main.c:223)
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
make && valgrind -s --leak-check=full ./ptest-runner -d tests/data2
==4154029== HEAP SUMMARY:
==4154029== in use at exit: 20 bytes in 2 blocks
==4154029== total heap usage: 45 allocs, 43 frees, 42,909 bytes allocated
==4154029==
==4154029== 20 (8 direct, 12 indirect) bytes in 1 blocks are definitely lost in loss record 2 of 2
==4154029== at 0x4839809: malloc (vg_replace_malloc.c:307)
==4154029== by 0x40252D: str2array (main.c:70)
==4154029== by 0x402768: main (main.c:119)
==4154029==
==4154029== LEAK SUMMARY:
==4154029== definitely lost: 8 bytes in 1 blocks
==4154029== indirectly lost: 12 bytes in 1 blocks
==4154029== possibly lost: 0 bytes in 0 blocks
==4154029== still reachable: 0 bytes in 0 blocks
==4154029== suppressed: 0 bytes in 0 blocks
==4154029==
==4154029== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
With this patch valgrind reports 0 errors.
From Anibal,
In standard executions this memleak not causes problems because after
main() function the program will end.
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Signed-off-by: Adrian Freihofer <adrian.freihofer@siemens.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Since we are using threads to read from child, no complex logic is
needed for handle timeouts by std{out,err} in the child using alarm(2).
[YOCTO #14220]
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
In order to handle large output add a thread for read from childs using
a pipe and remove non-blocking option.
Modify bash unittest to output large data and cover this scenario.
[YOCTO #14220]
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Modify testcase to validate the actual exit status.
[YOCTO #14217]
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Fixes,
$ ./ptest-runner -d ./tests/data bash
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Fixes error
ERROR: Unable to detach from controlling tty, Inappropriate ioctl for device
when running multiple ptests
ptest-runner a b
or when invoked over ssh single command, like
$ ssh localhost ptest-runner
For ssh case, fd 0 is not a tty. (isatty(0) is false).
When running multiple ptests, deattach for parent needs to be
done only once.
Signed-off-by: Tero Kinnunen <tero.kinnunen@vaisala.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Enables multiple folders in -d option the current use case is to be able
specify multilib folders in the same run.
The -d option allows directories without whitespaces inside, example to
run after compile:
$ ./ptest-runner -d "test/data tests/data2/"
[YOCTO #12604]
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Fixes,
...
alimon@blackbox:~/upstream/ptest-runner2$ make CC=gcc check
gcc -std=gnu99 -pedantic -Wall -Werror -I . -g -c tests/main.c -o tests/main.o
gcc -std=gnu99 -pedantic -Wall -Werror -I . -g -c tests/ptest_list.c -o tests/ptest_list.o
gcc -std=gnu99 -pedantic -Wall -Werror -I . -g -c tests/utils.c -o tests/utils.o
gcc -lm -lrt -lpthread tests/main.o tests/ptest_list.o tests/utils.o utils.o ptest_list.o -o ptest-runner-test -lcheck -lsubunit -lutil
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check.o): in function `tcase_create':
(.text+0x1b3): undefined reference to `floor'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check.o): in function `tcase_set_timeout':
(.text+0x66b): undefined reference to `floor'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check.o): in function `check_get_clockid':
(.text+0xd03): undefined reference to `timer_create'
/usr/bin/ld: (.text+0xd25): undefined reference to `timer_delete'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check_pack.o): in function `ppack':
(.text+0x4fc): undefined reference to `__pthread_register_cancel'
/usr/bin/ld: (.text+0x546): undefined reference to `__pthread_unregister_cancel'
/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check_run.o): in function `srunner_run_tagged':
(.text+0xbf7): undefined reference to `timer_create'
/usr/bin/ld: (.text+0xc3c): undefined reference to `timer_settime'
/usr/bin/ld: (.text+0xc6c): undefined reference to `timer_delete'
collect2: error: ld returned 1 exit status
make: *** [Makefile:39: ptest-runner-test] Error 1
...
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Drop unused function parameters in wait_child().
The remaining warning in the top dir code is:
utils.c:25:9: warning: macro name is a reserved identifier [-Wreserved-id-macro]
#define _GNU_SOURCE
and it's not clear how to deal with that in a portable manner.
Drop unused variable in analizer code.
Rename analizer to analyzer.
Avoid program scope global by adding a set function for
the opts directory variable.
Free strdup()ed memory before exit.
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
|
|
Make tests build using: clang -Weverything
There are a few warnings that remain that are not
variable casting or macro fixes.
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
|
|
Fix basic errors found when building with the clang compiler
with the option -Weverything. There are a few warnings that remain
that are not variable casting, macro fixes, or similarily simple
changes.
Makefile: add -lutil for 'check' builds for clang/gcc
builds.
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
|
|
When running the run-execscript bash ptest as a user rather than root, a warning:
bash: cannot set terminal process group (16036): Inappropriate ioctl for device
bash: no job control in this shell
contaminates the bash log files causing the test to fail. This happens only
when run under ptest-runner and not when interactively testing!
The changes made to fix this include:
1. Get the process group id (pgid) before forking,
2. Set the pgid in both the parent and child to avoid a race,
3. Find, open and set permission on the child tty, and
4. Allow the child to attach to controlling tty.
Also add '-lutil' to Makefile. This lib is from libc and provides openpty.
Upstream-Status: Submitted [yocto@yoctoproject.org]
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
|
|
There was a race in the code where the pipes may not be read after the process has exited
and data may be left behind in them. This change to ordering ensures the pipes are read
after the exit code has been read meaning no data can be left behind and the logs should
be complete.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Rather than just killing the process we've swawned, set the process group
for spawned children and then kill the group of processes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
There is no guarantee that the data written with fwrite will be flushed to the
buffer. If stdout and stderr are the same thing, this could lead to interleaved
writes. The common case is stdout output so flush the output pipes when writing to
stderr. Also flush stdout before the function returns.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
vredir ptest fails since too many file descriptors were open.
From the failed ptest log:
run-vredir
87,88c87,88
< ./vredir6.sub: line 10: /dev/null: Too many open files
< ./vredir6.sub: line 13: /dev/null: Too many open files
FAIL: run-vredir
Added function to close file descriptors before starting a new process.
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Randy Macleod <randy.macleod@windriver.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
If a process gets killed by timeout or OOM the output could end
without '\n' so add 'DURATION: N\n' after '\nERROR: ptest-dir\n'.
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
To fix buffer ordering problems.
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|
|
In yocto project ptest data are installed to
PTEST_PATH ?= "${libdir}/${BPN}/ptest". Here make DEFAULT_DIRECTORY
be able to be defined outside of c code (e.g. passed in by CFLAGS),
in case libdir isn't "/usr/lib", which will cause "No ptests found"
error when running "ptest-runner" command. We will pass libdir to
DEFAULT_DIRECTORY when compiling to avoid that error.
Signed-off-by: Li Zhou <li.zhou@windriver.com>
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
|