aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)Author
2021-03-24tests: Update to cover ptest timeout by filedevAníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2021-03-24Add support to specify timeout by ptestAníbal Limón
The ptest-runner has a default timeout of 300 secs and can be override usint -t option in the runner. There is a need to specify timeout by ptest because not all ptests takes the sametime also are affected by machine. So add support to read a file inside ptest folder (timeout-ptest) and override global timeout with it. For example in glib-2.0, /usr/lib/glib-2.0/ptest/run-ptest /usr/lib/glib-2.0/ptest/timeout-ptest Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2021-03-24utils.c: wait_child reimplement timeout using alarmv2.4.1Aníbal Limón
Since we are using threads to read from child, no complex logic is needed for handle timeouts by std{out,err} in the child using alarm(2). [YOCTO #14220] Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2021-03-23utils.c: Use a thread to read from childAníbal Limón
In order to handle large output add a thread for read from childs using a pipe and remove non-blocking option. Modify bash unittest to output large data and cover this scenario. [YOCTO #14220] Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2021-03-23utils.c: Fix exit status of a childAníbal Limón
Modify testcase to validate the actual exit status. [YOCTO #14217] Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2021-03-23utils.c: get_available_ptests allow to specify relative directoriesAníbal Limón
Fixes, $ ./ptest-runner -d ./tests/data bash Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2021-03-23README.md: Small fix mtrace callAníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2021-02-11tests/utils.c: Add braces in START_TEST/END_TEST now required in check 0.15.xAníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2020-07-13Fix inappropriate ioctl when detaching ttyTero Kinnunen
Fixes error ERROR: Unable to detach from controlling tty, Inappropriate ioctl for device when running multiple ptests ptest-runner a b or when invoked over ssh single command, like $ ssh localhost ptest-runner For ssh case, fd 0 is not a tty. (isatty(0) is false). When running multiple ptests, deattach for parent needs to be done only once. Signed-off-by: Tero Kinnunen <tero.kinnunen@vaisala.com> Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2020-04-10Add support to specify multiple folder for ptest-runnerv2.4Aníbal Limón
Enables multiple folders in -d option the current use case is to be able specify multilib folders in the same run. The -d option allows directories without whitespaces inside, example to run after compile: $ ./ptest-runner -d "test/data tests/data2/" [YOCTO #12604] Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2020-04-10Makefile: Fix TEST_LDFLAGS gcc cmdline positionAníbal Limón
Fixes, ... alimon@blackbox:~/upstream/ptest-runner2$ make CC=gcc check gcc -std=gnu99 -pedantic -Wall -Werror -I . -g -c tests/main.c -o tests/main.o gcc -std=gnu99 -pedantic -Wall -Werror -I . -g -c tests/ptest_list.c -o tests/ptest_list.o gcc -std=gnu99 -pedantic -Wall -Werror -I . -g -c tests/utils.c -o tests/utils.o gcc -lm -lrt -lpthread tests/main.o tests/ptest_list.o tests/utils.o utils.o ptest_list.o -o ptest-runner-test -lcheck -lsubunit -lutil /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check.o): in function `tcase_create': (.text+0x1b3): undefined reference to `floor' /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check.o): in function `tcase_set_timeout': (.text+0x66b): undefined reference to `floor' /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check.o): in function `check_get_clockid': (.text+0xd03): undefined reference to `timer_create' /usr/bin/ld: (.text+0xd25): undefined reference to `timer_delete' /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check_pack.o): in function `ppack': (.text+0x4fc): undefined reference to `__pthread_register_cancel' /usr/bin/ld: (.text+0x546): undefined reference to `__pthread_unregister_cancel' /usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libcheck.a(check_run.o): in function `srunner_run_tagged': (.text+0xbf7): undefined reference to `timer_create' /usr/bin/ld: (.text+0xc3c): undefined reference to `timer_settime' /usr/bin/ld: (.text+0xc6c): undefined reference to `timer_delete' collect2: error: ld returned 1 exit status make: *** [Makefile:39: ptest-runner-test] Error 1 ... Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-07-21Fix additional warnings when using clangv2.3.2Randy MacLeod
Drop unused function parameters in wait_child(). The remaining warning in the top dir code is: utils.c:25:9: warning: macro name is a reserved identifier [-Wreserved-id-macro] #define _GNU_SOURCE and it's not clear how to deal with that in a portable manner. Drop unused variable in analizer code. Rename analizer to analyzer. Avoid program scope global by adding a set function for the opts directory variable. Free strdup()ed memory before exit. Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
2019-07-21tests: fix clang warnings.Randy MacLeod
Make tests build using: clang -Weverything There are a few warnings that remain that are not variable casting or macro fixes. Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
2019-07-21main code: fix clang warningsRandy MacLeod
Fix basic errors found when building with the clang compiler with the option -Weverything. There are a few warnings that remain that are not variable casting, macro fixes, or similarily simple changes. Makefile: add -lutil for 'check' builds for clang/gcc builds. Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
2019-07-21utils: ensure child can be session leaderRandy MacLeod
When running the run-execscript bash ptest as a user rather than root, a warning: bash: cannot set terminal process group (16036): Inappropriate ioctl for device bash: no job control in this shell contaminates the bash log files causing the test to fail. This happens only when run under ptest-runner and not when interactively testing! The changes made to fix this include: 1. Get the process group id (pgid) before forking, 2. Set the pgid in both the parent and child to avoid a race, 3. Find, open and set permission on the child tty, and 4. Allow the child to attach to controlling tty. Also add '-lutil' to Makefile. This lib is from libc and provides openpty. Upstream-Status: Submitted [yocto@yoctoproject.org] Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com> Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com>
2019-06-27utils: Ensure pipes are read after exitRichard Purdie
There was a race in the code where the pipes may not be read after the process has exited and data may be left behind in them. This change to ordering ensures the pipes are read after the exit code has been read meaning no data can be left behind and the logs should be complete. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-06-27use process groups when spawningRichard Purdie
Rather than just killing the process we've swawned, set the process group for spawned children and then kill the group of processes. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-06-27utils: Ensure stdout/stderr are flushedRichard Purdie
There is no guarantee that the data written with fwrite will be flushed to the buffer. If stdout and stderr are the same thing, this could lead to interleaved writes. The common case is stdout output so flush the output pipes when writing to stderr. Also flush stdout before the function returns. Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-06-03Add SPDX-License-Identifier: GPL-2.0-or-later in source filesAníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-05-31utils.c: close all file descriptors after completing a ptestSakib Sajal
vredir ptest fails since too many file descriptors were open. From the failed ptest log: run-vredir 87,88c87,88 < ./vredir6.sub: line 10: /dev/null: Too many open files < ./vredir6.sub: line 13: /dev/null: Too many open files FAIL: run-vredir Added function to close file descriptors before starting a new process. Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com> Signed-off-by: Randy Macleod <randy.macleod@windriver.com> Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-01-30utils.c: Print DURATION after ERRORv2.3.1Aníbal Limón
If a process gets killed by timeout or OOM the output could end without '\n' so add 'DURATION: N\n' after '\nERROR: ptest-dir\n'. Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-01-29utils.c: run_child redirect stderr to stdoutAníbal Limón
To fix buffer ordering problems. Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-01-21ptest-runner: make DEFAULT_DIRECTORY be able to be defined when compilingLi Zhou
In yocto project ptest data are installed to PTEST_PATH ?= "${libdir}/${BPN}/ptest". Here make DEFAULT_DIRECTORY be able to be defined outside of c code (e.g. passed in by CFLAGS), in case libdir isn't "/usr/lib", which will cause "No ptests found" error when running "ptest-runner" command. We will pass libdir to DEFAULT_DIRECTORY when compiling to avoid that error. Signed-off-by: Li Zhou <li.zhou@windriver.com> Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-01-07ptest-runner: Add support to report duration of each ptestv2.3Aníbal Limón
In stdout reported as, ... BEGIN: ptest-dir ... DURATION: N END: ptest-dir ... In XML reported as, ... <testcase classname='test1' name='run-ptest'> <duration>N</duration> </testcase> ... Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2019-01-03ptest-runner: Add support timeout failure type to XML result fileAníbal Limón
The ptest-runner support logging results to stdout and to a XML file in stdout the ptest is mark as: ... ERROR: Exit status is 1 TIMEOUT: ptest-directory ... Add the same support in XML file for example, ... <testcase classname='ptest-directory' name='run-ptest'> <failure type='exit_code' message='run-ptest exited with code: 1'></failure> <failure type='timeout'/> </testcase> ... [YOCTO #13088] Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2018-04-25main.c: Add option (-e) to exclude certain tests for executionv2.2Aníbal Limón
You can specify a set of ptests to be excluded, it will not fail if some ptest excluded isn't found in the list of execution. $ ./ptest-runner -e "hang glibc" -d tests/data Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2018-04-25main.c: Use realpath to get the actual directory of ptestsAníbal Limón
Fix usage of relative paths in -d argument. $ ./ptest-runner -d ./tests/data Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2017-12-12README.md: Update to my current emailv2.1.1Aníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2017-12-12Add support to avoid load/run twice a run_ptest scriptAníbal Limón
In some ptest packages exists symlink in the ptest directory causing to load/run twice the same ptest, For example in perl5: /usr/lib/perl -> /usr/lib/perl5 Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2017-12-12Makefile: libcheck now requires to link subunitAníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linaro.org>
2017-11-02utils.c: Prefer monotonic clock to calculate elapsed timeJeffrey Pautler
The current implementation uses the system clock to calculate how long a ptest has been running with no output. If a ptest changes the system clock as part of the test, that can cause the current implementation to falsely trigger a timeout or miss an actual timeout. It is preferrable to use a monotonic clock for calculating elapsed time in order to avoid these issues. This change tries to use the monotonic clock first and falls back to the realtime clock if the monotonic clock is not supported. Signed-off-by: Jeffrey Pautler <jeffrey.pautler@ni.com>
2017-05-29timeout option missing the argument option ":" to getoptAnders Wallin
ptest-runner -t xx gives segfault Signed-off-by: Anders Wallin <wallinux@gmail.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-04-06Add xml outputv2.1Pascal Bach
The format follow the one defined in the Yocto wiki: https://wiki.yoctoproject.org/wiki/QA/xUnit_XML_Template It is only the minimum required but it can be extended in the future. Signed-off-by: Schuler Christian <schuler.christian@siemens.com> Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-04-06utils, main: Add ptest_options struct to run_ptestsPascal Bach
This allows to easily add additional parameters. Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2017-04-06utils.c: Improve user notice of TIMEOUTv2.0.2Aníbal Limón
Print exit status before TIMEOUT to give clear information to the user about exit status before TIMEOUT and END. [YOCTO #10842] Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-09-22tests/utils.c: run_timeout_ptest remove unnecessary codev2.0.1Aníbal Limón
Now the run_timeout_ptest runs only one test so is not needed to detect when this ptest starts as before. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-09-22tests/utils.c: Add test case for run_ptest failureEdwin Plauchu
This test case covers the case when a run-ptest script fails and prints ERROR in the output. [YOCTO #9752] Signed-off-by: Edwin Plauchu <edwin.plauchu.camacho@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-09-22tests/utils.c: Refactor test_run_ptestsEdwin Plauchu
Now test_run_ptests only runs succesful ptests, this commit creates test_timeout_ptest for test TIMEOUT run-ptest script. Also creates new functions for run specific ptest and search for specific output in this case TIMEOUT. Signed-off-by: Edwin Plauchu <edwin.plauchu.camacho@linux.intel.com Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-09-22utils.c: Add error report when run-ptests failsEdwin Plauchu
If some run-ptest script fails the user needs to be noticed so add print "ERROR" when fork fail or run-ptest exits different than zero. [YOCTO #9752] Signed-off-by: Edwin Plauchu <edwin.plauchu.camacho@linux.intel.com> Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-09-20README.md: Add Contributions section.Aníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-03-29README.md: Update to point to git.yoctoproject.org repository.Aníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-01-14wait_child, run_ptests: Display ptest directory instead of ptest scriptv2.0v0.1Aníbal Limón
Comply with old ptest-runner behaviour displaying ptest directory in, BEGIN, END and TIMEOUT lines. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-01-14utils.c: run_child adds missing chdirAníbal Limón
ptest-runner shell script currently change to the ptest directory before call run-ptest. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-01-14Create README.mdAnibal Limón
2016-01-14Add support for enable mcheck/mtrace.Aníbal Limón
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-01-14Makefile: Add RELEASE env variable.Aníbal Limón
When RELEASE env var is enabled the ptest-runner is compiled with optimizations otherwise is compiled with debug information. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-01-14tests/utils.c: Add full testcase for run_ptests.Aníbal Limón
- Case all tests. - Case of hang also adds data for simulate hang. - Case for read data in stderr. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-01-13run_ptests: Fixes and improvements.Aníbal Limón
- run_child: Fix pipe setup for capture output. - wait_child: Change usage of select for poll and fix TIMEOUT write now to stdout. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-01-12Adds full support for run_ptests.Aníbal Limón
run_ptests: Takes every ptests and executes it using fork/excve if timeout is enabled it look at stdout, stderr ptest output if no have output in more than N secods kills it. tests/data: Add simple run-ptest scripts for with echo name of the ptests. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
2016-01-12main.c: Add full logic for get ptests and run it only missingAníbal Limón
run_ptest implementation. Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
generated by cgit 1.2.3-korg (git 2.39.0) at 2024-04-28 05:15:52 +0000