Age | Commit message (Collapse) | Author |
|
In rare cases we see failures, often in linux-libc-headers for things like:
| INSTALL /XXX/linux-libc-headers/6.1-r0/image/usr/include
| abort()ing pseudo client by server request. See https://wiki.yoctoproject.org/wiki/Pseudo_Abort for more details on this.
Pseudo log:
path mismatch [2 links]: ino 46662476 db 'NAMELESS FILE' req '/XXX/linux-libc-headers/6.1-r0/image/usr'.
Setup complete, sending SIGUSR1 to pid 3630890.
Whilst this doesn't easily reproduce, the issue is that multiple different processes are
likely working on the directory and the creation in pseudo might not match accesses
made by other processes.
Ultimately, the "NAMELESS FILE" is harmless and pseudo will reconcile things
so rather than error out, we should ignore this case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
readlink() failures are normal. pseudo should just set errno and let the
program decide whether to report it. E.g. /proc/$pid/{cwd,exe,fd/*,root} are
unreadable for other users' processes. The "pidof" program calls stat() on
each /proc/$pid/exe, which causes pseudo to spam stderr. "pidof" is used e.g.
by various dpkg postinst scripts.
This if branch also forgot to update *pcurrent, so "cat /proc/1/cwd/stat"
would read /proc/1/stat instead of failing as it should.
Signed-off-by: Tomi Belan <tomi.belan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Apparently <attr/attributes.h> was only included in order to get ENOATTR.
pseudo does not use any other functions or structs from it.
More about ENOATTR:
https://git.savannah.nongnu.org/cgit/attr.git/tree/include/attributes.h
https://github.com/rust-lang/libc/issues/1356
https://github.com/golang/go/issues/753
Signed-off-by: Tomi Belan <tomi.belan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
- "make test" did not build "test/test-chroot-symlink", so the test failed
- "make clean" did not delete libpseudo.so correctly
- "test/test-umask.sh" did not clean up its test files
- build test binaries only if needed, and delete them in "make clean"
- remove var/pseudo from Makefile (run_tests.sh makes it anyway)
- add *.pyc and test binaries (no file extension) to .gitignore
- delete more things in "make distclean"
Signed-off-by: Tomi Belan <tomi.belan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
pseudo_fix_path() incorrectly assumed that 'base' never ends with a slash
because it's a canonical path. However, base can be "/" - a path which is
canonical and yet ends with a slash. This happens when pseudo_cwd is "/" or
when we're starting from a dirfd pointing to "/". The wrong result from
pseudo_fix_path() caused the database lookup to fail and made pseudo abort.
Signed-off-by: Tomi Belan <tomi.belan@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Absolute links should be evaluated starting from the chroot path but the
existing implementation drops the whole base path (which includes the
chroot path) when it encounters an absolute link.
Encountered the issue during root image creation process in an OE build
where ldconfig was deleting some absolute links as stat64 calls failed
and the symlinks were deemed dead.
Signed-off-by: Chaitanya Vadrevu <chaitanya.vadrevu@ni.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This reverts commit 300d7570720bc14d01f49d285498d71cce3ce44e.
This is causing faults for real world builds, e.g.:
Exception: subprocess.CalledProcessError: Command 'tar --exclude='.git' --exclude='__pycache__' --xattrs --xattrs-include='*' -chf - -C /home/pokybuild/yocto-worker/beaglebone/build/meta -p . | tar --xattrs --xattrs-include='*' -xf - -C /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-minimal/1.0-r0/sdk-ext/image//usr/local/oe-sdk-hardcoded-buildpath/layers/build/meta' returned non-zero exit status 134.
Subprocess output:
abort()ing pseudo client by server request. See https://wiki.yoctoproject.org/wiki/Pseudo_Abort for more details on this.
Check logfile: /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-minimal/1.0-r0/pseudo//pseudo.log
/bin/sh: line 1: 1732017 Broken pipe tar --exclude='.git' --exclude='__pycache__' --xattrs --xattrs-include='*' -chf - -C /home/pokybuild/yocto-worker/beaglebone/build/meta -p .
1732018 Aborted (core dumped) | tar --xattrs --xattrs-include='*' -xf - -C /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-minimal/1.0-r0/sdk-ext/image//usr/local/oe-sdk-hardcoded-buildpath/layers/build/meta
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-minimal/1.0-r0/temp/log.do_populate_sdk_ext.1725807
NOTE: recipe core-image-minimal-1.0-r0: task do_populate_sdk_ext: Failed
ERROR: Task (/home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/core-image-minimal.bb:do_populate_sdk_ext) failed with exit code '1'
Pseudo log:
path mismatch [1 link]: ino 14050447 db '/home/pokybuild/yocto-worker/beaglebone/build/build/tmp/work/beaglebone_yocto-poky-linux-gnueabi/core-image-minimal/1.0-r0/sdk-ext/image/usr/local/oe-sdk-hardcoded-buildpath/layers/build/meta/COPYING.MIT' req '/proc/self/fd/3/./COPYING.MIT'.
|
|
Crazy shell code (e.g. libtool) can pass in a command pipeline as a
path which exceeds the max path length the system can support (6000+
chars). This will fail in libc or the syscall but if we don't do
something here, we'd segfault before it can do that. Leave path
unchanged and let libc deal with it.
This was observed with segfaults in libfm:do_install after the libtool
upgrade. It does depend on the length of the local build path too.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some symlinks in /proc, such as those under /proc/[pid]/fd,
/proc/[pid]/cwd, and /proc/[pid]/exe that are not real and should not
have readlink called on them. These look like symlinks, but behave like
hardlinks. Readlink does not return actual paths. Previously
pseudo_fix_path would expand files such as /dev/stdin to paths such as
/proc/6680/fd/pipe:[1270830076] which do not exist.
This issue affects:
- deleted files
- deleted directories
- fifos
- sockets
- anon_inodes (epoll, eventfd, inotify, signalfd, timerfd, etc)
Testing:
- run_tests: all tests passed (3 tests check the new code path).
Checked test output to make sure the new codepath gets executed.
- perftest: measured time before and after applying the patch
had insignificant differences (roughly ~1%)
- world build: completed without warning/errors
Signed-off-by: Sakib Sajal <sakib.sajal@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add a wrapper for the fcntl64 glibc function based on the fcntl
wrapper which is effectively the same.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We have some challenges in ensuring the pseudo database is flushed to disk
in things like docker containers since the processes can be killed with no
warning at container termination. Rightly or wrongly, we need to handle this
better.
There is no current way to flush the DB but there is a shutdoen request
mechanism. Whilst we can't force a shutdown from bitbake due to the multiple
connected clients and the existing 3s shutdown isn't enough, we can take the
hint to flush the DB at this time. Further writes are unlikely.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
[Add tests accidentally missed in previous commit]
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
glibc 2.34 adds a closefrom() function call to close a range of file
descriptors. This one is problematic for us since pseudo can have its
own fds open in the close range.
To handle this we add a specific client side op, OP_CLOSEFROM, similar
to OP_CLOSE which closes the fds in the range which aren't pseudo fds.
This means manually closing some of the fds ourselves and then modifying
the call to closefrom for the rest. Not pretty but I'm struggling to see
a better way. It does mean msg/result is used in a new case to let the
caller know which fds to close as the range needs to change. This is
allowed after the previous static change.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The address of msg is returned by the function in the OP_CHROOT case.
Whilst the way it is used means it is just a way of saying true/false,
the compiler doesn't know this and can warn and this isn't really
allowed.
The other pseudo funcitons in this area already use a static msg so
make this function match the others. There wouldn't be re-enterancy
in this context.
Need to be careful about clearing the data upon each function entry
and not just once at init.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
glibc 2.34 adds a close_range() function call. This one is straight forward
as it allows ENOSYS to be returned and the caller has to handle it so
lets do that.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fixes the following warning:
pseudo_client.c: In function ‘pseudo_root_path’:
pseudo_client.c:848:17: warning: ‘%s’ directive argument is null [-Wformat-overflow=]
848 | pseudo_diag("couldn't allocate absolute path for '%s'.\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
849 | path);
| ~~~~~
Signed-off-by: Damian Wrobel <dwrobel@ertelnet.rybnik.pl>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
It just so happens that my /home/mac and /home directories have the same
inode number but on different filesystems.
This means that test-openat fails with "Recursion failed!" even when run
without pseudo.
Let's consider both the device number and the inode number before
assuming that we've found the same directory again.
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Adding this test case was erroneously omitted in
7c722296879906fe093e1e7c4b7537e150d492cd.
Signed-off-by: Philip Lorenz <philip@bithub.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
When running the test suite on my Debian 11 box I see many occurrences
of:
unknown fcntl argument 1032, assuming long argument.
(for example from test-execl.sh.)
It appears that this is F_GETPIPE_SZ and it takes no arguments. Let's
add it and the corresponding F_SETPIPE_SZ too to avoid the warning
messages.
F_SETPIPE_SZ accepts an int argument, which strictly speaking isn't the
same as the long that the wrapper expects. However, this is also true
for F_DUPFD which seems to be working correctly on all the targets that
people care about.
We need to define the command constants if the system headers don't
provide them to ensure that a binary built on an old system works
without the new commands works correctly only a newer one that tries to
use them. If the system values differ from the expected ones then such a
binary would also be incompatible, so fail the build in that case too.
Signed-off-by: Mike Crowe <mac@mcrowe.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This change seems to cause faults in OE builds, not investigated why
but revert until we can debug. "bitbake linux-libc-headers" reproduces
all the time.
This reverts commit 04bca0f85d57da1d0ed0419780df296f8b0ff81d.
|
|
We want a pseudo which can build on a system without statx support but work on
one with statx support. By adding a copy of the struct we can do this and then
work correclty with OE's uninative again.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Commit 60e25a36558f1f07dcce1a044fe976b475bec42b started dereferencing
the "path" parameter which for some functions is annotated with the
"nonnull" attribute. While the commit explicitly checks for NULL
pointers before dereferencing it, GCC (at optimization level 1 and
above) removes the check due to the "nonnull" attribute being set for
some parameters in the glibc headers (e.g. statx()).
However, the statx() man page explicitly allows calling with NULL
pointers (in which case the EFAULT is returned) and this behaviour is
used in the wild (e.g. in Rust) to determine whether the statx() system
call is supported.
Disabling the optimization is not possible ([1]) so prevent the compiler
optimization by referencing the parameter in a noop inline assembly
instruction instead.
[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100404
Signed-off-by: Philip Lorenz <philip@bithub.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Fix some warnings reported by GCC 10.2.0:
* NULL pointer passed to '%s' format string parameter:
pseudo_client.c: In function ‘pseudo_root_path’:
pseudo_client.c:848:3: warning: ‘%s’ directive argument is null
[-Wformat-overflow=]
848 | pseudo_diag("couldn't allocate absolute path for '%s'.\n",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
849 | path);
* Return of variable with local storage duration:
pseudo_client.c: In function ‘pseudo_client_op’:
cc1: warning: function may return address of local variable
[-Wreturn-local-addr]
pseudo_client.c:1592:15: note: declared here
1592 | pseudo_msg_t msg = { .type = PSEUDO_MSG_OP };
Signed-off-by: Philip Lorenz <philip@bithub.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
GCC emits this warning for mismatched function types unless the generic
void (*) (void) signature is used ([1]) - e.g.:
warning: cast between incompatible function types from ‘int (*)(const
char *)’ to ‘int (*)(void)’ [-Wcast-function-type]
[1] https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html#index-Wcast-function-type
Signed-off-by: Philip Lorenz <philip@bithub.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The pseudo client path map stores paths that have been sanitised, but
in the ignored-path (PSEUDO_IGNORE_PATHS) codepath for open() calls this
sanitising wasn't performed so it is possible for paths that end with a
trailing slash to be entered.
This then subsequently interacts badly with path manipulation, resulting
in the situation where doing:
fd = open("/some/path/")
parent_fd = openat(fd, "../)
results in parent_fd actually pointing at /some/path still.
Solve this by ensuring that any trailing slashes are removed from the
path when adding to the map in the ignore short-circuit.
Also add a test case for this to ensure that it doesn't regress in the
future.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In glibc 2.33 it makes calls like:
fstatat64 (pathfd, "", &st, AT_EMPTY_PATH);
where pathfd may be a symlink. This interacts badly with pseudo_root_path()
since the empty path is replaced with a pathname from the open fd but
AT_SYMLINK_NOFOLLOW is not set, hence the link is resolved and pseudo
throws an abort() due to inode mismatch.
Where the path is empty, an fd is passed and AT_EMPTY_PATH is set,
we can imply that AT_SYMLINK_NOFOLLOW is also effectly set. Adjust
the wrapper functions to ensure this, allowing the functions to behave
correctly in the AT_EMPTY_PATH case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're seeing failures where a file is renamed under pseudo but an
access appears to be made to the old filename before the OP_RENAME has
hit the database but after the real_rename has applied in the kernel.
This is effectively the MAY_UNLINK problem for the original filename. There
were protections for the newpath but not the oldpath.
To try and avoid these aborts(), mark the original path as MAY_UNLINK however
we need to be careful not to trigger the DID_UNLINK but instead update
the deleting entry in the database as the rename completes. To do this,
we no clear the deleting flag during the database rename operation in SQL.
Also, we have to stop removing the by_ino matches where by_ino.deleting is
set, else we may lose the renamed file's attributes that way.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Add wrappers for the faccessat and faccessat2 glibc functions matching
the way access.c works. faccessat2 was added in glibc 2.33.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The man page for access() does not allow for a return value of EPERM,
should be EACCES. Fix.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In glibc 2.33, they've removed the _STAT_VER and _MKNOD_VER definitions
from public headers. They have no plans to add these back so pseudo needs
to attempt its own definitions. There is some protection as if they were
wrong and there was a mismatch, we'd get an error art runtime.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is an issue in pseudo where if you open a file, rename the file,
then call fstat on the open fd, pseudo would thrown an abort. This is
because it needs to track the open fd mappings to files and it doesn't
update in the case of a rename.
Add code in pseudo to update the fd mappings in the case of a rename
call. Also add a test case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Pseudo will calculate the search paths used for passwd operations such
as lckpwdf and ulckpwdf using build_passwd_paths when it initiates. This
takes into account the chroot at the time.
The problem is that after a chroot is performed, pseudo continues to use
the search paths calculated from the start for lckpwdf and ulckpwdf.
This makes it write the pwd.lock files to a different sysroot if a chroot
is called during runtime.
This commit resolves that by calling build_passwd_paths again after
intercepting chroot calls so the search paths are up to date.
This bug manifests in Yocto when shadow is installed into an SDK target
rootfs. The postinst triggered will call shadow-native with -R to point
to the SDK target rootfs which in turn makes shadow call chroot to the
SDK target rootfs before it perform its actions including lckpwdf() and
ulckpwdf(). The lock files however will write instead to the normal image
target rootfs because it was specified in PSEUDO_PASSWD and was the first
path set when the pseudo environment was initiated. By rebuilding the
search path after chroot is applied, the lockfiles appear in the correct
rootfs.
Signed-off-by: Michael Ho <Michael.Ho@bmw.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The xattr functions need to use a specific version of the symbols to avoid
calling into libattr.so, which on Tumbleweed causes failures[1].
However on arm64 systems the glibc version is different. This means that
searching for llistattr(GLIBC_2.3) fails to initially match the symbol in
libc.so, and instead if libattr.so is linked then the symbol in that library
is used. This is simply a wrapper that is intended to call the symbol in
libc.so but instead calls the symbol in pseudo, so infinite loops.
Using the just-added architecture overrides, add the right versions for
arm64 systems so the correct symbols in libc.so are found.
[ YOCTO #14133 ]
[1] b94fa2fc81cde25865ee223ca437d07377229a53
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Pseudo allows wrappers to define special comments in the wrapper lists to
pass extra arguments such as version=GLIBC_2.3 to control which symbol
version to search for. However, these arguments can be architecture-specific.
When parsing the arguments, check for flags that end in the architecture name
(as returned by platform.machine()) and use those values instead.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
An except statement was still using Python 2 syntax so caused
SyntaxErrors if the exception was raised.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
pseudo_get_value() returns newly allocated memory that the caller must free,
so add some free() calls.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Pseudo has to call deprecated functions because it is wrapping them, so
disable deprecation warnings.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Slightly alter a fallthrough comment so that GCC recognises it, and
add a default: case to a switch which explicitly only handles a few
values.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Use strcmp() insead of memcmp() when searching for keys as otherwise
the comparison will run off the end of the NULL-terminated string.
Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Newer versions of glibc have a lchmod function which we need to wrap.
Add this, and tweak fchmodat to be able to handle the "no symlink
resolution" case rather than duplicate code.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
This also plugs a memory leak by making sure env is freed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
The logfile path should be more visible so it's easier investigate the error.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Similar to mkstemp64 when oflags=0, therefore move the wrapper and call
from mkstemp64. Note that some glibc versions would have one but not the other
so ensure fall back to the real function is correct on those versions.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
There is magic in the posts where specific variable names have specific
magic. For that magic to work, "path" needs to be used not "pathname" as
is currently there. Fix this, which fixes path issues on systems using
statx (Ubuntu 20.04 in particular).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
We're seeing systems in the wild (e.g. ubuntu 20.04) which call
with a dirfd set to the full filename and path set to "". Since
this seems to be expected to work, handle it accordingly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Some operations may call unlink() on an open fd, then call fchown/fchmod/fstat
on that fd. This would currently readd its entry to the database, which
is necessary to preserve its permissions information however since that
file will be lost when it is closed, we don't want the DB entry to persist.
Marking it as may_unlink means the code will know its likely been deleted
and ignore the entry later, giving improved behaviour that simple path
mismatch warnings. We can use an nlink of zero to detect this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
In the xattr handling functions, if result is NULL, which it can be
with the path ignore code, there is a NULL pointer dereference and
segfault. Everywhere else checks result first, this appears to just
be an omission.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|
|
Consider what happens if a program does:
fd = fopen("A")
link("A", "B")
unlink("A")
fchown(fd)
Assuming we can't use the database, in order to handle this correctly,
we need to change the open fd to point at B when A us unlinked.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
|