| Age | Commit message (Collapse) | Author |
|---|
|
gcc is better about warnings and spotted variables being assigned but
not used. Clever gcc. Cleaned up the old bits.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
Add some debug messages useful for tracking down xattr behaviors.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
When setting an extended attribute using the database, we create a
dummy entry for the file (so there will be a file row corresponding
to that path name for later lookups). But this entry was coming in
with host UID/GID values in some cases. Instead, use -1 uid/gid,
and have STAT report those as failures rather than as existing
values. (Other cases should not be copying them. I think.)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
OP_OPEN and OP_EXEC are used only when logging. The server can now
tell the client (in response to initial ping) whether or not it is
logging, and if it isn't, the client doesn't send those messages.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
Instead of allocating (and then freeing) these paths all the time,
use a rotating selection of buffers of fixed but probably large enough
size (the same size that would have been the maximum anyway in
general). With the exception of fts_open, there's no likely way to
end up needing more than two or three such paths at a time. fts_open
dups the paths since it could have a large number and need them for
a while. This dramatically reduces (in principle) the amount of allocation
and especially reallocation going on.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
In the case where an "oldpath" is actually the data for an xattr
operation, don't truncate it. Trailing slashes should only be removed
from things which are actually filenames.
|
|
There was supposed to be a check for filenames showing up
with a trailing slash when the file was not a directory. What
actually made it in was a check for a mismatch between "is
a directory" and "has trailing slash", which produced spurious
messages saying the path had a trailing slash whenever a
directory path did *not* have a trailing slash. But that's
valid and should not produce diagnostics. Let alone thousands
of diagnostics.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
It turns out that "a/" is equivalent to "a/.", and that in particular
it should fail when a is not a directory. Pseudo's been silently stripping
them and this breaks things. Attempt to fix that, lightly tested.
|
|
Trying to track down problems which sometimes result in files showing
up as nameless files, producing clashes later. Looks like there were two
issues; one is we were creating links for files that we'd already
found by inode. The other is that rename was sending bogus LINK messages
in some cases. Also simplified the find_file_dev path to extract the
path as part of the initial operation, since there wasn't any case where
that wasn't being done immediately afterwards.
|
|
So it turns out that the sanity checks should be skipped on did_unlink,
because otherwise if an inode gets reused for a different file type,
it'll get nuked. This is pretty rare, but appears to bite us occasionally
during debug stripping.
|
|
It turns out that, in the fairly common case where the did-unlink stuff
has saved us from worse problems, pseudo produces probably-spurious
error messages about the path mismatch when the did-unlink shows up.
Change that into a debug message. Also fix a typo in a comment.
|
|
More complicated, because we actually need to make com.apple stuff work
probably.
|
|
Issue #1: If an operation came in for an item with no path
provided by the wrapper, the client would not construct the
combined "path" value. Fixed, and missing paths are now
consistently handled as 0-byte paths.
Issue #2: The database code was assuming the values were
strings, and ignoring a specified length.
Issue #3: The computation of the length of the stored value
was off by one, because it was including the extra terminating
null the client added in case the value was a path.
With this in place, "cp -a" on CentOS is consistently
duplicating the system.posix_acl_access fields as expected,
but unfortunately not handling their permissions too.
(Intent is to translate a system.posix_acl_access setxattr
into corresponding permissions whenever possible.)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
Initial, incomplete, support for extended attributes. Extended
attributes are implemented fairly naively, using a second table
in the file database using the primary file table's id as a
foreign key. The ON DELETE CASCADE behavior requires sqlite 3.6.19
or later with foreign key and trigger support compiled in.
To reduce round-trips, the client does not check for existing
attributes, but rather, sends three distinct set messages;
OP_SET_XATTR, OP_CREATE_XATTR, OP_REPLACE_XATTR. A SET message
always succeeds, a CREATE fails if the attribute already
exists, and a REPLACE fails if the attribute does not already
exist.
The /* flags */ feature of makewrappers is used to correct
path names appropriately, so all functions are already working
with complete paths, and can always use functions that work
on links; if they were supposed to dereference, the path
fixup code got that.
The xattr support is enabled, for now, conditional on
whether getfattr --help succeeds.
Not yet implemented: Translation for system.posix_acl_access,
which is used by "cp -a" (or "cp --preserve-all") on some
systems to try to copy modes.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
When invoked as a non-daemon, either with a command or to get a
shell, shut the server down automatically. This is currently
implemented by sending a shutdown request; I also considered
adding an environment flag for "shut down when there's no clients",
but this was simpler.
Main reason this is useful: In development, it's often the case
that I want to run a single command under pseudo, then check
the database directly with sqlite.
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
|
|
This is a moderately intrusive change. The basic overall effect:
Debugging messages are now controlled, not by a numeric "level",
but by a series of flags, which are expressed as a string of
letters. Each flag has a single-letter form used for string
specifications, a name, a description, a numeric value (1 through N),
and a flag value (which is 1 << the numeric value). (This does mean
that no flag has the value 1, so we only have 31 bits available.
Tiny violins play.)
The other significant change is that the pseudo_debug calls
are now implemented with a do/while macro containing a conditional,
so that computationally-expensive arguments are never evaluated
if the corresponding debug flags weren't set. The assumption is
that in the vast majority of cases (specifically, all of them
so far) the debug flags for a given call are a compile-time constant,
so the nested conditional will never actually show up in code
when compiled with optimization; we'll just see the appropriate
conditional test.
The VERBOSE flag is magical, in that if the VERBOSE flag is
used in a message, the debug flags have to have both VERBOSE and
at least one other flag for the call to be made.
This should dramatically improve performance for a lot of cases
without as much need for PSEUDO_NDEBUG, and improve the ability of
users to get coherent debugging output that means something and is
relevant to a given case.
It's also intended to set the stage for future development work
involving improving the clarity and legibility of pseudo's diagnostic
messages in general.
Old things which used numeric values for PSEUDO_DEBUG will sort
of continue to work, though they will almost always be less verbose
than they used to. There should probably be a pass through adding
"| PDBGF_CONSISTENCY" to a lot of the messages that are specific
to some other type.
|
|
Most pseudo operations don't actually USE the server's response. So
why wait for a response?
This patch introduces a new message type, PSEUDO_MSG_FASTOP. It
also tags pseudo operation types with whether or not they need to
give a response. This requires updates to maketables to allow non-string
types for additional columns, and the addition of some quotes to the
SQL query enums/query_type.in table.
A few routines are altered to change their behavior and whether or not
they perform a stat operation. The only operations that do wait are
OP_FSTAT and OP_STAT, OP_MKNOD, and OP_MAY_UNLINK. Rationale:
You can't query the server for replacement information and not wait for
it. Makes no sense.
There's extra checking in mknod, because we really do want to fail out
if we couldn't do that -- that implies that we haven't created a thing
that will look like a node.
The result from OP_MAY_UNLINK is checked because it's used to determine
whether we need to send a DID_UNLINK or CANCEL_UNLINK. It might be cheaper
to send two messages without waiting than to send one, wait, and maybe
send another, but I don't want to send invalid messages.
This is highly experimental.
|
|
Change from internal PSEUDO_RELOADED to external PSEUDO_UNLOAD environment
variable. Enable external programs to have a safe and reliable way to unload
pseudo on the next exec*. PSEUDO_UNLOAD also will disable pseudo if we're in a
fork/clone situation in the same way PSEUDO_DISABLED=1 would.
Rename the PSEUDO_DISABLED tests, and create a similar set for the new
PSEUDO_UNLOAD.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
|
|
This is a spiffied-up rebase of a bunch of intermediate changes, presented
as a whole because it is, surprisingly, less confusing that way. The basic
idea is to separate the guts code into categories ranging from generic
stuff that can be the same everywhere and specific variants. The big scary
one is the Darwin support, which actually seems to run okay on 64-bit OS X
10.6. (No other variants were tested.) The other example given is support
for the old clone() syscall on RHEL 4, which affects some wrlinux use cases.
There's a few minor cleanup bits here, such as a function with inconsistent
calling conventions, but nothing really exciting.
|
|
directly rather than via an on-demand spawn from the client, the
directory is never created.
|
|
This reverts commit 49d4d35918d457b0e9206679ecad3b9c84f11e66.
|
|
The cached data values were being collected when an OP_EXEC call was made.
This is incorrect as the values are only for logging purposes. It's believed
this caused an occasional crash in certain instances.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
|
|
The problem is that path_by_ino could end up being the same pointer
as cache_path, after which, if cache_path were freed (or kept around
for later), there would be malloc arena problems.
Also, fix the calculation for pathlen to increase cache hits. The
IPC messages use length of path *plus one* as the length, because
the buffer is defined to include its terminating null byte.
|
|
The pathlen that is cached could be wrong in certain operations (RENAME).
Fix this by resetting it to the proper path length.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
|
|
Add a cache of the last object found in pseudo_op. Profiling has indiciated
that many operations come in clusters. So instead of doing select, operation
for each item in the cluster, we check to see if we already know the item and
perform the op..
Performance improvement when processing 500k or so files:
Previous:
real 7m11.778s
user 0m35.929s
sys 2m46.723s
This commit:
real 6m42.093s
user 0m34.321s
sys 2m46.086s
Also validation of the component can be added by compiling with NVALIDATE.
This verifies the result of the cache is the same as what would have come
from the database. Differences are logged to the standard pseudo.log.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
|
|
We restructure the pseudo_op file identification, which involves a fairly
significant performance increase.
The old method would do:
if pdb_find_file_path:
found_path = 1;
if found_path && pdb_find_file_exact:
found_ino = 1;
else if pdb_find_file_dev:
found_ino = 1;
This resulted in at least two select calls for each file. One for "path"
and one for exact or dev.
The new method instead does:
if pdb_find_file_exact:
found_path = 1;
found_ino = 1;
else
if pdb_find_file_path:
found_path = 1;
if pdb_find_file_dev:
found_ino = 1;
This shrinks the number of selects to either one or three. Potentially cutting
the number of selects in half -- or increasing the number to three on an empty
set... (Profiling has shown this is a net win)
Timing numbers when manipulating a large number (500k) of files in a ramdisk:
real 7m48.354s
user 0m32.895s
sys 2m50.274s
After this change:
real 7m11.778s
user 0m35.929s
sys 2m46.723s
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
|
|
Add sqlite call profiling, this allows us to see the sqlite calls
that are being made as the system runs, via the pseudo log.
It was noted that by this profiling that a small change to pseudo.c,
when a file was found, reduced the sqlite SELECT calls by about 1/3.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
|
|
2010-12-09:
* (mhatle) Add doc/program_flow to attempt to explain startup/running
* (mhatle) guts/* minor cleanup
* (mhatle) Reorganize into a new constructor for libpseudo ONLY
pseudo main() now manually calls the util init
new / revised init for client, wrappers and utils
* (mhatle) Add central "reinit" function
* (mhatle) Add manul execv* functions
* (mhatle) rename pseudo_populate_wrappers to pseudo_check_wrappers
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
|
|
one device to another, for instance.
|
|
bug in the speculative-unlink operation.
The intent is to mark and then confirm or cancel the delete. This
removes the quirk where we tried to stash old database entries,
which didn't handle directories anyway; "rmdir non-empty-directory"
is a bit too common a case to dismiss as unthinkable.
|
|
Add local variable cache via get_value and set_value. The local cache
is setup at constructor time (or soon after).
Rewrite the pseudo_setupenv and pseudo_dropenv routines, add a new
pseudo_setupenvp and pseudo_dropenvp as well to handle the execve
cases.
We can now successfully use /usr/bin/env -i env and get pseudo values
back!
|
|
Add PSEUDO_BINDIR, PSEUDO_LIBDIR, and PSEUDO_LOCALSTATEDIR to allow for more
easy customization of PSEUDO components at run-time. If these are not set
they will be automatically generated based on the existing PSEUDO_PREFIX path.
PSEUDO_BINDIR = PSEUDO_PREFIX /bin
PSEUDO_LIBDIR = PSEUDO_PREFIX /lib
PSEUDO_LOCALSTATEDIR = PSEUDO_PREFIX /var/pseudo
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
|
|
CQ: WIND00225366
When moving a directory, pseudo performs the following sequence: stat
old, unlink new, link old and then rename. When linking a file, pseudo
first makes sure the file does not already exist in the database and
does an unlink. So the full sequence was stat old, unlink new, link
old ( unlink old, unlink contents of old, relink old ), rename. The
fix removes the unlinking of the contents of old.
|
|
libpseudo-foo.so.
|
|
The PSEUDO_SUFFIX thing is an installation quirk to allow our
build system to tag libpseudo.so with a checksum of the host libc.
However, we reuse a prebuilt pseudo server with the new pseudo
libraries; this means that encoding the suffix in the environment
hackery is a Bad Idea.
Update version number to 0.3, since this seems to wrap up a
hunk of development effort.
|
|
You can't use setenv() to modify the environment that will
be passed to a child process through execve()...
Also, fix the setupenv() to use PSEUDO_SUFFIX if defined.
Use execve() to spawn child processes, so we can use setupenv()
and dropenv().
|
|
We don't canonicalize OP_EXEC names (because they don't
want to be adapted for the chroot environment -- maybe we
should be doing that anyway, but right now we're not), so
mismatches with them are meaningless. So are mismatches
with the inode 0 reported for something that we never tried
to stat.
|
|
When you rename across devices, inode can change. Until now,
pseudo had no tools for handling a change in inode, but this
is clearly a legitimate case.
|
|
If multiple clients are active at once, the following could occur:
* Client #1 unlinks file A
* Client #2 creates file B, which reuses A's inode
* Client #2 sends request to server
* Client #1 sends request to server
* Processing client #2's request creates a mismatch warning for
file A/B.
* Processing client #1's request creates a mismatch warning too.
Note that this can happen even if Client #2 sends its request later,
as there's no intrinsic guarantee of the order in which requests
are processed; any SINGLE client is presumably executing operations
in order, but multiple clients aren't.
Fixing this in rmdir, unlink, and rename.
|
|
|
|
It is possible for the database to get out of sync with the
filesystem. Detecting this after the fact can be hard. Provide a
hook for requesting a check.
Also merge in some LD_LIBRARY_PATH fixes.
|
|
Send program name (program_invocation_name from glibc) along with the
tag.
Along the way, restructure the fds/pids/tags arrays to be an array
of client structures in pseudo_server, and add the message type
to the set of things logged -- logging that a message was a ping is
more useful than appending the text "ping" to it. Add support
for type and program to pseudolog.
Add deletion to pseudolog.
Handle usage message formatting when there's an odd number of known
specifiers for pseudolog.
Conflicts:
ChangeLog.txt
pseudo_server.c
|
|
This is a first pass at handling password/group calls, allowing
the use of custom password/group files. In particular, when
chroot()ed to a particular directory, pseudo picks files in
that directory by default, to improve support for the typical
use case where pseudo uses chroot() only to jump into a virtual
target filesystem.
|
|
|
|
This allows us to track execution, although the tracking for it
requires some additional thought -- the basic assumption is that we
don't want to canonicalize names into the chroot() directory, but
since all the filename canonicalization assumes that we want this,
that will take some sneaking. It's a little useful as is, though,
so I'm running with it.
|
|
This patch adds support for checking whether a file was opened for
reading, writing, or both, as well as tracking append flags. It is
not very well tested. This is preparation for improved host
contamination checking.
|
|
None of them seem to have been genuine problems, but it's prettier now,
and some were questionable.
|
|
Add chroot() and a large number of things needed to make it work.
The list of intercepted calls is large but not exhaustive.
|
|
* Improve makewrappers handling of function pointer arguments.
* Regenerate wrappers when makewrappers is touched.
* Move path resolution from pseudo_client_op into wrapper
functions.
* Eliminate dependency on PATH_MAX.
* Related cleanup, such as tracking CWD better, and using
the tracked value for getcwd().
|
|
Add the -h (help) option to pseudo, and document -h for both
pseudo and pseudolog.
|
Peter Seebach
Seebs
Mark Hatle
seebs
Konrad Scherer
Peter Seebach