David Strauss [Tue, 3 Nov 2015 01:19:26 +0000 (17:19 -0800)]
man: Drop "internally," which is misleading
The existing text misleads readers into thinking how the notify socket protocol is "internals" and that they can only use the functions. However, the socket is part of the Interface Stability Promise. So, we should drop "internally" from the description so the man page both indicates both how the functions work and how one would talk to the socket directly.
log-generator: add option to generate easily compressible data
This is useful to check that compression actually works, and how
compression influences file size in the best-case-scenario for
compression. (The answer is that not as much as one would hope:
there's still a big overhead of the indexing and since every field
is compressed separately, even fields that compress very well
contribute to the file size. This overhead becomes negligible only
for very big fields.)
journalctl: when we fail to open a journal file, print why
When we enumerate journal files and encounter an invalid one, remember
which this, and show it to the user.
Note the possibly slightly surprising logic here: we store only one path
per error code. This means we show all error kinds but not every actual
error we encounter. This has the benefit of not requiring us to keep a
potentially unbounded list of errors with their sources around, but can
still provide a pretty complete overview on the errors we encountered.
- Always print a debug log message about files and directories we cannot
open right when it happens instead of the caller, thus reducing the
number of places where we need to generate the debug message.
- Always push the errors we encounter immediately into the error set,
when we run into them, instead of in the caller. Thus, we never forget
to push them in.
- Use stack instead of heap memory where we can.
- Make remove_file() void, since it cannot fail anyway and always
returned 0.
- Make local machine check of journal directories explicit in a
function, to make things more readable.
- Port to all directory listing loops FOREACH_DIRENT_ALL()
- sd-daemon is library code, hence never log at higher log levels than
LOG_DEBUG.
When reading stuff, we should only return EIO when an actual read error
occured, not when we don't like the data for whatever reason.
We already return ENODATA for all other kinds of file truncation, hence
do the same for the most obvious kind, so that callers know what ENODATA
means.
core: expose number of file descriptors in fd store on the bus
For each service expose how many file descriptors there are currently in
the fd store.
(Also, fix the exporting of the fdstore limit, given that the field is
just an "unsigned" but we exported it as "uint32_t". Not that there way
any effective difference, but let's make this clean...)
string-util: rework memory_erase() so that it cannot be optimized away
memory_erase() so far just called memset(), which the compiler might
optimize away under certain conditions if it feels there's benefit in
it. C11 knows a new memset_s() call that is like memset(), but may not
be optimized away. Ideally, we'd just use that call, but glibc currently
does not support it. Hence, implement our own simplistic version of it.
We use a GCC pragma to turn off optimization for this call, and also use
the "volatile" keyword on the pointers to ensure that gcc will use the
pointers as-is. According to a variety of internet sources, either one
does the trick. However, there are also reports that at least the
volatile thing isn't fully correct, hence let's add some snake oil and
employ both techniques.
Martin Pitt [Mon, 2 Nov 2015 16:05:20 +0000 (10:05 -0600)]
core: drop check for /etc/mtab
util-linux 2.27.1 now entirely stops looking at /etc/mtab, so we don't need to
verify /etc/mtab during early boot any more. Later on, tmpfiles.d/etc.conf will
fix /etc/mtab anyway, so there's not even a point in warning about it.
Drop test_mtab() and bump the util-linux dependency to >= 2.17.1.
journald: never block when sending messages on NOTIFY_SOCKET socket
Otherwise we might run into deadlocks, when journald blocks on the
notify socket on PID 1, and PID 1 blocks on IPC to dbus-daemon and
dbus-daemon blocks on logging to journald. Break this cycle by making
sure that journald never ever blocks on PID 1.
Note that this change disables support for event loop watchdog support,
as these messages are sent in blocking style by sd-event. That should
not be a big loss though, as people reported frequent problems with the
watchdog hitting journald on excessively slow IO.
sysctl.d: bump number of queueable AF_UNIX/SOCK_DGRAM datagrams
The default of 16 is pretty low, let's bump this to accomodate for more
queued datagrams. This is useful for AF_UNIX/SOCK_DGRAM logging and
sd_notify() sockets as this allows queuing more datagrams before things
start to block, thus improving parallelization and logging performance.
systemctl: add a way to explicitly request client-side unit installing
This adds support for a new environment variable
SYSTEMCTL_INSTALL_CLIENT_SIDE, that ensures that systemctl executes
install operations client-side instead of passing them to PID1. This is
useful in debugging situations, but even beyond that. However, we don't
want to make it official API, hence let's just make it an undocumented
environment variable.
Similar, add a second variable, SYSTEMCTL_SKIP_SYSV which allows
skipping the SysV chkconfig fall-back if set. This is useful for similar
reasons, and exposed as undocumented as environment variable for similar
reasons, too.
Dongsu Park [Wed, 21 Oct 2015 12:51:10 +0000 (14:51 +0200)]
journal-remote: increase memory limit per connection to avoid errors with HTTPS
Explicitly set MHD_OPTION_CONNECTION_MEMORY_LIMIT to a larger value,
when setting up microhttpd, to give more memory per HTTP(S) connection.
This way systemd-journal-remote can now prevent microhttpd from failing
in creating response headers with messages like "Not enough memory for
write", especially when lots of HTTPS requests arrive. That's precisely
because MHD_OPTION_CONNECTION_MEMORY_LIMIT in libmicrohttpd defaults to
32768, which is in practice insufficient in this case.
See also https://gnunet.org/bugs/view.php?id=4007 for more details.
David Herrmann [Wed, 28 Oct 2015 18:11:36 +0000 (19:11 +0100)]
core: fix priority ordering in notify-handling
Currently, we dispatch NOTIFY messages in a tight loop. Regardless how
much data is incoming, we always dispatch everything that is queued.
This, however, completely breaks priority event-handling of sd-event.
When dispatching one NOTIFY event, another completely different event
might fire, or might be queued by the NOTIFY handling. However, this
event will not get dispatched until all other further NOTIFY messages are
handled. Those might even arrive _after_ the other event fired, and as
such completely break priority ordering of sd-event (which several code
paths rely on).
Break this by never dispatching multiple messages. Just return after each
message that was read and let sd-event handle everything else.
(The patch looks scarier that it is. It basically just drops the for(;;)
loop and re-indents the loop-content.)
test-parse-util: Move parse-util tests into their own test case
Tests for the functions defined in src/basic/parse-util.c. Reorder them
to match the order in which the functions are defined in the source
file. Adjusted the list of include files to remove the ones no longer
needed in test-util.c.
Tested that `make check` still passes as expected. Also checked the
number of lines removed from test-util.c matches the expected, as an
additional verification that no tests were dropped or duplicated in the
move.