systemd-fsck: fix systemd-fsck/fsck pipe bad closure
Currently, when console is disabled but progress is tracked, pipe opened
for communication between systemd-fsck and fsck may be closed
inadvertently (when opening of /dev/console return in error). That lead
to finish fsck prematurely (because it receives a SIGPIPE) and so fsck
may not check correctly filesystems and do not have time to fix memory
corruptions.
This commit changes the opening of /dev/console to be done previously to
pipe creation and so fix the bug described just above.
Aaron Plattner [Thu, 17 Oct 2019 19:56:15 +0000 (12:56 -0700)]
coredump: Include module offsets in stack traces
These offsets can be useful to decode stack traces through modules that don't
have symbol names. For example, with a simple test that crashes after calling
through several static functions, systemd-coredump reports this:
Oct 17 : Process 640333 (a.out) of user 1000 dumped core.
core/load-fragment: constify Unit* arguments where possible
This makes it easy to tell that the function only uses the Unit* for
reporting, and only makes changes to the other argument (which most likely
also points at the same Unit structure) for modifications.
Jan Synacek [Mon, 7 Oct 2019 08:03:07 +0000 (10:03 +0200)]
udev: introduce CONST key name
Currently, there is no way to match against system-wide constants, such
as architecture or virtualization type, without forking helper binaries.
That potentially results in a huge number of spawned processes which
output always the same answer.
This patch introduces a special CONST keyword which takes a hard-coded
string as its key and returns a value assigned to that key. Currently
implemented are CONST{arch} and CONST{virt}, which can be used to match
against the system's architecture and virtualization type.
core: adjust load functions for other unit types to be more like service
No functional change, just adjusting code to follow the same pattern
everywhere. In particular, never call _verify() on an already loaded unit,
but return early from the caller instead. This makes the code a bit easier
to follow.
Now all unit types define .load. But even if it wasn't defined, we'd need
to call unit_load_fragment_and_dropin() anyway, so this code would not have
worked correctly.
Also, unit_load_fragment_and_dropin() either returns -ENOENT or changes
UNIT_STUB to UNIT_LOADED, so we don't need to repeat this here.
core/service: use common implementation of unit_load_fragment_and_dropin()
There is a slight functional change when load_state == UNIT_MERGED. Before,
we would not call unit_load_dropin(), but now we do. I'm not sure if this
causes an actual difference in behaviour, but since all other unit types do
this, I think it's better to do the same thing here too.
core: turn unit_load_fragment_and_dropin_optional() into a flag
unit_load_fragment_and_dropin() and unit_load_fragment_and_dropin_optional()
are really the same, with one minor difference in behaviour. Let's drop
the second function.
"_optional" in the name suggests that it's the "dropin" part that is optional.
(Which it is, but in this case, we mean the fragment to be optional.)
I think the new version with a flag is easier to understand.
parse_hwdb: fix compatibility with pyparsing 2.4.*
pyparsing 2.3.1/2.4.0 had some changes to grouping of And matches, and as a
result we'd report 0 properties and 0 matches, and not really do any checks.
With this change we get identical behaviour for pyparsing 2.3.1, 2.4.0, 2.4.2:
$ hwdb/parse_hwdb.py
hwdb/60-evdev.hwdb: 72 match groups, 94 matches, 262 properties
hwdb/60-input-id.hwdb: 3 match groups, 3 matches, 4 properties
hwdb/60-keyboard.hwdb: 173 match groups, 256 matches, 872 properties
Keycode KBD_LCD_MENU1 unknown
Keycode KBD_LCD_MENU4 unknown
Keycode KBD_LCD_MENU2 unknown
Keycode KBD_LCD_MENU3 unknown
hwdb/60-sensor.hwdb: 101 match groups, 120 matches, 105 properties
hwdb/70-joystick.hwdb: 2 match groups, 3 matches, 2 properties
hwdb/70-mouse.hwdb: 104 match groups, 119 matches, 123 properties
hwdb/70-pointingstick.hwdb: 8 match groups, 30 matches, 11 properties
hwdb/70-touchpad.hwdb: 6 match groups, 9 matches, 6 properties
parse_hwdb: bail with an error if no matches or groups are detected
pyparsing sometimes changes behaviour and stops giving matches. This should
allow us to detect such scenario. With this change, parse_hwdb fails with
pyparsing 2.4 on F31.
This change is only about the source tree. We have tmpfiles.d/, modprobe.d/,
sysctl.d/, and sysusers.d/, but for historical reasons, rules/ didn't fit this
pattern. We also *install* it as rules.d/. Let's rename to be consistent.
Hans de Goede [Mon, 23 Sep 2019 09:00:50 +0000 (11:00 +0200)]
hwdb: Update Primebook C11B sensor entry to also work with older BIOS versions
Older Primebook C11B BIOS versions use "Primebook C11B" as product name
instead of "PRIMEBOOK C11B", update the Primebook C11B 60-sensor.hwdb entries
to match on both spellings.
For executables which take a verb, we should list the verbs first, and
then options which modify those verbs second. The general layout of
the man page is from general description to specific details, usually
Overview, Commands, Options, Return Value, Examples, References.
test/TEST-31-DEVICE-ENUMERATION: do not use -x to avoid grep loop
https://github.com/systemd/systemd/pull/13746#issuecomment-539410752:
> [grep] now matches the grep command itself, as it's logged into journal as well, thanks to set -x.
Also, use journalctl --grep and -t to make things a bit quicker.
test: add function to reduce copied setup boilerplate
Many tests were also masking systemd-machined.service. But machined
should only start when activated, so having it not masked shouldn't be
noticable. TEST-25-IMPORT needs it.
test: drop redirection to tty in integration tests
I *think* this was originally added to make it easier to see what was happening
in tests. Later we added the functionality to print the journal on failure, so
this redirection has stopped being useful.
In https://github.com/systemd/systemd/pull/13719#issuecomment-539292650
@filbranden shows that grep tries to write to stdout and fails. In general,
we should not assume that writing to the console it always possible. We have
special code to handle this in pid1 after all:
Tim Teichmann [Sat, 5 Oct 2019 13:52:37 +0000 (15:52 +0200)]
Add missing license file and information for tools/chromeos/gen_autosuspend_rules.py (#13729)
The license file for the python script that was commited with b61d777abeecd8b6c76035e11899aae210633534 was missing. The license was copied from https://chromium.googlesource.com/chromiumos/platform2/+/master/LICENSE.
tty-ask-pwd-agent: treat SIGINT as a request to exit immediately
Unlike SIGTERM, SIGINT is now treated as a request to exit as soon as
possible. IOW, if SIGINT is received, the agent wont process all remaining
passwords before exiting.
This allows a more comprehensive behavior when C-c is pressed and when the
agent is spawned by systemctl.
Before that patch, pressing C-c killed systemctl but left the agent waiting
for a password since SIGINT was blocked. The result was pretty clumsy.
Semmle Security Reports report:
> The problem occurs on the way realloc is being used. When a size
> bigger than the chunk that wants to be reallocated is passed, realloc
> try to malloc a bigger size, however in the case that malloc fails
> (for example, by forcing a big allocation) realloc will return NULL.
>
> According to the man page:
> "The realloc() function returns a pointer to the newly allocated
> memory, which is suitably aligned for any built-in type and may be
> different from ptr, or NULL if the request fails. If size was
> equal to 0, either NULL or a pointer suitable to be passed to free()
> is returned. If realloc() fails, the original block is left
> untouched; it is not freed or moved."
>
> The problem occurs when the memory ptr passed to the first argument of
> realloc is the same as the one used for the result, for example in
> this case:
>
> dmesg = realloc(dmesg, dmesg_size + strlen(pe->dirent.d_name) +
> strlen(":\n") + pe->content_size + 1);
>
> https://lgtm.com/projects/g/systemd/systemd/snapshot/f8bcb81955f9e93a4787627e28f43fffb2a84836/files/src/pstore/pstore.c?sort=name&dir=A
> SC&mode=heatmap#L300
>
> If the malloc inside that realloc fails, then the original memory
> chunk will never be free but since realloc will return NULL, the
> pointer to that memory chunk will be lost and a memory leak will
> occur.
>
> In case you are curious, this is the query we used to find this problem:
> https://lgtm.com/query/8650323308193591473/
Let's use a more standard pattern: allocate memory using greedy_realloc, and
instead of freeing it when we wrote out a chunk, let's just move the cursor
back to the beginning and reuse the memory we allocated previously.
If we fail to allocate the memory for dmesg contents, don't write the dmesg
entry, but let's still process the files to move them out of pstore.
Ryan Attard [Fri, 4 Oct 2019 12:52:49 +0000 (07:52 -0500)]
ata_id: Add check for fixed format sense codes (#13654)
Original revisions of the SAT (SCSI-ATA Translation) specification
required that all sense data be reported in Descriptor Format (72h).
Later revisions specifcally allow and account for sense data being
reported in Fixed Format (70h).
The current code checks for a Descriptor Format sense structure (0x72),
then looks specifically at the first byte of the first descriptor for the
ATA specific code 0x9, cross referencing it with the first byte which is
just a length field 0x0c (as a sanity check).
In the Fixed Format case(0x70), we can fall back to using the top-level
SCSI Sense data for the Additional Sense code (0x0) and then the
Additional Sense Code Qualifier (0x1d),
That identifies that the sense data is of the format associated with:
`ATA PASS THROUGH INFORMATION AVAILABLE`.
This fallback mechanism retains support for SATLs compliant with ANSI
INCITS 431-2007, and enables support for Fixed Format Sense data
enabled by SATLs with later revisions.
Glad to do so. This patch allows ata_id to export attributes correctly. I believe that any drive can potentially return information in this format on any SATL using the libata-scsi (the Linux builtin SATL), but in this particular case, it appears it is the SATL itself. Attaching the disk to the AHCI controller changes the behavior impacted here. (Not entirely surprisingly, SATLs are are pretty inconsistent).
Test:
This case specifically is an LSI SATL. I'll illustrate that without the patch, ata_id does not return
any output for a valid SATA drive but after the patch does.
1. Verify the device is ATA, by looking at the vpd page specific to ATA drives
```
root@machine:~# sg_vpd -p ai /dev/sdn
ATA information VPD page:
SAT Vendor identification: LSI
SAT Product identification: LSI SATL
SAT Product revision level: 0008
Device signature indicates SATA transport
ATA command IDENTIFY DEVICE response summary:
model: HGST HUH728080ALE604
serial number: ZZZZH3VX
firmware revision: A4GNW7J0
```
2. Look at what udev thinks of the disk, it says ID_BUS=scsi
ATA information says ID_MODEL should be HGST_HUH728080ALE604
udev says it is HGST_HUH728080AL (Missing E604, 4 bytes), and no ATA attributes are
populated.
If I install the same disk into a machine using an ATA driver, this behavior changes:
```
root@machine2:~# sg_vpd -p ai /dev/sdb
ATA information VPD page:
SAT Vendor identification: linux
SAT Product identification: libata
SAT Product revision level: 3.00
Device signature indicates SATA transport
ATA command IDENTIFY DEVICE response summary:
model: HGST HUH728080ALE604
serial number: ZZZZH3VX
firmware revision: A4GNW7J0
root@machine-2:~# /lib/udev/ata_id -x /dev/sdb
ID_ATA=1
ID_TYPE=disk
ID_BUS=ata
ID_MODEL=HGST_HUH728080ALE604
ID_MODEL_ENC=HGST\x20HUH728080ALE604\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20
ID_REVISION=A4GNW7J0
<truncated>
```
The ChromeOS ecosystem has a large amount of testing, both automated
and manual across devices including measurement of power regressions.
It's safe to assume that any of these devices will handle USB
auto-suspend appropriately. Use the script from ChromeOS
https://chromium.googlesource.com/chromiumos/platform2/+/master/power_manager/udev/gen_autosuspend_rules.py
to generate udev rules at build time.
This script in systemd `tools/chromeos/gen_autosuspend_rules.py` should be kept
in sync with the ChromeOS version of the script.
Manually added autosuspend devices should be placed in the new
template `rules/61-autosuspend-manual.rules`
Suggested-by: Hans de Goede <hdegoede@redhat.com> Signed-off-by: Mario Limonciello <mario.limonciello@dell.com>
Zach Smith [Fri, 4 Oct 2019 01:09:35 +0000 (18:09 -0700)]
systemd-tmpfiles: rename force to append_or_force
The force field of the Item struct is used to indicate
force creation or appending in different context. This
change renames the field to append_or_force to improve
readability.
In case PID1 is going to switch to a new root and execute a new system manager
which is not systemd, we should restore the original environment as the new
manager might expect some variables to be set by default (more specifically
$HOME).
Dan Streetman [Mon, 16 Sep 2019 16:34:55 +0000 (12:34 -0400)]
test: add temporarily blacklisted tests
This temporarily blacklists some tests when run under Ubuntu CI.
This is the upstream side of the Debian 'upstream' test MR:
https://salsa.debian.org/systemd-team/systemd/merge_requests/52
The tests blacklisted here should only be temporarily blacklisted
until they can be fixed; the intention is that these blacklist files
will be added and removed over time while debugging/fixing flaky
and/or regressed tests, without causing test failure noise for other
PRs.
Chris Down [Thu, 3 Oct 2019 12:21:29 +0000 (13:21 +0100)]
cgroup: analyze: Report memory configurations that deviate from systemd
This is the most basic consumer of the new systemd-vs-kernel checker,
both acting as a reasonable standalone exerciser of the code, and also
as a way for easy inspection of deviations from systemd internal state.
Chris Down [Mon, 30 Sep 2019 15:13:32 +0000 (16:13 +0100)]
cgroup: Allow checking systemd-internal limits against the kernel
We currently don't have any mitigations against another privileged user
on the system messing with the cgroup hierarchy, bringing the system out
of line with what we've set in systemd. We also don't have any real way
to surface this to the user (we do have logs, but you have to know to
look in the first place).
There are a few possible solutions:
1. Maintaining our own cgroup tree with the new fsopen API and having a
read-only copy for everyone else. However, there are some
complications on this front, and this may be infeasible in some
environments. I'd rate this as a longer term effort that's tangential
to this patch.
2. Actively checking for changes with {fa,i}notify and changing them
back afterwards to match our configuration again. This is also
possible, but it's also good to have a way to do passive monitoring
of the situation without taking hard action. Also, currently daemons
like senpai do actually need to modify the tree behind systemd's
back (although hopefully this should be more integrated soon).
This patch implements another option, where one can, on demand, monitor
deviations in cgroup memory configuration from systemd's internal state.
Currently the only consumer is `systemd-analyze dump`, but the interface
is generic enough that it can also be exposed elsewhere later (for
example, over D-Bus).
Currently only memory limit style properties are supported, but later I
also plan to expand this out to other properties that systemd should
have ultimate control over.
projects / thirdparty / systemd.git / log
