Replace confusing libsystemd error messages (version < 257 or missing development files) during ./configure
with a message clearly indicating that sd-json support is disabled, as it is optional. Previously, users
saw:
```
$ ./configure
Package dependency requirement 'libsystemd >= 257' could not be satisfied.
Package 'libsystemd' has version '252', required version is '>= 257'
Package dependency requirement 'libsystemd >= 257' could not be satisfied.
Package 'libsystemd' has version '252', required version is '>= 257'
```
```
$ ./configure
Package libsystemd was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsystemd.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libsystemd', required by 'virtual:world', not found
Package libsystemd was not found in the pkg-config search path.
Perhaps you should add the directory containing `libsystemd.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libsystemd', required by 'virtual:world', not found
```
fix(dracut-install): install all suppliers of a supplier's module
After installing a module, we get its dependencies and suppliers' paths,
and recurse into two functions that handle installing those. The code
handling a supplier path tries to also handle suppliers-of-suppliers
relations. But when installing a supplier's module, it only recurses to
suppliers for that specific device provided by the module.
Having a subset of its suppliers might not be enough for a module to
work correctly. Recurse into all suppliers of the supplier modules we
are installing.
fix(dracut-install): do not limit supplier handling to platform bus
The code that explores a sysfs node for suppliers looks for directories
with a "supplier:platform" prefix. Only the "supplier:" prefix refers to
the devlink relation itself here, the rest is the bus and device name of
the supplier. The bus is not necessarily "platform", but can be "phy",
"regulator", etc.
Correct the searched prefix for sysfs supplier relations to "supplier:"
so we can handle all supplier dependencies to devices, not just those
under the platform bus.
The initial device search is still limited to "/sys/devices/platform".
It theoretically limits the dependencies we check to that directory, but
on device-tree systems it looks like all devices are available there and
we don't miss anything.
fix(dracut-install): add sysfs node parents' modules as dependencies
While searching a sysfs node for suppliers to add as dependencies, we
also consider suppliers of its parent nodes as dependencies. The code
doing so doesn't explore suppliers-of-suppliers chains, which is meant
to be handled elsewhere. More importantly it misses the parent nodes of
the devices themselves, parents-of-suppliers chains, and their
dependencies; all of which is necessary to get a device working.
Consider the parent nodes of a sysfs node as a supplier dependency when
building the `module_suppliers` hashmap. So the suppliers-of-suppliers
handling elsewhere can work with any combination of parent/supplier
relation.
James Le Cuirot [Thu, 27 Mar 2025 19:20:35 +0000 (19:20 +0000)]
test: extend TEST-40-SYSTEMD to cover add_dlopen_features
Use systemd's fido2 feature because nothing else is likely to pull in
libfido2.so.1. Not all of the containers currently have a new enough
systemd version for this to work. The test will effectively be skipped
until the container includes a new enough systemd-devel and libfido2.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
James Le Cuirot [Wed, 26 Mar 2025 13:02:35 +0000 (13:02 +0000)]
feat(dracut): allow users to choose which dlopen dependencies they want
Handling dlopen dependencies is nice, but installing these
unconditionally will install more than before rather than less, leading
to bigger images and unhappy users.
This introduces the add_dlopen_features and omit_dlopen_features
configuration options. Modules that are successfully loaded set the
default set of features to add_dlopen_features in the config() function.
Users can request additional features by appending to this variable.
They can also omit features by appending to omit_dlopen_features, which
takes precedence.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
James Le Cuirot [Wed, 5 Mar 2025 11:53:18 +0000 (11:53 +0000)]
feat(dracut): replace ldd with dracut-install --dry-run or header check
One instance checks whether */lib64/* is used by /bin/sh and another
checks whether libusb is used by scdaemon. These can be handled by the
new dracut-install --dry-run option.
find_binary currently uses ldd to check whether a given *.so* is a valid
ELF. ldd exits successfully even when libraries are missing, so it is
sufficient to replace this check with one that looks at the first 4
bytes.
Closes: https://github.com/dracut-ng/dracut-ng/issues/338 Closes: https://github.com/dracut-ng/dracut-ng/issues/1257 Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
James Le Cuirot [Tue, 4 Mar 2025 17:21:27 +0000 (17:21 +0000)]
feat(dracut-install): add --dry-run option to replace external ldd usage
To remove the remaining use of ldd, we need a way to show which
libraries a binary requires. I initially considered adding another small
tool, sharing code with dracut-install, but then I realised that adding
a --dry-run option to dracut-install would also meet that need with a
lot less effort.
It simply shows what would be installed and doesn't require you to
specify a destination directory. It is similar to the existing --logdir
option, but that cannot log to stdout and includes additional output.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
James Le Cuirot [Tue, 4 Mar 2025 11:09:32 +0000 (11:09 +0000)]
feat(dracut-install): extend new ELF parsing code to replace ldd calls
Now that dracut-install has its own ELF parsing code to handle dlopen
dependencies, it is only a small additional step to also handle
traditional DT_NEEDED dependencies, removing the need to call ldd, which
is not cross-friendly.
This avoids the earlier issue seen with musl in #1087.
We should no longer directly install libsystemd*.so because
libsystemd-core does not have the RUNPATH to find libsystemd-shared by
itself. Both get pulled in by the main systemd binary anyway. ldd had
the same issue, dracut-install silently ignored the failure.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
James Le Cuirot [Tue, 18 Feb 2025 17:24:05 +0000 (17:24 +0000)]
feat(dracut-install): parse ELF .note.dlopen entries for extra deps
Unlike traditional DT_NEEDED dependencies, there has not been a way to
determine what libraries an ELF may dlopen until recently. systemd has
documented a convention to declare such dependencies using JSON in the
ELF metadata. See https://systemd.io/ELF_DLOPEN_METADATA/ for details.
This metadata references sonames rather than full paths, so Dracut needs
to determine the full paths by itself. It cannot use ldd to do this as
that relies on DT_NEEDED. ldconfig can show the paths for all sonames in
the cache, but that relies on the cache having already been generated,
it isn't cross-friendly, and musl doesn't even have ldconfig. It
therefore makes sense for Dracut to parse the ELF headers directly. This
also paves the way for removing the dependency on ldd entirely, making
Dracut more cross-friendly as a whole.
To avoid adding an entirely new dependency, the JSON parsing is done by
libsystemd's sd-json API. This has been exposed since systemd v257. If
libsystemd is too old or not present at all, then this dlopen handling
is simply skipped. This is currently not an issue for non-systemd
distributions as systemd is the only project using this convention. If
that were to change, libsystemd can still be used without the rest of
systemd, as demonstrated by Gentoo.
The metadata itself has only been included by systemd since v256. If an
earlier version is detected, Dracut will unconditionally install the
same libraries that it did before.
There are different structs for 32-bit and 64-bit ELF headers, so this
new code makes heavy use of C macros to avoid a lot of code duplication.
One macro is also used heavily for endian conversion, as almost every
field needs to be adjusted.
See the code comments for the remaining details.
Closes: https://github.com/dracut-ng/dracut-ng/issues/154 Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Jo Zzsi [Wed, 23 Apr 2025 23:21:59 +0000 (19:21 -0400)]
fix: load more kernel modules in sloppy hostonly mode
When hostonly_mode is set to "sloppy" instmods consults the host for currently loaded kernel modules and
refuses to copy the Linux kernel module into the generated initramfs unless the kernel
module is already loaded on the host.
Let's set hostonly explicitly to a better default before calling instmods.
For compatibility, we should not change the default in the instmods function itself.
James Le Cuirot [Thu, 27 Mar 2025 16:44:30 +0000 (16:44 +0000)]
test: add TEST-13-SYSROOT to cover --sysroot option
This really just uses the regular root filesystem but symlinked to
another directory. Ideally, this would use an image for a different
architecture, but that would be more intensive.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
James Le Cuirot [Wed, 5 Mar 2025 17:29:51 +0000 (17:29 +0000)]
fix(dracut-install): rework broken destination existence logic
The return code of `stat` is checked twice when it only needs to be
checked once. The `dst_exists` condition will also never be true, making
the variable redundant.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
James Le Cuirot [Thu, 5 Sep 2024 11:19:09 +0000 (12:19 +0100)]
feat(dracut): set systemdversion global var using pkg-config
This falls back to 0 if the version cannot be determined. The version
isn't a regular pkg-config variable like the others, but we still want
the ability to override this through the Dracut config, so make
"modversion" a special case.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
James Le Cuirot [Thu, 5 Sep 2024 10:55:35 +0000 (11:55 +0100)]
fix(dracut): respect PKG_CONFIG env var instead of hardcoding pkg-config
When using a sysroot, we should use pkg-config data from the sysroot.
While we could set PKG_CONFIG_LIBDIR, the lib directory can vary.
Distributions typically set up pkg-config wrappers for this purpose and
it is customary to respect the PKG_CONFIG variable in build systems.
Users can still set simply PKG_CONFIG_LIBDIR instead if they prefer that
approach.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
fix(multipath): disable user_friendly_names with mpathconf
If dracut is creating /etc/multipath.conf by calling mpathconf in either
multipathd-configure.service or multipathd.sh, there is a chance that
the multipath config in the real root differs. Specifically, it might
have chosen different user_friendly_names for the devices. When the
systems switches to the real root, multipath may not be able to switch
the devices to their configured names because those might already be
in use. To avoid this, call mpathconf with "--user_friendly_names n"
to create a multipath.conf with user_friendly_names disabled. If all
devices use WWID names, it is always possible for multipath to rename
them later.
Fixes b8a92b715 ("multipath: add automatic configuration for multipath")
Mingcong Bai [Sun, 26 Jan 2025 21:00:38 +0000 (05:00 +0800)]
feat: strip out unused/unlikely AMDGPU firmware
Introduce logic to remove unlikely or impossible firmware on a platform-
specific basis to save space - this is the largest set of firmware in the
linux-firmware.git tree.
To update the list of prefixes:
1. Look for (grep) MODULE_FIRMWARE in /drivers/gpu/drm/amd (please note
that AMD does not always use the file name to the firmware as
parameter to this macro.
2. Reference "Misc AMDGPU driver information"[^1], Wikipedia, and
TechPowerUp to determine which class certain set(s) of firmware
belong out of the list below.
3. Refer to manufacturers and/or OEM contacts for state of support
(some should be obvious - there is no non-x86 APUs - yet?).
Jo Zzsi [Tue, 22 Apr 2025 22:04:18 +0000 (18:04 -0400)]
fix: improve hostonly sloppy mode
Sloppy mode is the original hostonly mode.
Strict mode was newly introduced much later in 2018 (v048) as a non-default
opt-in special optimization mode to allow modules to ignore more drivers
or do some extra job to save memory and disk space, while making the image less portable.
This PR includes more dracut modules (and with that more kernel modules) in sloppy mode.
fix(dracut): protect existing output file against build errors
If dracut fails to build the initrd image or EFI binary for any reason (e.g., if
`cp` fails because there is no space left on the device), it removes the
existing output file before exiting, which may result in an unbootable system.
Instead of copying the initrd image directly to the output, copy it alongside it
to the same output directory, and if the copy succeeds, replace it.
James Le Cuirot [Fri, 14 Mar 2025 14:08:00 +0000 (14:08 +0000)]
fix: add $dracutsysrootdir to paths where it should be present
inst_simpl is sometimes called with the sysroot (particular via moddir)
and sometimes without. dracut-install knows how to handle this, so the
inst_simpl existence check needs to handle it too.
Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Not only are the specific encryption modules not being installed on s390/s390x
in non-hostonly mode, but also it's failing the detection of the kernel modules
needed to decrypt PAES-encrypted volumes in hostonly mode via dmsetup. So,
since the increase in size is not that much (~ 150K), always add all the drivers
under s390/crypto in this architecture.
On usrmerge systems, /lib is a symlink to /usr/lib, so the real path of the
kernel directory is /usr/lib/modules/<kver>. The issue on these systems is that
a kernel module can be added to the initrd (via dracut-install) by different
dracut modules using both paths, and that causes libkmod to display the
following type of errors in the middle of the dracut build:
fix(nfs): libnfsidmap plugins not added in some distributions
`nfs-utils` can be configured using `--with-pluginpath` to avoid using the
default `/usr/lib/libnfsidmap`. For example, Fedora sets
`--with-pluginpath=%{_libdir}/libnfsidmap`, which is covered by the current
glob, but openSUSE sets `--with-pluginpath=%{_libdir}/libnfsidmap-1.0.0`.
Also, remove reference to the old `libnfsidmap_<plugin>.so` path.
Benjamin Drung [Thu, 3 Apr 2025 12:14:07 +0000 (14:14 +0200)]
feat: add simpledrm module (as subset of drm module)
Plymouth doesn't always show a splash screen if DRM drivers are
installed in initrd.
Provide a `simpledrm` module that only installs the SimpleDRM module
and the potentially needed privacy screen providers. This `simpledrm`
module is a subset of the `drm` module. It could be used instead of
`drm` to avoid pulling in drivers like amdgpu, nouveau, or nvidia-drm.
fix(multipath): skip default multipath.conf with mpathconf
Commit 1e802f15f creates a default multipath.conf file with
"find_multipaths strict" when run in non-hostonly mode if there are no
multipath devices and no multipath.conf. Unfortunately for systems that
want to use mpathconf to create a multipath.conf file (e.g. Fedora and
Centos) either through multipathd-configure.service or multipathd.sh,
this default file keeps that from occurring. To make sure mpathconf is
called to create the config file, do not install a default config file
if mpathconf is installed.
Fixes: ("fix(multipath): include module with find_multipaths strict") Signed-off-by: Benjamin Marzinski <bmarzins@redhat.com>
Xinhui Yang [Fri, 28 Feb 2025 16:54:31 +0000 (00:54 +0800)]
fix(90kernel-modules): explicitly include xhci-pci-renesas
Since Linux v6.12-rc1 (commit 25f51b76f90f), xhci-pci no longer depends
on xhci-pci-renesas, causing the Renesas driver to be omitted during
initramfs generation (when built as a module).
This makes platforms with such xHCI controllers unavailable during
initrd, and unable to boot from a USB drive. There are SuperSpeed ports
routed through such controller on some platforms, too, which also
renders the USB keyboard and mouse unusable.
Here's a snippet of the kernel log from such platform, showing a
keyboard and a mouse being detected only after the initrd switched root:
[ 9.352608] systemd-journald[187]: Received SIGTERM from PID 1 (systemd).
[ 9.500146] systemd[1]: systemd 257.2 running in system mode (OMITTED)
...
[ 11.187756] xhci-pci-renesas 0000:04:00.0: xHCI Host Controller
[ 11.187870] xhci-pci-renesas 0000:04:00.0: new USB bus registered, assigned bus number 7
[ 11.193261] xhci-pci-renesas 0000:04:00.0: hcc params 0x014051cf hci version 0x100 quirks 0x0000000100000010
[ 11.194806] xhci-pci-renesas 0000:04:00.0: xHCI Host Controller
[ 11.196601] xhci-pci-renesas 0000:04:00.0: new USB bus registered, assigned bus number 8
[ 11.196613] xhci-pci-renesas 0000:04:00.0: Host supports USB 3.0 SuperSpeed
[ 11.196927] usb usb7: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.13
[ 11.196931] usb usb7: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[ 11.196935] usb usb7: Product: xHCI Host Controller
[ 11.196938] usb usb7: Manufacturer: Linux 6.13.3-aosc-main xhci-hcd
[ 11.196941] usb usb7: SerialNumber: 0000:04:00.0
[ 11.199598] hub 7-0:1.0: USB hub found
[ 11.199630] hub 7-0:1.0: 4 ports detected
...
[ 11.439561] usb 7-2: new high-speed USB device number 2 using xhci-pci-renesas
[ 11.568361] usb 7-2: New USB device found, idVendor=1532, idProduct=0114, bcdDevice= 1.00
[ 11.568369] usb 7-2: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 11.568372] usb 7-2: Product: DeathStalker Ultimate
[ 11.568376] usb 7-2: Manufacturer: Razer
[ 11.600474] input: Razer DeathStalker Ultimate as /devices/pci0000:00/0000:00:0e.0/0000:04:00.0/usb7/7-2/7-2:1.0/0003:1532:0114.0001/input/input12
[ 11.600686] hid-generic 0003:1532:0114.0001: input,hidraw0: USB HID v1.11 Mouse [Razer DeathStalker Ultimate] on usb-0000:04:00.0-2/input0
[ 11.601137] input: Razer DeathStalker Ultimate Keyboard as /devices/pci0000:00/0000:00:0e.0/0000:04:00.0/usb7/7-2/7-2:1.1/0003:1532:0114.0002/input/input13
[ 11.652148] input: Razer DeathStalker Ultimate as /devices/pci0000:00/0000:00:0e.0/0000:04:00.0/usb7/7-2/7-2:1.1/0003:1532:0114.0002/input/input14
[ 11.652409] hid-generic 0003:1532:0114.0002: input,hidraw1: USB HID v1.11 Keyboard [Razer DeathStalker Ultimate] on usb-0000:04:00.0-2/input1
[ 11.653054] input: Razer DeathStalker Ultimate as /devices/pci0000:00/0000:00:0e.0/0000:04:00.0/usb7/7-2/7-2:1.2/0003:1532:0114.0003/input/input15
[ 11.703768] hid-generic 0003:1532:0114.0003: input,hidraw2: USB HID v1.11 Keyboard [Razer DeathStalker Ultimate] on usb-0000:04:00.0-2/input2
Martin Wilck [Wed, 26 Feb 2025 13:59:44 +0000 (14:59 +0100)]
fix(iscsi): make sure services are shut down when switching root
When systemd prepares switching root, it starts 'initrd-cleanup.service',
which runs 'systemctl --no-block isolate initrd-switch-root.target'.
This will stop all units on which initrd-switch-root.target does not
depend, including iscsid.service and iscsiuio.service. But systemd
doesn't guarantee a time ordering in this case. It can happen that
systemd switches root (i.e. restarts itself on the new root) before
iscsiuio is actually stopped, or at least before PID 1 receives
the notification that it has stopped. In this case, it considers
iscsiuio still running, and will not start it later in the boot
sequence when iscsid is coming up.
A typical log excerpt with systemd.log_level=debug looks like this:
When iscsid is started later on in the real root, it resets all existing iSCSI
connections, causing the root FS to come offline. In iSCSI offload scenarios
if iscsiuio is already running, it will re-establish the session after a few
seconds. But if iscsiuio has not been started at this point in time, it can't
be loaded any more from the root FS, and booting fails.
To avoid this problem, add "Conflicts" and a "Before" dependencies against
initrd-cleanup.service to the iSCSI service units.
See also https://github.com/systemd/systemd/issues/3436
Martin Wilck [Wed, 26 Feb 2025 13:54:51 +0000 (14:54 +0100)]
fix(iscsi): don't require network setup for qedi
This adds the logic of cc2c48a ("fix(iscsi): don't require network setup
for bnx2i") for the qedi iSCSI offload driver. Testing has shown
that for qedi, network setup in the initrd is even more superfluous
as it is for bnx2i. qedi devices are usually separate PCI functions
that don't show up as ethernet interfaces at all.
Benjamin Drung [Fri, 21 Feb 2025 22:49:04 +0000 (23:49 +0100)]
fix(systemd-sysusers): silence "Creating " on stderr
dracut prints 20 lines when creating users and groups even with
`--quiet` option. Sample output:
```
Creating group 'nobody' with GID 65534.
Creating group 'audio' with GID 997.
Creating group 'disk' with GID 995.
Creating group 'input' with GID 994.
Creating group 'kmem' with GID 993.
Creating group 'kvm' with GID 992.
Creating group 'lp' with GID 991.
Creating group 'optical' with GID 990.
Creating group 'render' with GID 989.
Creating group 'sgx' with GID 988.
Creating group 'storage' with GID 987.
Creating group 'tty' with GID 5.
Creating group 'uucp' with GID 986.
Creating group 'video' with GID 985.
Creating group 'users' with GID 984.
Creating group 'systemd-journal' with GID 983.
Creating user 'root' (Super User) with UID 0 and GID 0.
Creating user 'nobody' (Kernel Overflow User) with UID 65534 and GID 65534.
Creating group 'nobody' with GID 65534.
Creating group 'audio' with GID 997.
```
Filter "Creating " messages from stderr, but keep the other messages on
stderr and all messages on stdout untouched.
Fixes: https://github.com/dracut-ng/dracut-ng/issues/1195 Fixes: f3dacc013d90 ("feat(systemd-sysusers): run systemd-sysusers as part of the build process")
You-Sheng Yang [Wed, 22 Jan 2025 15:37:53 +0000 (23:37 +0800)]
fix(dracut-install): install compressed blobs that match wildcard fwpath
dracut-install tries to invoke `glob()` with full path expanded from
"intel/ish/ish_*.bin", but while all the binaries were compressed, this
matches no file and none of the custom ISH firmware blobs will be
installed.
Closes: #1150
Bug-Ubuntu: https://bugs.launchpad.net/bugs/2095518 Signed-off-by: You-Sheng Yang <vicamo.yang@canonical.com>