Lucas De Marchi [Wed, 16 Feb 2022 07:10:58 +0000 (23:10 -0800)]
libkmod: Add lookup from module name
Slightly different than kmod_module_new_from_lookup(): it doesn't
consider aliases, only module names. This is useful for cases we want to
force a tool to handle something as the module name, without trying to
interpret it as an alias.
Lucas De Marchi [Wed, 16 Feb 2022 03:12:05 +0000 (19:12 -0800)]
libkmod: Add helper function to iterate lookup options
The CHECK_ERR_AND_FINISH macro with conditional code flow changes has
been a source of bugs. Get rid of it replacing with a helper function
to iterate an array of lookup functions. This helper may also be useful
in future to create different lookup APIs in libkmod.
Lucas De Marchi [Tue, 15 Feb 2022 22:09:14 +0000 (14:09 -0800)]
libkmod-module: Fix return code for kmod_module_new_from_lookup()
When kmod_module_new_from_lookup() resolves to an alias, `err` will be
set to a positive value from the lookup function. Do not return a
positive value to follow the behavior when it matches a module name
and the documentation.
Lucas De Marchi [Tue, 15 Feb 2022 22:05:13 +0000 (14:05 -0800)]
test-initstate: Check for negative value on error
Documentation says kmod_module_new_from_lookup() returns < 0 on error
and 0 otherwise. There are bugs in libkmod however making it return
a positive value in some situations, that need to be fixed. However
it's best to check for the error explicitly like is done in the rest
of the library to avoid this kind of issues.
Lucas De Marchi [Thu, 10 Feb 2022 02:07:03 +0000 (18:07 -0800)]
depmod: Stop opening modules.modinfo once per module
Since the addition of modules.aliases.bin, depmod has to open that
index multiple times and parse it over and over again:
$ sudo strace -e openat ./tools/depmod 2>&1 | grep modules.builtin.modinfo | wc -l
299
$ time sudo ./tools/depmod
real 0m7.814s
user 0m7.571s
sys 0m0.237s
Rework the logic in depmod so it does everything: open, read and parse. The
format is very straightforward and we don't need to keep it in a data structure
since we only want to add the result to a index. New output:
$ sudo strace -e openat ./tools/depmod 2>&1 | grep modules.builtin.modinfo | wc -l
1
$ time sudo ./tools/depmod
real 0m7.663s
user 0m7.516s
sys 0m0.139s
From the kernel we get both modules.builtin and modules.builtin.modinfo.
depmod generates modules.builtin.bin and modules.builtin.alias.bin
from them respectively. modules.bultin is not going away: it's not
deprecated by the new index added. So, let's just stop duplicating the
information inside modules.builtin.alias.bin and just use the other
index.
Michal Suchanek [Wed, 18 Aug 2021 20:52:00 +0000 (22:52 +0200)]
libkmod: Set builtin to no when module is created from path.
A recent bug report showed that modinfo doesn't give the signature
information for certain modules, and it turned out to happen only on
the modules that are built-in on the running kernel; then modinfo
skips the signature check, as if the target module file never exists.
The behavior is, however, inconsistent when modinfo is performed for
external modules (no matter which kernel version is) and the module
file path is explicitly given by a command-line argument, which
guarantees the presence of the module file itself.
Luis Chamberlain [Tue, 10 Aug 2021 05:16:00 +0000 (22:16 -0700)]
libkmod: add a library notice log level print
When you use pass the -v argument to modprobe we bump
the log level from the default modprobe log level of
LOG_WARNING (4) to LOG_NOTICE (5), however the library
only has avaiable to print:
LOG_INFO (6) however is too high of a level for it to be
effective at printing anything when modprobe -v is passed.
And so the only way in which modprobe -v can trigger the
library to print a verbose message is to use ERR() but that
always prints something and we don't want that in some
situations.
We need to add a new log level macro which uses LOG_NOTICE (5)
for a "normal but significant condition" which users and developers
can use to look underneath the hood to confirm if a situation is
happening.
Signed-off-by: Luis Chamberlain <mcgrof@kernel.org> Signed-off-by: Lucas De Marchi <lucas.demarchi@intel.com>
Yauheni Kaliuta [Tue, 8 Jun 2021 06:29:22 +0000 (09:29 +0300)]
libkmod-module: check "new_from_name" return value in get_builtin
kmod_module_new_from_name() may fail and return error value. It is
handled properly across the code, but in this particular place the
check is missing.
Lucas De Marchi [Tue, 11 May 2021 16:48:04 +0000 (09:48 -0700)]
depmod: fix modules.builtin.alias.bin output
Due to wrong documentation on kmod_module_get_info() we ended up
checking for 0 as return. Check for > 0 to decided if we want to write
the index to the file, otherwise we would output a 0-sized index on
success.
Lucas De Marchi [Wed, 10 Mar 2021 16:33:01 +0000 (08:33 -0800)]
libkmod-config: more chars allowed as space
Recently in a discussion I noticed that kernel allows more chars to be
considered as space in the kernel command line. Switch to the equivalent
of isspace() instead of considering only ' '.
Lucas De Marchi [Fri, 12 Feb 2021 09:45:23 +0000 (01:45 -0800)]
testsuite: allow to re-use single function for tests
Add a new DEFINE_TEST_WITH_FUNC() that accepts the function
alongside the test name. This will allow us to share a single function
for different tests.
However, even though we could blame grub for doing that, the kernel
happily accepts and re-quotes it when the module is built-in.
So, it's better if kmod also understands it this way and does the same.
Here we basically add additional code to un-mangle it, moving the quote
in way that is acceptable to pass through init_module(). Note that the
interface [f]init_module() gives us mandates the quote to be part of the
value: the module name is not passed and the options are separated by
space.
Lucas De Marchi [Fri, 12 Feb 2021 09:45:21 +0000 (01:45 -0800)]
libkmod-config: revamp kcmdline parsing into a state machine
The handling of spaces and quotes is becoming hard to maintain. Convert
the parser into a state machine so we can check all the states. This
should make it easier to fix a corner case we have right now:
The kernel also accepts a quote before the module name instead of the
value. But this additional is left for later. This is purely an
algorithm change with no behavior change.
Lucas De Marchi [Sat, 30 Jan 2021 02:28:38 +0000 (18:28 -0800)]
testsuite: compress modules if feature is enabled
Since the output needs to be the same, regardless if the module is
compressed, change populate-modules.sh to conditionally compress the
module if that feature is enabled.
This way we can execute the tests with any build-time configuration and
it should still pass.
Suggested-by: Michal Suchánek <msuchanek@suse.de> Reviewed-by: Michal Suchánek <msuchanek@suse.de> Tested-by: Michal Suchánek <msuchanek@suse.de> Reviewed-by: Petr Vorel <pvorel@suse.cz>
Lucas De Marchi [Thu, 21 Jan 2021 02:50:33 +0000 (18:50 -0800)]
README: make github mirror official
For some time I've been maintaining a read-only mirror on github.
I think it's time to allow patches flowing from there besides the
mailing list: I created a new org to host the project:
Lucas De Marchi [Tue, 19 Jan 2021 02:13:25 +0000 (18:13 -0800)]
depmod: fix precedence order
Configuration in /etc should have higher prio than /run.
Given how rarely configuration in /run is used with depmod, this is
likely not to cause any problems, even if it's a change in behavior.
Marco d'Itri [Fri, 8 Jan 2021 04:17:48 +0000 (20:17 -0800)]
Fix "modinfo -F always shows name for built-ins"
Bug reported by Ben Hutchings <ben@decadent.org.uk>:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=970871
Now that the kernel provides module information for potentially
modular code that's actually built-in, it's possible to query these
built-ins with "modinfo -F". However, this doesn't work quite right:
Marius Bakke [Sat, 1 Aug 2020 16:02:21 +0000 (18:02 +0200)]
testsuite: Add facility to skip tests.
The Makefile helpfully warns that some tests will fail when
--sysconfdir != /etc, but there are no provisions to easily disable
those. This commit provides an escape hatch.
[ Lucas: add comment detailing the purpose of the field ]
Lucas De Marchi [Fri, 18 Dec 2020 16:02:07 +0000 (08:02 -0800)]
depmod: unconditionally write builtin.alias.bin
The file is always created and unless we return an error, the temporary
file is renamed to its final destination. All other places write the
index without checking if the index is empty, so just do the same.
Yauheni Kaliuta [Sun, 29 Nov 2020 16:47:37 +0000 (18:47 +0200)]
libkmod: kmod_log_null: qualify ctx argument as const
kmod_log_null() does not change ctx (does nothing).
Fix warnings
In file included from libkmod/libkmod-index.c:33:
libkmod/libkmod-index.c: In function ‘index_mm_open’:
libkmod/libkmod-index.c:757:6: warning: passing argument 1 of ‘kmod_log_null’ discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
757 | DBG(ctx, "file=%s\n", filename);
Yauheni Kaliuta [Sun, 29 Nov 2020 16:47:35 +0000 (18:47 +0200)]
libkmod: kmod_builtin_get_modinfo: free modinfo on error
The function allocates array but on building it if get_string()
fails it returns the error leaving the array allocated. The caller
does not care about it in error case either.
I changed the style of the hackargs variable in autogen.sh to multiline
because said line was becoming a bit long with the new --with-zstd arg
added.
A previous version of this patch has been running on my two Arch Linux
installations (with an accompanying mkinitcpio patch) for several months
over many kernel updates without any issues.
Any additional testing and/or patch review would of course be appreciated.
Lucas De Marchi [Tue, 10 Mar 2020 05:00:27 +0000 (22:00 -0700)]
libkmod: fix return error when opening index
When calling kmod_load_resources() we could end up getting a bogus
return value -ENOMEM due to several other reasons, like the index not
existing. Change index_mm_open() to propagate the failure reason so we
can take actions on it or return to the caller.
Lucas De Marchi [Tue, 10 Mar 2020 05:00:26 +0000 (22:00 -0700)]
libkmod: simplify lookup when builtin.modinfo.bin file is missing
When we try to lookup a module and builtin.modinfo.bin is missing, we
would do the right thing because the caller was replacing the return
code with 0 (and the list was not modified).
Make it simpler by allowing the caller to check and differentiate the
errors between module not found and index not found.
Lucas De Marchi [Tue, 10 Mar 2020 08:17:12 +0000 (01:17 -0700)]
testsuite: check for ill-formed kcmdline
Commit ca3bf5d47cec ("iommu/amd: Introduces ivrs_acpihid kernel parameter")
in the kernel introduced an ill-formed kernel parameter, ivrs_acpihid.
The problem is that it may have a dot on the key side:
"ivrs_acpihid[00:14.5]=AMD0020:0". This could potentially trip our
parser of module options, but right now it's working as intended: the
only thing that happens is that after deciding "ivrs_acpihid[00:14" is a
module name, it will fail the underscores() routine and the option will
be ignored from the kmod pov (not kernel's pov since that driver parsers
the kernel command line by itself).
Lucas De Marchi [Mon, 20 Jan 2020 19:15:38 +0000 (16:15 -0300)]
libkmod: reset was_space on second pass
The softdep config parser uses a 2-pass approach to use a single
allocation for all the softdep struct. However "was_space" variable
isn't reset between them. This can lead to a buffer overflow.
Alexey Gladkov [Fri, 8 Nov 2019 17:25:22 +0000 (18:25 +0100)]
Lookup aliases in the modules.builtin.modinfo
New modules.builtin.modinfo duplicates modules.builtin in the built-in
module name search. If it exists, then we can use this file, but if not,
then we need to fallback to the old file.
Alexey Gladkov [Fri, 8 Nov 2019 17:25:20 +0000 (18:25 +0100)]
libkmod: Add parser for modules.builtin.modinfo
The kernel since version v5.2-rc1 exports information about built-in
modules in the modules.builtin.modinfo. Information is stored in
the same format as in the separate modules (null-terminated string
array). The module name is a prefix for each line.
$ tr '\0' '\n' < modules.builtin.modinfo
ext4.softdep=pre: crc32c
ext4.license=GPL
ext4.description=Fourth Extended Filesystem
ext4.author=Remy Card, Stephen Tweedie, Andrew Morton, Andreas Dilger, Theodore Ts'o and others
ext4.alias=fs-ext4
ext4.alias=ext3
ext4.alias=fs-ext3
ext4.alias=ext2
ext4.alias=fs-ext2
md_mod.alias=block-major-9-*
md_mod.alias=md
md_mod.description=MD RAID framework
md_mod.license=GPL
md_mod.parmtype=create_on_open:bool
md_mod.parmtype=start_dirty_degraded:int
...
Fabrice Fontaine [Mon, 18 Nov 2019 21:56:17 +0000 (22:56 +0100)]
Makefile.am: filter -Wl,--no-undefined
Commit 1d14ef82f4a3be741bcdf6b1c6d51ce9dce43567 does not completely fix
the build with python 3.8 as we still get link failure due to
'-z undefs' being ignored by some versions of ld.
Indeed, -z undefs was added by commit 97a232d7335f3bd0231fd9cd39455bde1d563922 in upstream binutils, and this
commit was first present in binutils 2.30.
So any toolchain using binutils version older than that won't have
-z undefs and will build fail on:
Lucas De Marchi [Thu, 7 Nov 2019 16:34:15 +0000 (08:34 -0800)]
travis: remove old compiler failing to build kernel module
This is when building the kernel modules for testsuite:
Makefile:718: Cannot use CONFIG_CC_STACKPROTECTOR_STRONG: -fstack-protector-strong not supported by compiler
gcc: error: unrecognized command line option ‘-fstack-protector-strong’
Just drop gcc 4.8 from running tests. Failure not really related to kmod.
Thomas Petazzoni [Thu, 24 Oct 2019 17:47:10 +0000 (19:47 +0200)]
Do not check for undefined symbols when building the Python modules
kmod's configure.ac uses the -Wl,--no-undefined linker flag to verify
at link time that all symbols of shared libraries are available, and
that there are no undefined symbols.
This make perfect sense for regular shared libraries. However, for
Python extensions, which will be dlopen()ed inside the Python
interpreter, it makes less sense.
Since Python 3.8, there is a change in python-config script and
Python's pkg-config file: it no longer links Python extensions with
the libpython library. See
https://docs.python.org/dev/whatsnew/3.8.html#debug-build-uses-the-same-abi-as-release-build
which states:
On the other hand, pkg-config python3.8 --libs no longer contains
-lpython3.8. C extensions must not be linked to libpython (except on
Android and Cygwin, whose cases are handled by the script); this
change is backward incompatible on purpose. (Contributed by Victor
Stinner in bpo-36721.)
So, when linking the kmod Python extensions, it currently fails with
numerous unresolved symbols, that were previously provided by
libpython:
/home/test/autobuild/run/instance-3/output-1/host/opt/ext-toolchain/bin/../lib/gcc/powerpc64-buildroot-linux-gnu/7.4.0/../../../../powerpc64-buildroot-linux-gnu/bin/ld: libkmod/python/kmod/.libs/list_la-list.o: in function `__Pyx_PyObject_GetAttrStr':
list.c:(.text.__Pyx_PyObject_GetAttrStr+0x48): undefined reference to `PyObject_GetAttr'
/home/test/autobuild/run/instance-3/output-1/host/opt/ext-toolchain/bin/../lib/gcc/powerpc64-buildroot-linux-gnu/7.4.0/../../../../powerpc64-buildroot-linux-gnu/bin/ld: libkmod/python/kmod/.libs/list_la-list.o: in function `__pyx_tp_dealloc_4kmod_4list_ModListItem':
list.c:(.text.__pyx_tp_dealloc_4kmod_4list_ModListItem+0x78): undefined reference to `PyObject_CallFinalizerFromDealloc'
/home/test/autobuild/run/instance-3/output-1/host/opt/ext-toolchain/bin/../lib/gcc/powerpc64-buildroot-linux-gnu/7.4.0/../../../../powerpc64-buildroot-linux-gnu/bin/ld: libkmod/python/kmod/.libs/list_la-list.o: in function `__pyx_tp_dealloc_4kmod_4list_ModList':
list.c:(.text.__pyx_tp_dealloc_4kmod_4list_ModList+0x30): undefined reference to `PyErr_Fetch'
Linking with libpython is no longer recommended: those symbols should
remain unresolved in the Python extensions, as they wil be properly
resolved when the Python extension gets loaded into the Python
interpreter.
Since we want to keep -Wl,--no-undefined globally in kmod, we leave
the configure.ac file unchanged, and instead, specifically in the
LDFLAGS used to build the Python extensions, we override
-Wl,--no-undefined with -Wl,-z,undefs. Ideally, -Wl,--no-undefined is
the same as -Wl,-z,defs, and the effect of these options can be
canceled on the linker command line by a following -Wl,-z,undefs (see
the ld man page for details).
Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com> Cc: Victor Stinner <victor.stinner@gmail.com>
Stefan Strogin [Sun, 19 May 2019 00:42:01 +0000 (03:42 +0300)]
libkmod-signature: use PKCS#7 instead of CMS
Linux uses either PKCS #7 or CMS for signing modules (see
scripts/sign-file.c). CMS is not supported by LibreSSL or older OpenSSL,
so PKCS #7 is used on systems with these libcrypto providers.
CMS and PKCS #7 formats are very similar. CMS is newer but is as much as
possible backward compatible with PKCS #7 [1]. PKCS #7 is supported in
the latest OpenSSL as well as CMS. The fields used for signing kernel
modules are supported both in PKCS #7 and CMS.
For now modinfo uses CMS with no alternative requiring OpenSSL 1.1.0 or
newer.
Use PKCS #7 for parsing module signature information, so that modinfo
could be used both with OpenSSL and LibreSSL.
Adrian Bunk [Wed, 20 Feb 2019 12:22:04 +0000 (14:22 +0200)]
build: Stop using dolt
This does regress "make -12" from 0.7s to 0.9s on my
Coffee Lake machine, but even on slower hardware this
will not amount to a noticable slowdown.
On the other hand using dolt can create problems for
people doing cross-compilation, e.g. Yocto has two
hacks just for dolt in kmod:
https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-kernel/kmod/kmod.inc?id=a17abae00785c76cfffe5381a22fb2c86b982e82
(Lucas: remove leftover entry in Makefile and reformat commit message)
We don't need any symbols from libssl, though. This patch ensures that
we pass 'libcrypto' to pkgconfig rather than 'openssl', getting only the
library that we need:
Yauheni Kaliuta [Fri, 1 Feb 2019 20:20:02 +0000 (22:20 +0200)]
libkmod-signature: implement pkcs7 parsing with openssl
The patch adds data fetching from the PKCS#7 certificate using
openssl library (which is used by scripts/sign-file.c in the linux
kernel to sign modules).
In general the certificate can contain many signatures, but since
kmod (modinfo) supports only one signature at the moment, only first
one is taken.
With the current sign-file.c certificate doesn't contain signer
key's fingerprint, so "serial number" is used for the key id.
Yauheni Kaliuta [Fri, 1 Feb 2019 20:36:41 +0000 (22:36 +0200)]
testsuite: add modinfo pkcs7 signature test
Use the same approach to generate the signed module, like in the
old signature test: just append the pregenerated binary signature
to the module (the signature check will fail).
In case of need of generating correct signature, from the linux
kernel makefiles (certs/Makefile) it could be like:
Yauheni Kaliuta [Fri, 4 Jan 2019 21:41:34 +0000 (23:41 +0200)]
testsuite: factor out fd related parameters to a structure
This is a more abstract implementation of "file descriptor
comparation". With the current implementation the code is full of
conditions based on the descriptor type. It makes sense to
initialize the parameters once based on the descriptor type.
stdout and stderr are handled in almost the same way, but for
monitor descriptor branch, based on the type check is necessary in
some cases.
Since epoll's context now contains pointers to the structures, so no
direct manipulations there.
Most of the patch is just replacing direct buffer manipulations with
the structures' ones.
Yauheni Kaliuta [Fri, 4 Jan 2019 21:41:33 +0000 (23:41 +0200)]
testsuite: track number of descriptors instead of their state
use the number of tracked descriptors to determine the end of the
loop.
This is a preparation for more abstract descriptor comparation
implementation where checking of the descriptor state may be more
expensive than just checking of the local variables.
Lucas De Marchi [Tue, 18 Dec 2018 22:11:58 +0000 (14:11 -0800)]
Add semaphoreci 2.0 configuration
Add another CI as alternative to travis-ci. Test on gcc 6, 7 and 8 on
Ubuntu 18.04. Not sure if this is the best way to define the yml file,
but it works.
The old badge doesn't work. It will be added back later.
Lucas De Marchi [Tue, 18 Dec 2018 23:36:11 +0000 (15:36 -0800)]
testsuite: add support for testing output against regex
Allow to test outputs when they don't match exactly, but should follow
some regex patterns. This can be used when the info we are printing is
randomized or depends on kernel configuration.
Lucas De Marchi [Tue, 18 Dec 2018 21:42:09 +0000 (13:42 -0800)]
testsuite: split out function to compare outputs exactly
Move functionality to compare the exact output to a separate function
and allocate one buffer per output/match pair. This will allow us to
extend this to allow other types of comparisons. Since now we are using
heap-allocated buffer, keep the buffer allocation to the caller, so we
don't have to allocate and free it on every invocation. It also avoids
the different comparison functions to have to deal with it.
Michal Suchanek [Mon, 10 Dec 2018 21:29:34 +0000 (22:29 +0100)]
depmod: shut up gcc insufficinet buffer warning
In a couple of places depmod concatenates the module directory and filename
with snprintf. This can technically overflow creating an unterminated string if
module directory name is long. Use openat instead as is done elsewhere in
depmod. This avoids the snprintf, the extra buffer on stack, and the gcc
warning. It may even fix a corner case when the module direcotry name is just
under PATH_MAX.
[ Lucas: fix up coding style and closing fd on error path ]
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
Michal Suchanek [Mon, 17 Dec 2018 22:46:28 +0000 (23:46 +0100)]
depmod: prevent module dependency files corruption due to parallel invocation.
Depmod does not use unique filename for temporary files. There is no
guarantee the user does not attempt to run mutiple depmod processes in
parallel. If that happens a temporary file might be created by
depmod(1st), truncated by depmod(2nd), and renamed to final name by
depmod(1st) resulting in corrupted file seen by user.
Due to missing mkstempat() this is more complex than it should be.
Adding PID and timestamp to the filename should be reasonably reliable.
Adding O_EXCL as mkstemp does fails creating the file rather than
corrupting existing file.
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
Michal Suchanek [Mon, 10 Dec 2018 21:29:32 +0000 (22:29 +0100)]
depmod: prevent module dependency files missing during depmod invocation
depmod deletes the module dependency files before moving the temporary
files in their place. This results in user seeing no dependency files
while they are updated. Remove the unlink call. The rename call should
suffice to move the new file in place and unlink the old one. It should
also do both atomically so there is no window when no dependency file
exists.
Signed-off-by: Michal Suchanek <msuchanek@suse.de>
Lucas De Marchi [Mon, 17 Dec 2018 17:52:05 +0000 (09:52 -0800)]
Remove bootstrap* scripts
Let's just use autogen.sh, no need for wrapper scripts. Now
`autogen.sh c` uses the same recommended options for developing kmod and
also accepts extra arguments.