Recent meson versions include the directory name in the target name,
so there is no conflict for files with the same name in different
directories. But at least with meson-0.49.2 in buster we have conflict
with sysusers.d/systemd.conf.
This doesn't matter too much, but makes things a bit more consistent.
A minor advantage is that the file is not a configuration file for meson
anymore, so:
a) It is not built unless pulled in by another target. Since
we don't usually build man pages by default, this saves a tiny
amount of work.
b) When the .in file is updated, meson does not reconfigure everything,
but just rebuilds the dependent targets.
Now that the conversion is finished, time for benchmarking:
a full build with default settings (and -Dstandalonebinaries=true), yields
before this pull request: 1687 targets, 148.13s user 35.17s system 317% cpu 57.697 total
with the full pull request: 1714 targets, 143.07s user 27.87s system 314% cpu 54.369 total
The difference doesn't seem significant. Partial rebuilds might be faster as
mentioned before.
We had two big 'configuration_data' objects in meson config. (There are in fact
more. On is added in this series, and there's one for efi… But those others
have a handful variables only for specific purposes and don't matter). The two
sets are 'conf' and 'substs', and were inherited from the original autotools
system. In the past there was even a third set ('m4_defines'), but @yuwata
removed it in 348b44372f36010d48d9a7dda14ef67155753a71. And those two/three
systems had very similar data, but with different variable names, because of
historical reasons. They also used subtly different quoting (.set()
vs. .set10() vs. .set_quoted()), which was required because the templating
engines were not flexible enough. This meants we had more work when changing
things, and we needed to search for different variable names, etc.
With a more flexible templating engine we can do with just one
configuration_data object.
One stanza had "if install_sysconfdir_samples", while the other
"if install_sysconfdir", which looks like a mistake.
install_sysconfdir_samples is now used for both.
The naming of variables is very inconsistent. I tried to use more
modern style naming (UNDERSCORED_TITLE_CASE), but I didn't change existing
names too much. Only SYSTEM_DATA_UNIT_PATH is renamed to SYSTEM_DATA_UNIT_DIR
to match SYSTEM_CONFIG_UNIT_DIR.
meson: use conf configuration_data object to generate test-sysusers.sh
I wanted to use jinja2 templating here too, but it's hard to get right:
custom_target() strips the executable bit by default (unlike configure_file
apparently). custom_target() has install_mode setting, but it was only added
in meson-0.47, so it can't be used while we support 0.46. And without the
executable bit the test is not invoked properly. For example, "root-unittests"
in the debian package calls test-* after installation, so the executable bit
there is necessary. It would be possible to adjust the file mode after the
fact, but it would make things more complicated.
So let's use the native meson substitutions here. We don't need anything more
fancy.
meson: use custom configuration_data() object for man/man and man/html helpers
I want to stop using 'substs'. But in this case, configure_file() is nicer
than custom_target(), because it causes meson to immediately generate the
helpers after configuration, so it's possible to do
'meson build && build/man/man ...', without building anything first.
We only substitute one variable here, so let's use a custom configuration_data()
object.
m4 was hugely popular in the past, because autotools, automake, flex, bison and
many other things used it. But nowadays it much less popular, and might not even
be installed in the buildroot. (m4 is small, so it doesn't make a big difference.)
(FWIW, Fedora dropped make from the buildroot now,
https://fedoraproject.org/wiki/Changes/Remove_make_from_BuildRoot. I think it's
reasonable to assume that m4 will be dropped at some point too.)
The main reason to drop m4 is that the syntax is not very nice, and we should
minimize the number of different syntaxes that we use. We still have two
(configure_file() with @FOO@ and jinja2 templates with {{foo}} and the
pythonesque conditional expressions), but at least we don't need m4 (with
m4_dnl and `quotes').
Jinja2 inserts an empty line after the first macro body, which I don't know how
to get rid of. Only the first macro causes problems: the other ones don't have
conditional statements at the end and the issue does not occur. As a work-around
I moved ProtectHostname to the end of the first macro.
Output is identical, except for horizontal whitespace and change in position of
ProtectHostname.
HAVE_SMACK_RUN_LABEL was dropped back in 348b44372f36010d48d9a7dda14ef67155753a71,
so one line in etc.conf was not rendered as expected ;(
Checking if names are defined is paying for itself!
units: strip out the developer comment in .service unit again
The comment talks about upstream development steps and doesn't make
sense for users. We used special '## ' syntax to strip it out during
build, but it got inadvertently reformatted as a normal comment
in 3982becc92197b920d86f03c3c52ae085e26ca60.
m4 was nice in '85, but the syntax feels a bit dated. Since we use python for
meson, let's use a popular python templating engine to replace some m4 usage.
A little nicety is that typos are caught:
FAILED: sysusers.d/systemd-remote.conf
/usr/bin/meson --internal exe --capture sysusers.d/systemd-remote.conf -- /home/zbyszek/src/systemd/tools/meson-render-jinja2.py config.h ../sysusers.d/systemd-remote.conf.j2
Traceback (most recent call last):
File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 28, in <module>
print(render(sys.argv[2], defines))
File "/home/zbyszek/src/systemd/tools/meson-render-jinja2.py", line 24, in render
return template.render(defines)
File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 1090, in render
self.environment.handle_exception()
File "/usr/lib/python3.9/site-packages/jinja2/environment.py", line 832, in handle_exception
reraise(*rewrite_traceback_stack(source=source))
File "/usr/lib/python3.9/site-packages/jinja2/_compat.py", line 28, in reraise
raise value.with_traceback(tb)
File "<template>", line 8, in top-level template code
jinja2.exceptions.UndefinedError: 'HAVE_MICROHTTP' is undefined
Yu Watanabe [Fri, 14 May 2021 06:58:15 +0000 (15:58 +0900)]
network: fix Link reference counter issue
Previously, when link_new() fails, `link_unref()` was called, so,
`Manager::links` may become dirty.
This introduces `link_drop_or_unref()` and it will be called on
failure.
Yu Watanabe [Mon, 17 May 2021 06:23:57 +0000 (15:23 +0900)]
network: fix possible message counter double decrement
When after_configure() for a request fails, then the request is not
removed from the queue at that time, and the link enters to failed
state. After that, if the link is reconfigured or its carrier is lost,
then the request is dropped from the queue, and the message_counter is
decreased. However, the counter is already or will be also decreased
when the corresponding netlink reply is received.
So, the counter is decreased twice.
Anita Zhang [Tue, 18 May 2021 08:23:36 +0000 (01:23 -0700)]
man: only the system manager does not pass vars in ManagerEnvironment=
User managers always pass their environment on to their children.
Make that clear in the description of ManagerEnvironment= which
states that none of those args will get passed to child processes of
service managers.
Olle Lundberg [Mon, 17 May 2021 14:20:23 +0000 (16:20 +0200)]
nspawn: bring back the word `may` in error text
In the change set 6c045a999800c62368470938307951bb669f5afc the error
text for the old flag `--private-users-chown` was repurposed for the
new flag `--private-users-ownership=own` and while doing so the word
`may` was dropped leading to a grammatically incorrect error text.
network:dhcp4_server_configure returns if not able to get timezone
When /etc/localtime is a symbolic link pointing to another symbolic
link, get_timezone will return -EINVAL instead of the timezone.
This issue can cause systemd-networkd DHCPServer to fail.
Instead of returning failure, log a warning indicating that that
the timezone will not be sent.
Sebastian Blunt [Sat, 15 May 2021 03:39:54 +0000 (20:39 -0700)]
Add crypttab option silent
Adds a crypttab option 'silent' that enables the AskPasswordFlag
ASK_PASSWORD_SILENT. This allows usage of systemd-cryptsetup to default
to silent mode, rather than requiring the user to press tab every time.
Dan Streetman [Fri, 30 Apr 2021 00:21:10 +0000 (20:21 -0400)]
test: combine stdout/stderr from failed test
Printing stdout and stderr from a failed test makes it harder to
interpret what the specific problem was; instead let's print out
the lines in order as we got them when the test was run
Also save failed test output to file if ARTIFACT_DIRECTORY is defined
resolved: fix braino with reference counting and linked lists
In 0e0fd08fc832b8f42e567d722d388eba086da5ff I added reference counts to keep
track of the DnsQueryCandidate objects. Unfortunately, dns_query_unref_candidates()
was written as
while (q->candidates)
dns_query_candidate_unref(q->candidates);
i.e. it would keep dropping the reference count as many times as needed for it
to hit 0, making the patch less than fully effective.
dns_query_unref_candidates() is renamed to dns_query_detach_candidates() and
changed to drop exactly one reference from each of the linked candidates.
Example failure:
==463== Invalid read of size 8
==463== at 0x419C93: dns_query_candidate_go (resolved-dns-query.c:159)
==463== by 0x41A143: dns_query_candidate_notify (resolved-dns-query.c:304)
==463== by 0x434BD6: dns_transaction_complete (resolved-dns-transaction.c:437)
==463== by 0x436A0F: dns_transaction_process_dnssec (resolved-dns-transaction.c:976)
==463== by 0x4378C1: dns_transaction_process_reply (resolved-dns-transaction.c:1387)
==463== by 0x437CE9: on_dns_packet (resolved-dns-transaction.c:1444)
==463== by 0x4B2DC9B: source_dispatch (sd-event.c:3512)
==463== by 0x4B2FB1F: sd_event_dispatch (sd-event.c:4077)
==463== by 0x4B2FFFA: sd_event_run (sd-event.c:4138)
==463== by 0x4B301D6: sd_event_loop (sd-event.c:4159)
==463== by 0x464A24: run (resolved.c:92)
==463== by 0x464B3C: main (resolved.c:99)
==463== Address 0x5f409d0 is 32 bytes inside a block of size 72 free'd
==463== at 0x48410E4: free (vg_replace_malloc.c:755)
==463== by 0x418EDF: mfree (alloc-util.h:48)
==463== by 0x4197E8: dns_query_candidate_free (resolved-dns-query.c:67)
==463== by 0x4198B7: dns_query_candidate_unref (resolved-dns-query.c:70)
==463== by 0x41A2E3: dns_query_unref_candidates (resolved-dns-query.c:337)
==463== by 0x41C5FE: dns_query_cname_redirect (resolved-dns-query.c:1028)
==463== by 0x41CA04: dns_query_process_cname_one (resolved-dns-query.c:1128)
==463== by 0x41CA80: dns_query_process_cname_many (resolved-dns-query.c:1157)
==463== by 0x40C0BD: bus_method_resolve_hostname_complete (resolved-bus.c:198)
==463== by 0x41B312: dns_query_complete (resolved-dns-query.c:562)
==463== by 0x41C1AC: dns_query_accept (resolved-dns-query.c:922)
==463== by 0x41C2C4: dns_query_ready (resolved-dns-query.c:955)
==463== by 0x41A162: dns_query_candidate_notify (resolved-dns-query.c:314)
==463== by 0x434BD6: dns_transaction_complete (resolved-dns-transaction.c:437)
==463== by 0x438995: dns_transaction_prepare (resolved-dns-transaction.c:1728)
==463== by 0x43921D: dns_transaction_go (resolved-dns-transaction.c:1928)
==463== by 0x419C7C: dns_query_candidate_go (resolved-dns-query.c:163)
==463== by 0x41A143: dns_query_candidate_notify (resolved-dns-query.c:304)
==463== by 0x434BD6: dns_transaction_complete (resolved-dns-transaction.c:437)
==463== by 0x436A0F: dns_transaction_process_dnssec (resolved-dns-transaction.c:976)
==463== by 0x4378C1: dns_transaction_process_reply (resolved-dns-transaction.c:1387)
==463== by 0x437CE9: on_dns_packet (resolved-dns-transaction.c:1444)
==463== by 0x4B2DC9B: source_dispatch (sd-event.c:3512)
==463== by 0x4B2FB1F: sd_event_dispatch (sd-event.c:4077)
==463== by 0x4B2FFFA: sd_event_run (sd-event.c:4138)
==463== by 0x4B301D6: sd_event_loop (sd-event.c:4159)
==463== by 0x464A24: run (resolved.c:92)
==463== by 0x464B3C: main (resolved.c:99)
==463== Block was alloc'd at
==463== at 0x483E86F: malloc (vg_replace_malloc.c:380)
==463== by 0x418F81: malloc_multiply (alloc-util.h:96)
==463== by 0x419378: dns_query_candidate_new (resolved-dns-query.c:23)
==463== by 0x41B42C: dns_query_add_candidate (resolved-dns-query.c:582)
==463== by 0x41BB7A: dns_query_go (resolved-dns-query.c:762)
==463== by 0x40CE3A: bus_method_resolve_hostname (resolved-bus.c:464)
==463== by 0x4A84B86: method_callbacks_run (bus-objects.c:414)
==463== by 0x4A87961: object_find_and_run (bus-objects.c:1323)
==463== by 0x4A87FEE: bus_process_object (bus-objects.c:1443)
==463== by 0x4AA3434: process_message (sd-bus.c:2964)
==463== by 0x4AA3623: process_running (sd-bus.c:3006)
==463== by 0x4AA4110: bus_process_internal (sd-bus.c:3226)
==463== by 0x4AA41EF: sd_bus_process (sd-bus.c:3253)
==463== by 0x4AA5343: io_callback (sd-bus.c:3604)
==463== by 0x4B2DC9B: source_dispatch (sd-event.c:3512)
==463== by 0x4B2FB1F: sd_event_dispatch (sd-event.c:4077)
==463== by 0x4B2FFFA: sd_event_run (sd-event.c:4138)
==463== by 0x4B301D6: sd_event_loop (sd-event.c:4159)
==463== by 0x464A24: run (resolved.c:92)
==463== by 0x464B3C: main (resolved.c:99)
The patch itself seems OK, but it exposes a bug in lxml or libxml2-2.9.12 which
was just released. This is being resolved in
https://gitlab.gnome.org/GNOME/libxml2/-/issues/255, but it might be while. So
let's revert this for now to unbreak our CI.
meson: revert the change to unquote commands in add_install_script
Old meson fails with:
Element not a string: [<Holder: <ExternalProgram 'sh' -> ['/bin/sh']>>, '-c', 'test -n "$DESTDIR" || /bin/journalctl --update-catalog']
I'm doing it as a revert so that it's easy to undo the revert when we require
newer meson. The effect is not so bad, maybe a dozen or so lines about finding
'sh'.
meson: call find_program() once and reuse the variable everywhere
Meson 0.58 has gotten quite bad with emitting a message every time
a quoted command is used:
Program /home/zbyszek/src/systemd-work/tools/meson-make-symlink.sh found: YES (/home/zbyszek/src/systemd-work/tools/meson-make-symlink.sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program sh found: YES (/usr/bin/sh)
Program xsltproc found: YES (/usr/bin/xsltproc)
Configuring custom-entities.ent using configuration
Message: Skipping bootctl.1 because ENABLE_EFI is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Message: Skipping journal-remote.conf.5 because HAVE_MICROHTTPD is false
Message: Skipping journal-upload.conf.5 because HAVE_MICROHTTPD is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Message: Skipping loader.conf.5 because ENABLE_EFI is false
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
Program ln found: YES (/usr/bin/ln)
...
Let's suffer one message only for each command. Hopefully we can silence
even this when https://github.com/mesonbuild/meson/issues/8642 is
resolved.
man: reorder items in [DHCPv4] and [DHCPv6] sections
The settings were listen in a completely random order, also different
between the v4 and v6 sections. Order by "options sent", "options received",
"communication settings" in both sections.
Also minor formatting changes are done, e.g. "=" is added in various places.
Yu Watanabe [Wed, 12 May 2021 15:22:14 +0000 (00:22 +0900)]
networkctl: introduce --json option for "status" and "list" commands
When `--json` option is specified, "status" and "list" commands gives
the same information, as originally "list" just gives partial
information of "status" in different format.
Anita Zhang [Thu, 13 May 2021 08:17:51 +0000 (01:17 -0700)]
run: update checks to allow running with a user's bus
systemd-run is documented to as being able to connect and run on a
specific user bus with "--user --machine=lennart@.host" arguments.
This PR updates some logic that prevented this from working.
I occasionally do 'build/man/man systemd.directives' when working on man pages,
and it's annoying slow. By paralellizing the parsing of xml, we can make it a
bit faster.
This is still rather innefficient. Only the parsing part is serialized, xml is
still produced serially at the end, which is hard to avoid.
$ ninja -C build man/systemd.directives.xml
before:
8.20s user 0.21s system 99% cpu 8.460 total
8.33s user 0.18s system 98% cpu 8.619 total
8.72s user 0.19s system 98% cpu 9.019 total
after:
13.99s user 0.73s system 345% cpu 4.262 total
14.15s user 0.35s system 348% cpu 4.161 total
14.33s user 0.35s system 339% cpu 4.321 total
I.e. it uses almost twice as much cpu, but cuts the wallclock time down (on a
2-core/4-thread cpu) to about half too, which is an overall win if you're just
trying to render the man page.
The change from list and .append() to set and .add() is something that could
have been done before too, but it's noticable now. It cuts down on the
serialization/deserialization time (about .2s).
Simon Watts [Wed, 12 May 2021 21:22:39 +0000 (22:22 +0100)]
Add Microsoft Pro Intellimouse MOUSE_DPI to hwdb
- Internet specifications give 1600 DPI @ 1000Hz for this sensor
- Confirmed experimentally via `mouse-dpi-tool`
- vid, pid, and name match string from `mouse-dpi-tool`
resolved: be more careful with weird links with low MTUs
Apparently CAN links will show up in rtnetlink with very low MTUs. We
shouldn't consider them relevant if no IP is spoken over them, since
these MTUs are irrelevant for us then.
Hence, let's check if there's an address assigned to the link before
considering its MTU.
As additional safety net filter out MTUs smaller than the minimum DNS
packet size, too.
Finally, in case we don't find any suitable interface MTU, let's default
to 1500 as the generic Ethernet MTU.
This was a copy/paste mistae apparently, there's not "try_authtok" and
this was supposed to copy what Fedora uses, which uses "use_authtok"
correctly. Hence adjust this.