pid1: disable printk ratelimit in early boot

We have the problem that many early boot or late shutdown issues are harder
to solve than they could be because we have no logs. When journald is not
running, messages are redirected to /dev/kmsg. It is also the time when many
things happen in a rapid succession, so we tend to hit the kernel printk
ratelimit fairly reliably. The end result is that we get no logs from the time
where they would be most useful. Thus let's disable the kernels ratelimit.

Once the system is up and running, the ratelimit is not a problem. But during
normal runtime, things also log to journald, and not to /dev/kmsg, so the
ratelimit is not useful. Hence, there doesn't seem to be much point in trying
to restore the ratelimit after boot is finished and journald is up and running.

See kernel's commit 750afe7babd117daabebf4855da18e4418ea845e for the
description of the kenrel interface. Our setting has lower precedence than
explicit configuration on the kenrel command line.

diff --git a/src/core/main.c b/src/core/main.c
index 4633184b79ad8b75e7633e0f7ff7b5966b92ab8b..2e95edfbd8691648dadad93677acc00d0c0eaa09 100644 (file)
--- a/src/core/main.c
+++ b/src/core/main.c
@@ -2435,6 +2435,8 @@ int main(int argc, char *argv[]) {
                                  * available, and it previously wasn't. */
                                 log_open();
 
+                                disable_printk_ratelimit();
+
                                 r = initialize_security(
                                                 &loaded_policy,
                                                 &security_start_timestamp,

diff --git a/src/core/manager.c b/src/core/manager.c
index dac4248356b2b02731520e47285af1fa6c99139a..600718e4bf1b78757c73f32f389ef25e3c029475 100644 (file)
--- a/src/core/manager.c
+++ b/src/core/manager.c
@@ -72,6 +72,7 @@
 #include "string-util.h"
 #include "strv.h"
 #include "strxcpyx.h"
+#include "sysctl-util.h"
 #include "syslog-util.h"
 #include "terminal-util.h"
 #include "time-util.h"
@@ -4024,6 +4025,19 @@ static bool manager_journal_is_running(Manager *m) {
         return true;
 }
 
+void disable_printk_ratelimit(void) {
+        /* Disable kernel's printk ratelimit.
+         *
+         * Logging to /dev/kmsg is most useful during early boot and shutdown, where normal logging
+         * mechanisms are not available. The semantics of this sysctl are such that any kernel command-line
+         * setting takes precedence. */
+        int r;
+
+        r = sysctl_write("kernel/printk_devkmsg", "on");
+        if (r < 0)
+                log_debug_errno(r, "Failed to set sysctl kernel.printk_devkmsg=on: %m");
+}
+
 void manager_recheck_journal(Manager *m) {
 
         assert(m);

diff --git a/src/core/manager.h b/src/core/manager.h
index 308ee013bda9d38b7a431ab5094ebf54db6b81c4..424bf71704143bd497256eb8e03d6bb43d06b593 100644 (file)
--- a/src/core/manager.h
+++ b/src/core/manager.h
@@ -499,6 +499,7 @@ bool manager_unit_inactive_or_pending(Manager *m, const char *name);
 
 void manager_check_finished(Manager *m);
 
+void disable_printk_ratelimit(void);
 void manager_recheck_dbus(Manager *m);
 void manager_recheck_journal(Manager *m);