tools/rtla: Add remaining support for osnoise actions

The basic functionality came with the consolidation; now hook up the
command line options, and add documentation and tests.

Cc: John Kacur <jkacur@redhat.com>
Cc: Costa Shulyupin <costa.shul@redhat.com>
Link: https://lore.kernel.org/20250907022325.243930-8-crwood@redhat.com
Reviewed-by: Tomas Glozar  <tglozar@redhat.com>
Signed-off-by: Crystal Wood <crwood@redhat.com>
Signed-off-by: Steven Rostedt (Google) <rostedt@goodmis.org>

diff --git a/Documentation/tools/rtla/common_options.rst b/Documentation/tools/rtla/common_options.rst
index 2dc1575210aa1142f40b326fdf1e9fc208443596..77ef35d3f831716b51034ce24c9ea4d23a1469ab 100644 (file)
--- a/Documentation/tools/rtla/common_options.rst
+++ b/Documentation/tools/rtla/common_options.rst
@@ -53,6 +53,67 @@
 **--trace-buffer-size** *kB*
         Set the per-cpu trace buffer size in kB for the tracing output.
 
+**--on-threshold** *action*
+
+        Defines an action to be executed when tracing is stopped on a latency threshold
+        specified by |threshold|.
+
+        Multiple --on-threshold actions may be specified, and they will be executed in
+        the order they are provided. If any action fails, subsequent actions in the list
+        will not be executed.
+
+        Supported actions are:
+
+        - *trace[,file=<filename>]*
+
+          Saves trace output, optionally taking a filename. Alternative to -t/--trace.
+          Note that nlike -t/--trace, specifying this multiple times will result in
+          the trace being saved multiple times.
+
+        - *signal,num=<sig>,pid=<pid>*
+
+          Sends signal to process. "parent" might be specified in place of pid to target
+          the parent process of rtla.
+
+        - *shell,command=<command>*
+
+          Execute shell command.
+
+        - *continue*
+
+          Continue tracing after actions are executed instead of stopping.
+
+        Example:
+
+        $ rtla |tool| |thresharg| 20 --on-threshold trace
+        --on-threshold shell,command="grep ipi_send |tracer|\_trace.txt"
+        --on-threshold signal,num=2,pid=parent
+
+        This will save a trace with the default filename "|tracer|\_trace.txt", print its
+        lines that contain the text "ipi_send" on standard output, and send signal 2
+        (SIGINT) to the parent process.
+
+        Performance Considerations:
+
+        |actionsperf|
+
+**--on-end** *action*
+
+        Defines an action to be executed at the end of tracing.
+
+        Multiple --on-end actions can be specified, and they will be executed in the order
+        they are provided. If any action fails, subsequent actions in the list will not be
+        executed.
+
+        See the documentation for **--on-threshold** for the list of supported actions, with
+        the exception that *continue* has no effect.
+
+        Example:
+
+        $ rtla |tool| -d 5s --on-end trace
+
+        This runs rtla with the default options, and saves trace output at the end.
+
 **-h**, **--help**
 
         Print help menu.

diff --git a/Documentation/tools/rtla/common_osnoise_options.rst b/Documentation/tools/rtla/common_osnoise_options.rst
index d73de2d58f5f3c134eb96763577cb070c720d743..bd3c4f499193952b1129824ece67a3c2c435f5b1 100644 (file)
--- a/Documentation/tools/rtla/common_osnoise_options.rst
+++ b/Documentation/tools/rtla/common_osnoise_options.rst
@@ -1,3 +1,11 @@
+.. |threshold|  replace:: **-a/--auto**, **-s/--stop**, or **-S/--stop-total**
+.. |thresharg|  replace:: -s
+.. |tracer|     replace:: osnoise
+
+.. |actionsperf| replace::
+        Due to implementational limitations, actions might be delayed
+        up to one second after tracing is stopped.
+
 **-a**, **--auto** *us*
 
         Set the automatic trace mode. This mode sets some commonly used options

diff --git a/Documentation/tools/rtla/common_timerlat_options.rst b/Documentation/tools/rtla/common_timerlat_options.rst
index 7854368f1827f6878b935dfe2871253eada6d7df..1f5d024b53aa03620ddc8020ef27db449c7d1ceb 100644 (file)
--- a/Documentation/tools/rtla/common_timerlat_options.rst
+++ b/Documentation/tools/rtla/common_timerlat_options.rst
@@ -1,3 +1,13 @@
+.. |threshold|  replace:: **-a/--auto**, **-i/--irq**, or **-T/--thread**
+.. |thresharg|  replace:: -T
+.. |tracer|     replace:: timerlat
+
+.. |actionsperf| replace::
+        For time-sensitive actions, it is recommended to run **rtla timerlat** with BPF
+        support and RT priority. Note that due to implementational limitations, actions
+        might be delayed up to one second after tracing is stopped if BPF mode is not
+        available or disabled.
+
 **-a**, **--auto** *us*
 
         Set the automatic trace mode. This mode sets some commonly used options
@@ -55,67 +65,3 @@
         Set timerlat to run without workload, waiting for the user to dispatch a per-cpu
         task that waits for a new period on the tracing/osnoise/per_cpu/cpu$ID/timerlat_fd.
         See linux/tools/rtla/sample/timerlat_load.py for an example of user-load code.
-
-**--on-threshold** *action*
-
-        Defines an action to be executed when tracing is stopped on a latency threshold
-        specified by **-i/--irq** or **-T/--thread**.
-
-        Multiple --on-threshold actions may be specified, and they will be executed in
-        the order they are provided. If any action fails, subsequent actions in the list
-        will not be executed.
-
-        Supported actions are:
-
-        - *trace[,file=<filename>]*
-
-          Saves trace output, optionally taking a filename. Alternative to -t/--trace.
-          Note that nlike -t/--trace, specifying this multiple times will result in
-          the trace being saved multiple times.
-
-        - *signal,num=<sig>,pid=<pid>*
-
-          Sends signal to process. "parent" might be specified in place of pid to target
-          the parent process of rtla.
-
-        - *shell,command=<command>*
-
-          Execute shell command.
-
-        - *continue*
-
-          Continue tracing after actions are executed instead of stopping.
-
-        Example:
-
-        $ rtla timerlat -T 20 --on-threshold trace
-        --on-threshold shell,command="grep ipi_send timerlat_trace.txt"
-        --on-threshold signal,num=2,pid=parent
-
-        This will save a trace with the default filename "timerlat_trace.txt", print its
-        lines that contain the text "ipi_send" on standard output, and send signal 2
-        (SIGINT) to the parent process.
-
-        Performance Considerations:
-
-        For time-sensitive actions, it is recommended to run **rtla timerlat** with BPF
-        support and RT priority. Note that due to implementational limitations, actions
-        might be delayed up to one second after tracing is stopped if BPF mode is not
-        available or disabled.
-
-**--on-end** *action*
-
-        Defines an action to be executed at the end of **rtla timerlat** tracing.
-
-        Multiple --on-end actions can be specified, and they will be executed in the order
-        they are provided. If any action fails, subsequent actions in the list will not be
-        executed.
-
-        See the documentation for **--on-threshold** for the list of supported actions, with
-        the exception that *continue* has no effect.
-
-        Example:
-
-        $ rtla timerlat -d 5s --on-end trace
-
-        This runs rtla timerlat with default options and save trace output at the end.

diff --git a/Documentation/tools/rtla/rtla-hwnoise.rst b/Documentation/tools/rtla/rtla-hwnoise.rst
index fb1c52bbc00b79dee1b8c8e38b74c94bfad6d157..3a7163c02ac8e8f4f462b63a15106022181b65cd 100644 (file)
--- a/Documentation/tools/rtla/rtla-hwnoise.rst
+++ b/Documentation/tools/rtla/rtla-hwnoise.rst
@@ -1,5 +1,7 @@
 .. SPDX-License-Identifier: GPL-2.0
 
+.. |tool| replace:: hwnoise
+
 ============
 rtla-hwnoise
 ============

diff --git a/Documentation/tools/rtla/rtla-osnoise-hist.rst b/Documentation/tools/rtla/rtla-osnoise-hist.rst
index f2e79d22c4c43f67b7859f1a2f6ab5c3e2482598..1fc60ef2610677f71bc663693e85a75be9c16308 100644 (file)
--- a/Documentation/tools/rtla/rtla-osnoise-hist.rst
+++ b/Documentation/tools/rtla/rtla-osnoise-hist.rst
@@ -1,3 +1,5 @@
+.. |tool| replace:: osnoise hist
+
 ===================
 rtla-osnoise-hist
 ===================

diff --git a/Documentation/tools/rtla/rtla-osnoise-top.rst b/Documentation/tools/rtla/rtla-osnoise-top.rst
index 5d75d1394516fd8b3374423019278ea4a2f6c15e..b1cbd7bcd4aed2c6483016c675f0b9b28acc4747 100644 (file)
--- a/Documentation/tools/rtla/rtla-osnoise-top.rst
+++ b/Documentation/tools/rtla/rtla-osnoise-top.rst
@@ -1,3 +1,5 @@
+.. |tool| replace:: osnoise top
+
 ===================
 rtla-osnoise-top
 ===================

diff --git a/Documentation/tools/rtla/rtla-timerlat-hist.rst b/Documentation/tools/rtla/rtla-timerlat-hist.rst
index b2d8726271b3d4adecf03c14f3685ea6556bde4f..4923a362129bbddf431fecbae919aec409858a80 100644 (file)
--- a/Documentation/tools/rtla/rtla-timerlat-hist.rst
+++ b/Documentation/tools/rtla/rtla-timerlat-hist.rst
@@ -1,3 +1,5 @@
+.. |tool| replace:: timerlat hist
+
 =====================
 rtla-timerlat-hist
 =====================

diff --git a/Documentation/tools/rtla/rtla-timerlat-top.rst b/Documentation/tools/rtla/rtla-timerlat-top.rst
index ab6cb60c90839de6d3ce614d6a0a3236b3e39730..50968cdd2095a1a5547f24d258007a68504c1ede 100644 (file)
--- a/Documentation/tools/rtla/rtla-timerlat-top.rst
+++ b/Documentation/tools/rtla/rtla-timerlat-top.rst
@@ -1,3 +1,5 @@
+.. |tool| replace:: timerlat top
+
 ====================
 rtla-timerlat-top
 ====================

diff --git a/tools/tracing/rtla/src/actions.c b/tools/tracing/rtla/src/actions.c
index aaf0808125d7231f96088fb69a3e9f53d34e5bc4..991139f9069f149518ed15e105a865687d8dd735 100644 (file)
--- a/tools/tracing/rtla/src/actions.c
+++ b/tools/tracing/rtla/src/actions.c
@@ -127,17 +127,17 @@ actions_add_continue(struct actions *self)
  * actions_parse - add an action based on text specification
  */
 int
-actions_parse(struct actions *self, const char *trigger)
+actions_parse(struct actions *self, const char *trigger, const char *tracefn)
 {
        enum action_type type = ACTION_NONE;
-       char *token;
+       const char *token;
        char trigger_c[strlen(trigger)];
 
        /* For ACTION_SIGNAL */
        int signal = 0, pid = 0;
 
        /* For ACTION_TRACE_OUTPUT */
-       char *trace_output;
+       const char *trace_output;
 
        strcpy(trigger_c, trigger);
        token = strtok(trigger_c, ",");
@@ -160,7 +160,7 @@ actions_parse(struct actions *self, const char *trigger)
        case ACTION_TRACE_OUTPUT:
                /* Takes no argument */
                if (token == NULL)
-                       trace_output = "timerlat_trace.txt";
+                       trace_output = tracefn;
                else {
                        if (strlen(token) > 5 && strncmp(token, "file=", 5) == 0) {
                                trace_output = token + 5;

diff --git a/tools/tracing/rtla/src/actions.h b/tools/tracing/rtla/src/actions.h
index b10a19d55c49762e06278154c10912dd310ae0a4..a4f9b570775b55e352f2aa2eebac0a221c165dfe 100644 (file)
--- a/tools/tracing/rtla/src/actions.h
+++ b/tools/tracing/rtla/src/actions.h
@@ -48,5 +48,5 @@ int actions_add_trace_output(struct actions *self, const char *trace_output);
 int actions_add_signal(struct actions *self, int signal, int pid);
 int actions_add_shell(struct actions *self, const char *command);
 int actions_add_continue(struct actions *self);
-int actions_parse(struct actions *self, const char *trigger);
+int actions_parse(struct actions *self, const char *trigger, const char *tracefn);
 int actions_perform(struct actions *self);

diff --git a/tools/tracing/rtla/src/osnoise_hist.c b/tools/tracing/rtla/src/osnoise_hist.c
index 2c2cdd467a67d1e6cd0fb0d334febd2fe6cab773..dffb6d0a98d7d95bc3d84d1755c20eb834d797c9 100644 (file)
--- a/tools/tracing/rtla/src/osnoise_hist.c
+++ b/tools/tracing/rtla/src/osnoise_hist.c
@@ -462,6 +462,8 @@ static void osnoise_hist_usage(char *usage)
                "                                                      in nanoseconds",
                "            --warm-up: let the workload run for s seconds before collecting data",
                "            --trace-buffer-size kB: set the per-cpu trace buffer size in kB",
+               "            --on-threshold <action>: define action to be executed at stop-total threshold, multiple are allowed",
+               "            --on-end <action>: define action to be executed at measurement end, multiple are allowed",
                NULL,
        };
 
@@ -531,6 +533,8 @@ static struct common_params
                        {"filter",              required_argument,      0, '5'},
                        {"warm-up",             required_argument,      0, '6'},
                        {"trace-buffer-size",   required_argument,      0, '7'},
+                       {"on-threshold",        required_argument,      0, '8'},
+                       {"on-end",              required_argument,      0, '9'},
                        {0, 0, 0, 0}
                };
 
@@ -692,6 +696,22 @@ static struct common_params
                case '7':
                        params->common.buffer_size = get_llong_from_str(optarg);
                        break;
+               case '8':
+                       retval = actions_parse(&params->common.threshold_actions, optarg,
+                                              "osnoise_trace.txt");
+                       if (retval) {
+                               err_msg("Invalid action %s\n", optarg);
+                               exit(EXIT_FAILURE);
+                       }
+                       break;
+               case '9':
+                       retval = actions_parse(&params->common.end_actions, optarg,
+                                              "osnoise_trace.txt");
+                       if (retval) {
+                               err_msg("Invalid action %s\n", optarg);
+                               exit(EXIT_FAILURE);
+                       }
+                       break;
                default:
                        osnoise_hist_usage("Invalid option");
                }

diff --git a/tools/tracing/rtla/src/osnoise_top.c b/tools/tracing/rtla/src/osnoise_top.c
index 5a56c276f9da16024776904735ab886ae53f1eb0..95418f7ecc9613f4fd416dc170c1f0ab0bb7576e 100644 (file)
--- a/tools/tracing/rtla/src/osnoise_top.c
+++ b/tools/tracing/rtla/src/osnoise_top.c
@@ -291,6 +291,8 @@ static void osnoise_top_usage(struct osnoise_params *params, char *usage)
                "                                                      in nanoseconds",
                "            --warm-up s: let the workload run for s seconds before collecting data",
                "            --trace-buffer-size kB: set the per-cpu trace buffer size in kB",
+               "            --on-threshold <action>: define action to be executed at stop-total threshold, multiple are allowed",
+               "            --on-end: define action to be executed at measurement end, multiple are allowed",
                NULL,
        };
 
@@ -371,6 +373,8 @@ struct common_params *osnoise_top_parse_args(int argc, char **argv)
                        {"filter",              required_argument,      0, '1'},
                        {"warm-up",             required_argument,      0, '2'},
                        {"trace-buffer-size",   required_argument,      0, '3'},
+                       {"on-threshold",        required_argument,      0, '4'},
+                       {"on-end",              required_argument,      0, '5'},
                        {0, 0, 0, 0}
                };
 
@@ -511,6 +515,22 @@ struct common_params *osnoise_top_parse_args(int argc, char **argv)
                case '3':
                        params->common.buffer_size = get_llong_from_str(optarg);
                        break;
+               case '4':
+                       retval = actions_parse(&params->common.threshold_actions, optarg,
+                                              "osnoise_trace.txt");
+                       if (retval) {
+                               err_msg("Invalid action %s\n", optarg);
+                               exit(EXIT_FAILURE);
+                       }
+                       break;
+               case '5':
+                       retval = actions_parse(&params->common.end_actions, optarg,
+                                              "osnoise_trace.txt");
+                       if (retval) {
+                               err_msg("Invalid action %s\n", optarg);
+                               exit(EXIT_FAILURE);
+                       }
+                       break;
                default:
                        osnoise_top_usage(params, "Invalid option");
                }

diff --git a/tools/tracing/rtla/src/timerlat_hist.c b/tools/tracing/rtla/src/timerlat_hist.c
index a9154f83f1a9ac4d3af3e1430915f5588a7fb4b7..606c1688057b2737e25613b0303a44fb4780d014 100644 (file)
--- a/tools/tracing/rtla/src/timerlat_hist.c
+++ b/tools/tracing/rtla/src/timerlat_hist.c
@@ -1047,14 +1047,16 @@ static struct common_params
                        params->deepest_idle_state = get_llong_from_str(optarg);
                        break;
                case '\5':
-                       retval = actions_parse(&params->common.threshold_actions, optarg);
+                       retval = actions_parse(&params->common.threshold_actions, optarg,
+                                              "timerlat_trace.txt");
                        if (retval) {
                                err_msg("Invalid action %s\n", optarg);
                                exit(EXIT_FAILURE);
                        }
                        break;
                case '\6':
-                       retval = actions_parse(&params->common.end_actions, optarg);
+                       retval = actions_parse(&params->common.end_actions, optarg,
+                                              "timerlat_trace.txt");
                        if (retval) {
                                err_msg("Invalid action %s\n", optarg);
                                exit(EXIT_FAILURE);

diff --git a/tools/tracing/rtla/src/timerlat_top.c b/tools/tracing/rtla/src/timerlat_top.c
index 4f1ce72d6a05a5f8663830235baba093cde39e7a..fc479a0dcb597a56abd3ba52aca574ed3a6ef534 100644 (file)
--- a/tools/tracing/rtla/src/timerlat_top.c
+++ b/tools/tracing/rtla/src/timerlat_top.c
@@ -783,14 +783,16 @@ static struct common_params
                        params->deepest_idle_state = get_llong_from_str(optarg);
                        break;
                case '9':
-                       retval = actions_parse(&params->common.threshold_actions, optarg);
+                       retval = actions_parse(&params->common.threshold_actions, optarg,
+                                              "timerlat_trace.txt");
                        if (retval) {
                                err_msg("Invalid action %s\n", optarg);
                                exit(EXIT_FAILURE);
                        }
                        break;
                case '\1':
-                       retval = actions_parse(&params->common.end_actions, optarg);
+                       retval = actions_parse(&params->common.end_actions, optarg,
+                                              "timerlat_trace.txt");
                        if (retval) {
                                err_msg("Invalid action %s\n", optarg);
                                exit(EXIT_FAILURE);

diff --git a/tools/tracing/rtla/tests/osnoise.t b/tools/tracing/rtla/tests/osnoise.t
index 7574ec6a5a53adc0ee251726d989ea9dcefe7d84..e3c89d45a6bb04bf1eb5a98b0d816d9842ea2506 100644 (file)
--- a/tools/tracing/rtla/tests/osnoise.t
+++ b/tools/tracing/rtla/tests/osnoise.t
@@ -8,7 +8,8 @@ set_timeout 2m
 check "verify help page" \
        "osnoise --help" 0 "osnoise version"
 check "verify the --priority/-P param" \
-       "osnoise top -P F:1 -c 0 -r 900000 -d 10s -q"
+       "osnoise top -P F:1 -c 0 -r 900000 -d 10s -q -S 1 --on-threshold shell,command=\"tests/scripts/check-priority.sh osnoise/ SCHED_FIFO 1\"" \
+       2 "Priorities are set correctly"
 check "verify the --stop/-s param" \
        "osnoise top -s 30 -T 1" 2 "osnoise hit stop tracing"
 check "verify the  --trace param" \
@@ -22,4 +23,28 @@ check "verify the --entries/-E param" \
 check_with_osnoise_options "apply default period" \
        "osnoise hist -s 1" 2 period_us=600000000
 
+# Actions tests
+check "trace output through -t with custom filename" \
+       "osnoise hist -S 2 -t custom_filename.txt" 2 "^  Saving trace to custom_filename.txt$"
+check "trace output through --on-threshold trace" \
+       "osnoise hist -S 2 --on-threshold trace" 2 "^  Saving trace to osnoise_trace.txt$"
+check "trace output through --on-threshold trace with custom filename" \
+       "osnoise hist -S 2 --on-threshold trace,file=custom_filename.txt" 2 "^  Saving trace to custom_filename.txt$"
+check "exec command" \
+       "osnoise hist -S 2 --on-threshold shell,command='echo TestOutput'" 2 "^TestOutput$"
+check "multiple actions" \
+       "osnoise hist -S 2 --on-threshold shell,command='echo -n 1' --on-threshold shell,command='echo 2'" 2 "^12$"
+check "hist stop at failed action" \
+       "osnoise hist -S 2 --on-threshold shell,command='echo -n 1; false' --on-threshold shell,command='echo -n 2'" 2 "^1# RTLA osnoise histogram$"
+check "top stop at failed action" \
+       "timerlat top -T 2 --on-threshold shell,command='echo -n abc; false' --on-threshold shell,command='echo -n defgh'" 2 "^abc" "defgh"
+check "hist with continue" \
+       "osnoise hist -S 2 -d 1s --on-threshold shell,command='echo TestOutput' --on-threshold continue" 0 "^TestOutput$"
+check "top with continue" \
+       "osnoise top -q -S 2 -d 1s --on-threshold shell,command='echo TestOutput' --on-threshold continue" 0 "^TestOutput$"
+check "hist with trace output at end" \
+       "osnoise hist -d 1s --on-end trace" 0 "^  Saving trace to osnoise_trace.txt$"
+check "top with trace output at end" \
+       "osnoise top -d 1s --on-end trace" 0 "^  Saving trace to osnoise_trace.txt$"
+
 test_end