]> git.ipfire.org Git - thirdparty/collectd.git/log
thirdparty/collectd.git
19 months agoMove resource attributes into `metric_family_t`. 4187/head
Florian Forster [Wed, 13 Dec 2023 20:00:14 +0000 (21:00 +0100)] 
Move resource attributes into `metric_family_t`.

19 months agosrc/daemon/metric.h: Fix typos in doc comment.
Florian Forster [Wed, 13 Dec 2023 14:37:13 +0000 (15:37 +0100)] 
src/daemon/metric.h: Fix typos in doc comment.

Co-authored-by: Eero Tamminen <eero.t.tamminen@intel.com>
19 months agoChange the semantic of `label_set_clone`.
Florian Forster [Wed, 13 Dec 2023 14:35:23 +0000 (15:35 +0100)] 
Change the semantic of `label_set_clone`.

It checks to ensure that `dest` is empty and returns `EINVAL` otherwise.

19 months agosrc/daemon/metric.c: improve readability of "first_label" logic.
Florian Forster [Wed, 13 Dec 2023 14:30:37 +0000 (15:30 +0100)] 
src/daemon/metric.c: improve readability of "first_label" logic.

Co-authored-by: Eero Tamminen <eero.t.tamminen@intel.com>
19 months agosrc/daemon/metric.h: Fix typo.
Florian Forster [Wed, 13 Dec 2023 14:29:36 +0000 (15:29 +0100)] 
src/daemon/metric.h: Fix typo.

Co-authored-by: Eero Tamminen <eero.t.tamminen@intel.com>
19 months agoformat_graphite: Add support for resource attributes.
Florian Forster [Wed, 13 Dec 2023 12:53:33 +0000 (13:53 +0100)] 
format_graphite: Add support for resource attributes.

19 months agoAdd resource labels to `metric_t`.
Florian Forster [Wed, 6 Dec 2023 07:51:31 +0000 (08:51 +0100)] 
Add resource labels to `metric_t`.

19 months agoMerge pull request #4176 from octo/6/write_http
Florian Forster [Tue, 12 Dec 2023 09:55:49 +0000 (10:55 +0100)] 
Merge pull request #4176 from octo/6/write_http

[collectd 6] add reference counting to plugins that register multiple callbacks with the same `user_data_t`.

19 months agowrite_riemann plugin: Unify reference counting code with other plugins. 4176/head
Florian Forster [Fri, 8 Dec 2023 13:31:58 +0000 (14:31 +0100)] 
write_riemann plugin: Unify reference counting code with other plugins.

19 months agowrite_http plugin: Unify coding style.
Florian Forster [Wed, 29 Nov 2023 12:59:52 +0000 (13:59 +0100)] 
write_http plugin: Unify coding style.

A few plugins implement reference counting. Ensure that they use locks
etc. consistently.

19 months agowrite_tsdb plugin: Implement reference counting.
Florian Forster [Wed, 29 Nov 2023 12:55:20 +0000 (13:55 +0100)] 
write_tsdb plugin: Implement reference counting.

19 months agowrite_syslog plugin: Implement reference counting.
Florian Forster [Wed, 29 Nov 2023 12:53:08 +0000 (13:53 +0100)] 
write_syslog plugin: Implement reference counting.

19 months agowrite_riemann plugin: use reference counting to when freeing user data.
Florian Forster [Wed, 29 Nov 2023 12:41:32 +0000 (13:41 +0100)] 
write_riemann plugin: use reference counting to when freeing user data.

While reference counting was present previously, it had problems:

*   The reference passed to `plugin_register_flush()` was not counted.
*   If a flush plugin was registered, `free_func` was set to NULL but
    the reference passed to `plugin_register_notification()` was still
    counted, meaning in that case the counter never went to zero.
*   Mutexes must be unlocked when calling `pthread_mutex_destroy()`.
*   The code limped on after an error, returning a failure eventually.
    This is unnecessarily complex control flow that has been simplified.

19 months agowrite_stackdriver plugin: Implement reference counting.
Florian Forster [Wed, 29 Nov 2023 12:24:00 +0000 (13:24 +0100)] 
write_stackdriver plugin: Implement reference counting.

19 months agosrc/write_http.c: use reference counting to decide when to free user_data
Leonard Göhrs [Fri, 24 Mar 2023 09:49:24 +0000 (10:49 +0100)] 
src/write_http.c: use reference counting to decide when to free user_data

The teardown code for the wh_callback_t struct previously relied on
the order in which the different callback functions are de-initialized
to be known and to never change.

This is prone to failure and is indeed currently broken, leading to a
segmentation fault on collectd exit.

Fix this by counting the active references to the user data and freeing
it once it reaches zero.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
20 months agoMerge pull request #4182 from octo/6/configure-write_http
Florian Forster [Wed, 6 Dec 2023 12:12:27 +0000 (13:12 +0100)] 
Merge pull request #4182 from octo/6/configure-write_http

configure: Disable the write_http plugin if libyajl2 is not found.

20 months agoconfigure: Disable the write_http plugin if libyajl2 is not found. 4182/head
Florian Forster [Tue, 5 Dec 2023 12:30:33 +0000 (13:30 +0100)] 
configure: Disable the write_http plugin if libyajl2 is not found.

20 months agoMerge pull request #4181 from octo/6/otep119
Florian Forster [Wed, 6 Dec 2023 12:04:56 +0000 (13:04 +0100)] 
Merge pull request #4181 from octo/6/otep119

Update standard system metric names in accordance with OpenTelemetry.

20 months agomemory plugin: Use standard system metric names (OTEP #119) 4181/head
Florian Forster [Tue, 5 Dec 2023 10:27:41 +0000 (11:27 +0100)] 
memory plugin: Use standard system metric names (OTEP #119)

https://github.com/open-telemetry/oteps/blob/main/text/0119-standard-system-metrics.md

20 months agodisk plugin: Use standard system metric names (OTEP #119)
Florian Forster [Tue, 5 Dec 2023 10:27:28 +0000 (11:27 +0100)] 
disk plugin: Use standard system metric names (OTEP #119)

https://github.com/open-telemetry/oteps/blob/main/text/0119-standard-system-metrics.md

20 months agodf plugin: Use standard system metric names (OTEP #119)
Florian Forster [Tue, 5 Dec 2023 10:27:15 +0000 (11:27 +0100)] 
df plugin: Use standard system metric names (OTEP #119)

https://github.com/open-telemetry/oteps/blob/main/text/0119-standard-system-metrics.md

20 months agocpu plugin: Use standard system metric names (OTEP #119)
Florian Forster [Tue, 5 Dec 2023 10:26:43 +0000 (11:26 +0100)] 
cpu plugin: Use standard system metric names (OTEP #119)

https://github.com/open-telemetry/oteps/blob/main/text/0119-standard-system-metrics.md

20 months agoMerge pull request #4179 from octo/6/write_log
Florian Forster [Mon, 4 Dec 2023 18:15:05 +0000 (19:15 +0100)] 
Merge pull request #4179 from octo/6/write_log

write_log plugin: Remove empty lines, add example config

20 months ago./contrib/format.sh src/write_log.c 4179/head
Florian Forster [Mon, 4 Dec 2023 17:44:09 +0000 (18:44 +0100)] 
./contrib/format.sh src/write_log.c

20 months agowrite_log plugin: Add example config block to `collectd.conf`.
Florian Forster [Fri, 1 Dec 2023 10:33:12 +0000 (11:33 +0100)] 
write_log plugin: Add example config block to `collectd.conf`.

20 months agowrite_log plugin: remove trailing newlines in the Graphite format.
Florian Forster [Fri, 1 Dec 2023 10:31:58 +0000 (11:31 +0100)] 
write_log plugin: remove trailing newlines in the Graphite format.

The `INFO()` log macro will add a newline, which was leading to empty lines
being printed.

20 months agoMerge pull request #4174 from octo/workflows6
Florian Forster [Fri, 1 Dec 2023 09:50:39 +0000 (10:50 +0100)] 
Merge pull request #4174 from octo/workflows6

[collectd 6.0] Fix the build

20 months agoIntel RDT plugin: use `pqos_mon_start_cores()` if available. 4174/head
Florian Forster [Fri, 24 Nov 2023 10:39:00 +0000 (11:39 +0100)] 
Intel RDT plugin: use `pqos_mon_start_cores()` if available.

20 months agoIntel RDT plugin: use `pqos_mon_start_pids2()` if available.
Florian Forster [Fri, 24 Nov 2023 10:20:01 +0000 (11:20 +0100)] 
Intel RDT plugin: use `pqos_mon_start_pids2()` if available.

Newer versions of the pqos library deprecate the previous function, so
we have to migrate even though it makes memory management quite ugly.

ChangeLog: Intel RDT plugin: support for PQOS ≥5.0.0 has been added.

20 months agomdevents plugin: Re-enable the `compile_regex` test.
Florian Forster [Wed, 29 Nov 2023 19:54:47 +0000 (20:54 +0100)] 
mdevents plugin: Re-enable the `compile_regex` test.

20 months agoconfigure.ac: De-cuplicate `plugin_foo` definitions.
Florian Forster [Wed, 29 Nov 2023 19:41:24 +0000 (20:41 +0100)] 
configure.ac: De-cuplicate `plugin_foo` definitions.

Apparently we defined a bunch of `plugin_foo` variables that were never
used. The generated list from the arguments to `AC_PLUGIN` now appear as
duplicates. This removes the previously unused definitions and leaves
the generated ones.

20 months agocapabilities plugin: Make state tracking clearer in the MDH handler.
Florian Forster [Wed, 29 Nov 2023 19:24:03 +0000 (20:24 +0100)] 
capabilities plugin: Make state tracking clearer in the MDH handler.

20 months agoprocevent plugin: Consistently mention the unit in log message.
Florian Forster [Wed, 29 Nov 2023 13:40:17 +0000 (14:40 +0100)] 
procevent plugin: Consistently mention the unit in log message.

Co-authored-by: Eero Tamminen <eero.t.tamminen@intel.com>
20 months agoprocevent plugin: Reference buffer sizing guideline.
Florian Forster [Wed, 29 Nov 2023 13:32:30 +0000 (14:32 +0100)] 
procevent plugin: Reference buffer sizing guideline.

Co-authored-by: Eero Tamminen <eero.t.tamminen@intel.com>
20 months agoprocevent plugin: report errors in human readable form.
Florian Forster [Wed, 29 Nov 2023 09:02:08 +0000 (10:02 +0100)] 
procevent plugin: report errors in human readable form.

20 months agoprocevent plugin: ensure the ring buffer length is not zero.
Florian Forster [Wed, 29 Nov 2023 09:00:43 +0000 (10:00 +0100)] 
procevent plugin: ensure the ring buffer length is not zero.

The manpage documents a default of 10, which was not implemented. This
change brings the implementation in line with the documentation.

20 months agoprocevent plugin: remove use of a nested flexible array member.
Florian Forster [Tue, 28 Nov 2023 18:31:11 +0000 (19:31 +0100)] 
procevent plugin: remove use of a nested flexible array member.

The previous code used an ad-hoc struct to construct or parse a Netlink
message. This relied on allocating a field _after_ the struct with the
flexible array member, which is prohibited by the C standard, leading to
compiler warnings.

20 months agoSMART plugin: initialize struct passed to `ioctl(2)`.
Florian Forster [Sat, 25 Nov 2023 13:12:59 +0000 (14:12 +0100)] 
SMART plugin: initialize struct passed to `ioctl(2)`.

Valgrind is complaining about a conditional jump based on uninitialized
memory:

```
==66462== Conditional jump or move depends on uninitialised value(s)
==66462==    at 0x10C500: smart_read_nvme_intel_disk (in /__w/collectd/collectd/test_plugin_smart)
==66462==    by 0x10D366: test_x (in /__w/collectd/collectd/test_plugin_smart)
==66462==    by 0x10D638: main (in /__w/collectd/collectd/test_plugin_smart)
```

This may be due to the `struct nvme_additional_smart_log` being
uninitialized when it's being passed to `ioctl(2)`.

This there, this removed an unnecessary level of indentation.

20 months agoNetlink plugin: complete initialize structs used for testing.
Florian Forster [Sat, 25 Nov 2023 12:51:57 +0000 (13:51 +0100)] 
Netlink plugin: complete initialize structs used for testing.

Valgrind complains about a conditional jump based on uninitialized
memory:

```
==66438== Conditional jump or move depends on uninitialised value(s)
==66438==    at 0x10CA06: vf_info_submit (in /__w/collectd/collectd/test_plugin_netlink)
==66438==    by 0x1110F2: test_vf_submit_test (in /__w/collectd/collectd/test_plugin_netlink)
==66438==    by 0x112EAC: main (in /__w/collectd/collectd/test_plugin_netlink)
```

This is likely caused by the `vf_stats_t` being only partially
initialized. Using a struct initializer is not only cleaner, it also
ensures the remainder of the struct is initialized to zero.

20 months agoAdd Valgrind suppression for libvirt memory leak.
Florian Forster [Thu, 23 Nov 2023 15:17:55 +0000 (16:17 +0100)] 
Add Valgrind suppression for libvirt memory leak.

20 months agoNUT plugin: import `<time.h>` before `<upsclient.h>`.
Florian Forster [Sun, 26 Nov 2023 14:27:28 +0000 (15:27 +0100)] 
NUT plugin: import `<time.h>` before `<upsclient.h>`.

20 months agoconfigure.ac: if neither UPSCONN{,_t} type was found, refuse to build NUT plugin
Jim Klimov [Wed, 31 Aug 2022 13:32:46 +0000 (15:32 +0200)] 
configure.ac: if neither UPSCONN{,_t} type was found, refuse to build NUT plugin

NOTE: src/nut.c also has pragmas to error out in this situation,
but that handling is compiler-dependent and happens too late in
the checkout/configure/build loop.

Presumably this inability to find the type in the earlier-found header file
is also triggered by build environment "inconsistencies" like lack of basic
types in the libc implementation (maybe highlighting the need for additional
headers or macros for the platform).

20 months agoconfigure.ac, src/nut.c: detect int types required by NUT API we build against
Jim Klimov [Wed, 31 Aug 2022 09:40:01 +0000 (11:40 +0200)] 
configure.ac, src/nut.c: detect int types required by NUT API we build against

Either use the stricter int types required by NUT headers since v2.8.0 release,
or the relaxed (arch-dependent) types required by older NUT releases - depending
on which NUT API version the collectd is building against at the moment.

Inspired by discussion at https://github.com/networkupstools/nut/issues/1638

20 months agocapabilities plugin: remove warning about "leaking" a pointer to a local variable.
Florian Forster [Tue, 28 Nov 2023 13:50:36 +0000 (14:50 +0100)] 
capabilities plugin: remove warning about "leaking" a pointer to a local variable.

```
src/capabilities.c: In function 'cap_http_handler':
src/capabilities.c:209:23: error: storing the address of local variable '({anonymous})' in '*connection_state' [-Werror=dangling-pointer=]
  209 |     *connection_state = &(int){44};
      |     ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
src/capabilities.c:209:31: note: '({anonymous})' declared here
  209 |     *connection_state = &(int){44};
      |                               ^
src/capabilities.c:209:31: note: 'connection_state' declared here
```

20 months agocpu plugin: Fix potential buffer overflow.
Florian Forster [Tue, 28 Nov 2023 13:42:54 +0000 (14:42 +0100)] 
cpu plugin: Fix potential buffer overflow.

```
In function 'cpu_commit_without_aggregation',
    inlined from 'cpu_commit' at src/cpu.c:563:5,
    inlined from 'cpu_read' at src/cpu.c:925:3:
src/cpu.c:534:50: note: directive argument in the range [0, 18446744073709551614]
  534 |       snprintf(cpu_num_str, sizeof(cpu_num_str), "%zu", cpu_num);
      |                                                  ^~~~~
src/cpu.c:534:7: note: 'snprintf' output between 2 and 21 bytes into a destination of size 16
  534 |       snprintf(cpu_num_str, sizeof(cpu_num_str), "%zu", cpu_num);
      |       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
```

20 months agogmond plugin: Improve `mc_handle_metadata_msg()`.
Florian Forster [Sat, 25 Nov 2023 23:10:52 +0000 (00:10 +0100)] 
gmond plugin: Improve `mc_handle_metadata_msg()`.

This function was not properly formatted. While there:

* Handle the error instead of indenting the successful flow.
* Declare variables late.
* Initialize variables when declaring them.

20 months agogmond plugin: Look for RPC headers in /usr/include/tirpc.
Florian Forster [Sat, 25 Nov 2023 22:47:47 +0000 (23:47 +0100)] 
gmond plugin: Look for RPC headers in /usr/include/tirpc.

glibc removed the SunRPC headers, and the replacement headers are in a
different location. Try and look for them there.

20 months agogmond plugin: Import `<ganglia.h>`.
Florian Forster [Sat, 25 Nov 2023 16:35:27 +0000 (17:35 +0100)] 
gmond plugin: Import `<ganglia.h>`.

`<gm_protocol.h>` has been removed in new versions of libganglia.

20 months agocurl_stats: fix compatibility with new versions of cURL.
Florian Forster [Fri, 24 Nov 2023 07:28:06 +0000 (08:28 +0100)] 
curl_stats: fix compatibility with new versions of cURL.

Use integer based keys for metrics if available.

cURL ≥ 7.55.0 provides additional keys that allow getting certain
metrics as integers rather than doubles, e.g. content length. In some
newer versions of cURL, the original keys (using doubles) are marked as
deprecated.

ChangeLog: cURL, cURL-JSON, cURL-XML, Write HTTP plugins: fix compatibility with new versions of cURL.

20 months agoImport workflow definitions from `main`.
Florian Forster [Tue, 28 Nov 2023 10:06:22 +0000 (11:06 +0100)] 
Import workflow definitions from `main`.

20 months agoconfigure: disable all plugins not yet supporting collectd 6.
Florian Forster [Tue, 28 Nov 2023 12:56:27 +0000 (13:56 +0100)] 
configure: disable all plugins not yet supporting collectd 6.

This should allow us (and users) to just run `./configure` without
further flags, lowering the barrier to entry. It also allows us to
remove these configure flags from the CI configuration.

20 months agoconfigure: ensure there is a `plugin_foo` variable for each plugin.
Florian Forster [Tue, 28 Nov 2023 12:47:32 +0000 (13:47 +0100)] 
configure: ensure there is a `plugin_foo` variable for each plugin.

20 months agoMerge pull request #4173 from eero-t/log-metrics
Florian Forster [Wed, 29 Nov 2023 19:18:08 +0000 (20:18 +0100)] 
Merge pull request #4173 from eero-t/log-metrics

[Collectd 6] gpu_sysman: improve metric variants handling

20 months agogpu_sysman: minor improvements to Sysman plugin documentation 4173/head
Eero Tamminen [Tue, 21 Nov 2023 16:24:08 +0000 (18:24 +0200)] 
gpu_sysman: minor improvements to Sysman plugin documentation

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
20 months agogpu_sysman: rename "counter" output variant to more generic "base"
Eero Tamminen [Fri, 24 Nov 2023 17:05:51 +0000 (19:05 +0200)] 
gpu_sysman: rename "counter" output variant to more generic "base"

And make it control output for all base metric values, not just
counters.  That allows disabling output of values for:
- Memory usage
- Frequency
- Temperature
If one wants to see only their rates.

That will be useful with the new "LogMetrics" option in next commit.

Did also small optimization for output variant checks (no need for
free() if they're moved earlier).

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
20 months agogpu_sysman: better metrics check interval counter name
Eero Tamminen [Fri, 17 Nov 2023 16:48:55 +0000 (18:48 +0200)] 
gpu_sysman: better metrics check interval counter name

Collectd value_t struct has also "counter" field, so this makes Sysman
plugin check interval counter easier to find.

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
20 months agogpu_sysman: add unit (s) to interval settings output
Eero Tamminen [Mon, 27 Nov 2023 13:48:48 +0000 (15:48 +0200)] 
gpu_sysman: add unit (s) to interval settings output

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
20 months agoMerge pull request #3810 from manuelluis/mlsr/collectd6-df
Florian Forster [Wed, 29 Nov 2023 15:32:39 +0000 (16:32 +0100)] 
Merge pull request #3810 from manuelluis/mlsr/collectd6-df

[collectd 6] df: migration to v6.0

20 months ago[collectd 6] df: migration to v6.0 3810/head
Manuel Luis Sanmartín Rozada [Wed, 20 Jan 2021 22:56:30 +0000 (23:56 +0100)] 
[collectd 6] df: migration to v6.0

20 months agoMerge pull request #4175 from octo/write_log6
Florian Forster [Wed, 29 Nov 2023 13:18:16 +0000 (14:18 +0100)] 
Merge pull request #4175 from octo/write_log6

write_log plugin: remove the "write_log values:" line when logging metrics.

20 months agowrite_log plugin: remove the "write_log values:" line when logging metrics. 4175/head
Florian Forster [Wed, 29 Nov 2023 09:54:40 +0000 (10:54 +0100)] 
write_log plugin: remove the "write_log values:" line when logging metrics.

23 months agogpu_sysman: fix initial min memory check value
Eero Tamminen [Wed, 5 Jul 2023 14:00:34 +0000 (17:00 +0300)] 
gpu_sysman: fix initial min memory check value

High end GPUs can have more memory than earlier initial dummy value.

Reported-by: Valtteri Rantala <valtteri.rantala@intel.com>
Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
23 months agogpu_sysman: Improve counter metrics documentation
Eero Tamminen [Fri, 2 Jun 2023 14:29:22 +0000 (17:29 +0300)] 
gpu_sysman: Improve counter metrics documentation

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
2 years agogpu_sysman: fix "Enabled metrics: none" logging
Eero Tamminen [Mon, 24 Apr 2023 13:03:38 +0000 (16:03 +0300)] 
gpu_sysman: fix "Enabled metrics: none" logging

"none" could be logged erronously when "DisableEngine" is set.

Add test code for that (does not validate the log output, but
runs that and "no metrics + logging" cases so output can be
checked manually).

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
2 years agogpu_sysman: log correct power limit error value
Eero Tamminen [Fri, 10 Mar 2023 14:12:37 +0000 (16:12 +0200)] 
gpu_sysman: log correct power limit error value

Fixes: 55a9296a0ec1b
Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
2 years agogpu_sysman: Improve disabled/enabled metrics logging
Eero Tamminen [Wed, 19 Oct 2022 15:18:00 +0000 (18:18 +0300)] 
gpu_sysman: Improve disabled/enabled metrics logging

- Move enabled/disabled metric reporting to a separate function
- Report metrics enabling and metric details enabling separately
- Error if all metrics are disabled, regardless of detail options
- Explicitly log what metrics are still being reported if any of
  them were disabled at run-time

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
2 years agogpu_sysman: Output device ECC state with other GPU info at start
Eero Tamminen [Thu, 22 Sep 2022 18:44:34 +0000 (21:44 +0300)] 
gpu_sysman: Output device ECC state with other GPU info at start

Added in L0 spec v1.4.

Requires loader 1.8.0 version released in May 2022.

(With minor cleanup comments from Alexey applied.)

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
2 years agogpu_sysman: Add mapping for new memory types
Eero Tamminen [Thu, 22 Sep 2022 17:27:00 +0000 (20:27 +0300)] 
gpu_sysman: Add mapping for new memory types

Added in L0 spec v1.3.

Signed-off-by: Eero Tamminen <eero.t.tamminen@intel.com>
2 years ago[collectd 6] exec: make PUTMETRIC work
Leonard Göhrs [Mon, 9 Jan 2023 13:43:44 +0000 (14:43 +0100)] 
[collectd 6] exec: make PUTMETRIC work

The cmd_handle_putmetric function checks if the command actually is
a PUTMETRIC command, at least that is what is supposed to check.
Prior to this fix it actually checked for PUTVAL and always prints a

    -1 Unexpected command: `PUTMETRIC'.

error. While at it also remove the development printf that results in

    set_option("type", "GAUGE")
    set_option("type", "Counter")
    …

being printed during normal operation.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years ago[collectd 6] exec: add PUTMETRIC command
Leonard Göhrs [Thu, 5 Jan 2023 09:11:23 +0000 (10:11 +0100)] 
[collectd 6] exec: add PUTMETRIC command

Most existing setups using exec will use PUTVAL, which should just continue
to work with collectd 6 due to the plugin_dispatch_values compatibility
function.
New plugins should however use the new PUTMETRIC.
The respective command handler already exists. This commit just pipes it
through.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years agosrc/daemon/plugin.c: init fam.{time,interval} before uc_update(fam) on dispatch
Leonard Göhrs [Fri, 24 Mar 2023 13:03:43 +0000 (14:03 +0100)] 
src/daemon/plugin.c: init fam.{time,interval} before uc_update(fam) on dispatch

The changes in commit

  55efb56a ([collectd 6] src/daemon/plugin.c: Use one thread per write plugin)

changed the order in which the time and interval on a to-be-dispatched metric
family were set and uc_update() was called on the metric family.

The time and interval inside a metric family have to be set before calling
uc_update() on it, otherwise warnings like these will be generated en masse[1]:

  uc_update: Value too old: … value time = 0.000; last cache update = 0.000;

And the "missing" handlers for the metric family will be called resulting in
plugins like write_prometheus dropping the value, resulting in it not showing
up when queried [2].

This change requires the inroduction of a second metric_family_clone() in the
dispatch path:

 plugin_dispatch_metric_family()
 | \
 |  Has to modify the passed fam to initialize the correct time and interval.
 |  So it has to metric_family_clone() the input.
 v
 plugin_dispatch_metric_internal()
 | \
 |  Calls the filter chains and uc_update(fam), which both require correct
 |  times and intervals to be set.
 |
 |     plugin_dispatch_metric_internal() calls either fc_process_chain() or
 v <-- fc_default_action() depending on if a chain is configured.
 |\
 | fc_process_chain()
 | |
 | v
 | plugin_write()
 |   \
 |    A chain may call plugin_write() multiple times with the same fam.
 |    This means plugin_write() can not take "ownership" of the fam,
 |    put it in a queue and free it once it feels like it.
 |    It must instead create another clone to put into the queue.
 v
 fc_default_action()
 |
 v
 plugin_write()
   \
    This is the much more common case, as filter chains are a niche feature,
    and in this case we could actually transfer ownership of the fam to
    plugin_write and save a clone, but we would have to tell plugin_write()
    that it is responsible for freeing the passed fam.

The negative performance impact of the clone could be mitigated by adding a
reference count to the metric_family_t struct and only freeing it once all
references to it are gone. But that would be a larger change and not a bug fix.

Only fix the "uninitialized time and interval" bug for now.

[1]: https://github.com/collectd/collectd/pull/4026#issuecomment-1463884930
[2]: https://github.com/collectd/collectd/issues/4098

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years agosrc/daemon/plugin.c: don't store references to stack allocated values
Leonard Göhrs [Thu, 23 Mar 2023 12:12:27 +0000 (13:12 +0100)] 
src/daemon/plugin.c: don't store references to stack allocated values

The changes in commit

  55efb56a ([collectd 6] src/daemon/plugin.c: Use one thread per write plugin)

wrongly assume that the references passed in to plugin_register_write()
somehow outlive the spawned write thread.
While this is true for some plugins that pass staticly allocated strings
and global user_data_t structs to plugin_register_write() it is not correct
for all plugins.

See [1] for an example of a plugin (write_http) mis-behaving due to this.

Store owned versions of the passed values instead. For user_data this means
the content of the struct and for the name it means a strdup()ed version of
the string.

[1]: https://github.com/collectd/collectd/issues/4098

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years ago[collectd 6] entropy: migration to v6.0
Manuel Luis Sanmartín Rozada [Wed, 20 Jan 2021 23:11:47 +0000 (00:11 +0100)] 
[collectd 6] entropy: migration to v6.0

2 years ago[collectd 6] src/daemon/plugin.h: Revert plugin_write documentation change
Leonard Göhrs [Fri, 28 Oct 2022 08:08:01 +0000 (10:08 +0200)] 
[collectd 6] src/daemon/plugin.h: Revert plugin_write documentation change

The behaviour of plugin_write with regards to fam being NULL has not changed
with the switch to one thread per write plugin, so the documentation should
not change as well.

2 years ago[collectd 6] src/collectd.conf.pod polish the WriteQueueLimitHigh/Low docs
Leonard Göhrs [Fri, 28 Oct 2022 08:02:01 +0000 (10:02 +0200)] 
[collectd 6] src/collectd.conf.pod polish the WriteQueueLimitHigh/Low docs

The behaviour of LimitHight/LimitLow has changed with the switch to one
thread per write plugin. This warrants a rewrite of the respective
documentation. Thanks to @eero-t for the suggestion.

2 years ago[collectd 6] src/daemon/plugin.c: restore previous position of plugin_write
Leonard Göhrs [Mon, 15 Aug 2022 06:43:51 +0000 (08:43 +0200)] 
[collectd 6] src/daemon/plugin.c: restore previous position of plugin_write

The new queue design resulted in plugin_write being based on
enqueue_metric_family, which resulted in the functions moving around in the
file. This made the diff harder to read. Restore old position.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years ago[collectd 6] src/daemon/plugin.c: work around missing stpcpy on solaris
Leonard Göhrs [Mon, 15 Aug 2022 06:32:08 +0000 (08:32 +0200)] 
[collectd 6] src/daemon/plugin.c: work around missing stpcpy on solaris

The CI tests on solaris failed due to missing stpcpy. Work around it using
memcpy and strlen.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years ago[collectd 6] src/daemon/plugin.c: Use one thread per write plugin
Leonard Göhrs [Tue, 19 Jul 2022 09:20:09 +0000 (11:20 +0200)] 
[collectd 6] src/daemon/plugin.c: Use one thread per write plugin

ChangeLog: collectd: Use one write thread per write plugin

The previous write thread design used a single queue with a single read head
from which one of the write threads would de-queue an element and would then
sequentially call each registered write callback.
This meant that all write plugins would have to cooperate in order to not drop
values. If for example all write threads are stalled by the same write plugin's
callback function not returning, the queue will start to fill up until elements
start to be dropped, even though there are other plugins that could still make
progress. In addition to that, all write callbacks have to be designed to be
reentrant right now, which increases complexity.

This new design uses a single linked-list write queue with one read head per
output plugin. Each output plugin is serviced in a dedicated write thread.
Elements are freed based on a reference count, which is shown in the ASCII-Art
below:

          +- Thread #1 Head       +- Thread #2 Head       +- Tail
          v                       v                       v
   +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+
   | 0|->| 1|->| 1|->| 1|->| 1|->| 2|->| 2|->| 2|->| 2|->| 2|->X
   +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+  +--+
    ^
    +- to be free()d

The changes introduced by this commit have some side-effects:

- The WriteThreads config option no longer exists, as a strict 1:1 ratio of
  write plugins and write threads is used.
- The data flow has changed. The previous data flow was:

    (From one of the ReadThreads)
      plugin_dispatch_{values,multivalue}()
      plugin_dispatch_metric_family()
      enqueue_metric_family()
      write_queue_enqueue() -----{Queue}----+
                                            |
    (In one of the WriteThreads threads)    |
      plugin_write_thread()                 |
        ^- plugin_write_dequeue()         <-+
      plugin_dispatch_metric_internal()
        ^- fc_process_chain(pre_cache_chain)
      fc_process_chain(fc_process_chain)
      fc_bit_write_invoke()
      plugin_write(NULL) / plugin_write(plugin_name)
      plugin callback()

  The data flow now is:

    (From one of the ReadThreads)
      plugin_dispatch_{values,multivalue}()
      plugin_dispatch_metric_family()
      plugin_dispatch_metric_internal()
        ^- fc_process_chain(pre_cache_chain)
      fc_process_chain(post_cache_chain)
      fc_bit_write_invoke()
      plugin_write(NULL) / plugin_write(plugin_name)
      write_queue_enqueue() -----{Queue}----+
                                            |
    (In one of the WriteThreads threads)    |
      plugin_write_thread()               <-+
      plugin callback()

  One result of this change is, that the behaviour of plugin_write has changed
  from running the plugin callback immediately and in the same thread, to
  always enqueueing the value and de-queing in the dedicated thread.
- The behaviour of the WriteQueueLimitHigh and WriteQueueLimitLow options has
  changed. The Queue will be be capped to a length of LimitHigh by dropping
  random queue elements between the queue end and LimitLow.
  Setting LimitLow to a reasonably large value ensures that fast write plugins
  do not loose values, even in the vicinity of a slow plugin.
  The diagram below shows the random element selected for removal (###) in
  Step 1 and the queue with the element removed in Step 2.

  Step 1:
    +- Thread #1 Head    |  +- Thread #2 Head       +- Tail
    v  |                 |  v                       v
   +--+| +--+  ####  +--+| +--+  +--+  +--+  +--+  +--+
   | 1|->| 1|-># 1#->| 1|->| 2|->| 2|->| 2|->| 2|->| 2|->X
   +--+| +--+  ####  +--+| +--+  +--+  +--+  +--+  +--+
       |                 |
       | LimitHigh       | LimitLow

  Step 2:
       |  +- Thread #1 Head +- Thread #2 Head       +- Tail
       |  v              |  v                       v
       | +--+  +--+  +--+| +--+  +--+  +--+  +--+  +--+
       | | 1|->| 1|->| 1|->| 2|->| 2|->| 2|->| 2|->| 2|->X
       | +--+  +--+  +--+| +--+  +--+  +--+  +--+  +--+
       |                 |
       | LimitHigh       | LimitLow

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years ago[collectd 6] mmc: port to collectd 6
Leonard Göhrs [Wed, 14 Dec 2022 12:23:14 +0000 (13:23 +0100)] 
[collectd 6] mmc: port to collectd 6

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years agommc: cache open file descriptors to block devices
Leonard Göhrs [Tue, 27 Sep 2022 06:03:14 +0000 (08:03 +0200)] 
mmc: cache open file descriptors to block devices

Udev rules can contain a "watch" option, which is described in the man page as:

  Watch the device node with inotify; when the node is closed after being
  opened for writing, a change uevent is synthesized.

This watch option is enabled by default for all block devices[1].
The intention behind this is to be notified about changes to the partition
table. The mmc plugin does however also need to open the block device for
writing, even though it never modifies its content, in order to be able to
issue ioctls with vendor defined MMC-commands.

Reduce the amount of generated change events from one per read to one per
collectd runtime by caching the open file descriptor.

[1]: https://github.com/systemd/systemd/blob/ef2ad30aee9fa99b0fdb8fe7efda397513cec6af/rules.d/60-block.rules

Fixes: 2f15c704 (mmc: add more vendor specific and generic data sources (#4006))
Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years agommc: add more vendor specific and generic data sources (#4006)
Leonard Göhrs [Fri, 3 Jun 2022 13:31:54 +0000 (15:31 +0200)] 
mmc: add more vendor specific and generic data sources (#4006)

* mmc plugin: integrate into configure.ac

The mmc plugin is not fully integrated in the configure.ac.
Change that.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
* mmc plugin: Skip mmc paths in /sys that start with a '.' (like "." and "..")

The plugin tries to (and obiously fails to) use "." and "..", that come out of
listdir, as mmc devices.
Filter these two out by skipping hidden files/directories.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
* mmc plugin: read standard eMMC 5.0 health metrics

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
* mmc plugin: remove type-name defines

These defines can become confusing, especially when combined with the defines
for attribute names in the sysfs. This will only get worse when more
vendor-specific metrics are supported.
Remove the defines and use the type names directly.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
* mmc plugin: remove sysfs-attribute defines

These defines are used only once or twice and do not help with readability.
Replace them with just the raw strings.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
* mmc plugin: port to libudev

While using the sysfs directly works fine for the swissbit and generic eMMC
driver it does not scale well to other vendor-specific interfaces where one has
to open the block device in /dev to perform ioctls.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
* mmc plugin: add micron eMMC support

While this patch was only tested with a single product (MTFC16GAPALBH) I am
fairly confident that it will generalize to others as well, as micron
themselves ship a single tool[1], which this patch uses as a reference, to read
similar info from all of their eMMCs.

This patch also increases the maximum value of mmc_bad_blocks to infinity,
as it can be any 16 bit integer for micron eMMC but could be even larger for
other vendors.

[1]: https://github.com/arcus-smart-home/arcushubos/blob/master/meta-iris/recipes-core/iris-utils/files/emmcparm_1.0.c

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
* mmc plugin: add sandisk eMMC support

While this patch was only tested with a single product (SDINBDG4-8G), I am
fairly confident that it should generalize to other devices as well,
as the current product portfolio on their website looks very similar to the one
I tested and new devies will likely use a Western Digital manufacturer ID.

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years agommc plugin: initial checkin (#3887)
Florian Eckert [Mon, 13 Sep 2021 13:00:46 +0000 (15:00 +0200)] 
mmc plugin: initial checkin (#3887)

Signed-off-by: Florian Eckert <fe@dev.tdt.de>
2 years agoFormatted files
Matthias Runge [Sat, 11 Feb 2023 11:04:55 +0000 (12:04 +0100)] 
Formatted files

2 years ago[collectd 6] disk: cherry-pick "In macOS 12, `IOMasterPort` is deprecated in favor...
Clay Caviness [Fri, 15 Jul 2022 17:29:11 +0000 (13:29 -0400)] 
[collectd 6] disk: cherry-pick "In macOS 12, `IOMasterPort` is deprecated in favor of `IOMainPort`"

git cherry-pick 2711ebe7d671c ("In macOS 12, `IOMasterPort` is deprecated in favor of `IOMainPort`")
from the collectd 5 branch on top of the collectd 6 version of the plugin.

Original commit message:

In macOS 12, `IOMasterPort` is deprecated in favor of `IOMainPort`

```
src/battery.c:250:7: error: 'kIOMasterPortDefault' is deprecated: first deprecated in macOS 12.0 [-Werror,-Wdeprecated-declarations]
      kIOMasterPortDefault, IOServiceNameMatching("battery"), &iterator);
      ^~~~~~~~~~~~~~~~~~~~
      kIOMainPortDefault
```

2 years ago[collectd 6] cherry-pick "disk: add utilization (#3967)"
Marek Bečka [Fri, 4 Feb 2022 07:13:17 +0000 (08:13 +0100)] 
[collectd 6] cherry-pick "disk: add utilization (#3967)"

git cherry-pick f703a4ad700a4 ("disk: add utilization (#3967)")
from the collectd 5 branch on top of the collectd 6 version of the plugin.

Original commit message:

* added disk utilization

* fix disk utilization

* fix interval back to ms

2 years agoFix syntax errors and casting.
Manuel Luis Sanmartín Rozada [Tue, 16 Feb 2021 17:21:04 +0000 (18:21 +0100)] 
Fix syntax errors and casting.

2 years ago[collectd 6] disk: migration to v6.0
Manuel Luis Sanmartín Rozada [Wed, 20 Jan 2021 22:35:43 +0000 (23:35 +0100)] 
[collectd 6] disk: migration to v6.0

2 years ago[collectd 6] contextswitch: migration to v6.0
Manuel Luis Sanmartín Rozada [Wed, 20 Jan 2021 23:02:18 +0000 (00:02 +0100)] 
[collectd 6] contextswitch: migration to v6.0

2 years ago[collectd 6] memory: add laundry and user wired pages (#3962)
François Charlier [Tue, 15 Feb 2022 12:02:55 +0000 (13:02 +0100)] 
[collectd 6] memory: add laundry and user wired pages (#3962)

This ports "add laundry and user wired pages (#3962)" (de33b26ba4d67)
from the main branch to collectd-6.0.

Changelog: memory: report for laundry and user_wire pages on FreeBSD

Add the `vm.stats.vm.v_laundry_count` and
`vm.stats.vm.v_user_wire_count`  which have been present on FreeBSD for
a little while now.

2 years ago[collectd 6] port Do not account reclaimable slab as used
Weiping Zhang [Mon, 17 May 2021 07:59:50 +0000 (15:59 +0800)] 
[collectd 6] port Do not account reclaimable slab as used

This ports "Do not account reclaimable slab as used" (77e2fcd91e27)
from the main branch to collectd-6.0

ChangeLog: memory plugin: do not account reclaimable slab as used.

Align this counter with free(1).
https://gitlab.com/procps-ng/procps/-/blob/v3.3.17/proc/sysinfo.c#L789

2 years ago[collectd 6] port Report MemAvailable when present in meminfo (#3916)
Leonard Göhrs [Tue, 21 Sep 2021 06:32:57 +0000 (08:32 +0200)] 
[collectd 6] port Report MemAvailable when present in meminfo (#3916)

This ports "Report MemAvailable when present in meminfo (#3916)" (848b2394dc2)
from the main branch to collectd-6.0.

2 years ago[collectd 6] chrony: migration to v6.0
Manuel Luis Sanmartín Rozada [Tue, 26 Jan 2021 23:53:00 +0000 (00:53 +0100)] 
[collectd 6] chrony: migration to v6.0

2 years agogpu_sysman: Add fabric port metric type and related metrics support
Eero Tamminen [Thu, 17 Feb 2022 18:02:50 +0000 (20:02 +0200)] 
gpu_sysman: Add fabric port metric type and related metrics support

Already in L0 spec v1.0.

Fabric ports have a lot of properties which means lot of labels and
extra functions.  This required increasing some things in the test
code too.

2 years ago[collectd 6] disable the same plugins in github worker as in cirrus CI
Leonard Göhrs [Mon, 30 Jan 2023 14:06:09 +0000 (15:06 +0100)] 
[collectd 6] disable the same plugins in github worker as in cirrus CI

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years agocirrus: drop CentOS6
Kentaro Hayashi [Sat, 5 Dec 2020 04:05:57 +0000 (13:05 +0900)] 
cirrus: drop CentOS6

It had been reached EOL at November 30th, 2020.
There is no security updates available anymore.

ref. https://wiki.centos.org/About/Product

Signed-off-by: Kentaro Hayashi <hayashi@clear-code.com>
2 years ago[collectd 6] trigger github worker on collectd-6.0 branch
Leonard Göhrs [Mon, 30 Jan 2023 13:09:28 +0000 (14:09 +0100)] 
[collectd 6] trigger github worker on collectd-6.0 branch

Signed-off-by: Leonard Göhrs <l.goehrs@pengutronix.de>
2 years agoUpdate build.yml
Alex [Mon, 29 Aug 2022 18:57:45 +0000 (19:57 +0100)] 
Update build.yml

Signed-off-by: sashashura <93376818+sashashura@users.noreply.github.com>
2 years ago[gha] Add el9_x86_64 to tested distros
Emma Foley [Thu, 9 Jun 2022 17:43:46 +0000 (18:43 +0100)] 
[gha] Add el9_x86_64 to tested distros

2 years ago[CI] Update Fedora versions used in GHA
Emma Foley [Thu, 9 Jun 2022 13:49:26 +0000 (14:49 +0100)] 
[CI] Update Fedora versions used in GHA

2 years ago[ci][gha] Miscellaneous improvements and sync with cirrus (#3976)
Emma Foley [Tue, 15 Feb 2022 14:14:15 +0000 (14:14 +0000)] 
[ci][gha] Miscellaneous improvements and sync with cirrus (#3976)

* [ci][gha] Rename tasks

* [ci][gha] Update and use MAKEFLAGS

* [ci][gha] Remove continue-on-error from ``make distcheck`` tasks

Installation of bzip2 and make distcheck were failing on el8.
This was resolved by updating it to use CentOS Stream 8 in [1]

[1] https://github.com/collectd/ci-docker/pull/55