sources: disk: Create a metrics object and populate as we go

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

metrics: Add support for bool

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Convert all sources to submit metrics the new way

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

metrics: Create a shorthand to submit stats with less code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Implement parsing the entire file as JSON object

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Add a new metrics object

When we have larger RRD files, it will become increasingly difficult to
serialize the format strings. This new object is supposed to store
metrics more efficiently and with more metadata.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

jenkins: Install libudev-dev on Debian-based machines

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Release memory when the system is under pressure

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: disk: Read SMART data if available

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

source: Implement actually migrating databases

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

source: Check if RRD files need migration

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Add source to collect disk IO stats

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Return the error code if we could not read the file

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Connect to udev

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

colors: Define a special color for rx/tx traffic

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Add a graph to show rate information

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Correct rate information of hostapd stations

hostapd returns this in hundredth of kilobits. We will store it in bits.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Try to make the signal graph a bit more colourful

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Add bandwidth graphs for hostapd stations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Rename hostapd station signal graph

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

po: Add telemetry.po to .gitignore

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: hostapd: Add graph that shows the station signal

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Add some more macros

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Take VDEFs from the graphs macros

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

command: Cleanup events before closing file descriptors

This makes the trace slightly cleaner when FDs will be removed from the
event loop before they are closed and not the other way round.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

command: Defer cleaning up the command to avoid zombies

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Simplify headers by removing option to pass extra arguments

We should keep it so that the macros print simple things and we won't
have to deal with the extra arguments.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Add automatic field composition for objects

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: hostapd: Don't use colons

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

queue: Don't allow colons in object names

This breaks generating graphs where colons are used as delimiter.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: hostapd: Zero-pad MAC addresses

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

client: Pass the object in the options array

This makes it all a little bit easier

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Add source for hostapd clients

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Only finish parsing when we have found enough fields

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: There is no need to attempt to parse empty lines

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Add function to run the parser on a single line

This is useful if we want to use the parser when we cannot walk through
the file line by line.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Allow to select sources on the CLI

This is only useful for testing to basically disable all other sources
for better log readability.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Properly turn off the event when we disable a source

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

po: Update translations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Rename the entire project to "IPFire Telemetry"

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

source: Disable all sources that fail all the time

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Always show the top of the memory graph

I am not sure whether this is what I actually want but some members on
the team think that it should  be this way. So I will commit this for
now and maybe later revert it again.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: df: Cleanup /proc/self/mounts on error

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

command: Close the pidfd

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Reset the file pointer on error

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Another attempt to make the static analyzer happy

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Try to make the static analyzer happy

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Add a memory graph

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

proc: Also convert total memory to bytes

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

queue: Flush all data when we want to draw a graph

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Write title/vlabel to the stack

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

po: Update translations

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

configure: Add option to enable tests

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Remove the old file walking functions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

proc: Use the new parser to parse PSI information

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Implement parsing multiple values per line

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: unbound: Cleanup some code

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

command: Use the buffer implementation for internal buffering

In order to use the file parser, we convert the output of the command to
a file object which then provides a nice and simple parser.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Allow to open a buffer as a file object

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Fix format string check

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

buffer: Add a buffer implementation to write larger data to

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Build a simple parser

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Add a function to read a single integer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Rename collect() to heartbeat()

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

parse: Add custom functions to parse token and integers

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Add function to strip non-whitespace

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Include <stdarg.h>

This was required for a while and when not included will break the
build.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Add function to copy strings of a set input length

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

proc: Refatcor reading /proc/softirqs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

file: Add an abstraction to read files

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

loadavg: Make the graph make sense

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Make it easier to compute custom CDEFs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

source: Remove stepsize

We can fetch this from the heartbeat which is easier to understand in
its function.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

source: Stagger sources to avoid load peaks

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

command: Kill commands after 30 seconds

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Add a source for Unbound stats

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

command: Add a helper object to run a command

This is designed to run any shell commands and parse the output. This
has to be done asynchronously so that we won't block the event loop.
That slightly adds some complications.

The command will be executed and events will be installed with the event
loop to read any stdout/stderr and store it as well as receive a signal
when the process terminates.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Add a helper function to walk through a buffer

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Mask SIGCHLD

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Add a helper function to iterate through file handles

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

proto: Reduce the amount of logging

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

queue: Actually store the object in the queue

This used to never happen which is why sources with multiple objects had
their data written into the main RRD file.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

graphs: Add uptime graph

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Check format strings

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Add source for the system's uptime

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Make all functions static

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Add formatting functions

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

string: Add our string comparison function

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

util: Add macros for branch prediction

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Add softirqs

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

queue: Fail if the object contains invalid characters

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Add sources for IPv4 fragmentation information

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

Add source for memory usage

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

daemon: Use macros to convert time

That way, I will hopefully not get the maths wrong all the time.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

source: Complain if the heartbeat was called too late

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

source: Complain if something stalls the event loop

This source will also be de-prioritized so that other sources can work
better.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Add a test that stalls the event loop

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

source: Lower the heartbeat if a source is in error state

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Store result codes and check for patterns

That way we can disable any sources which constantly fail and we might
as well detect flapping.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>

sources: Add some tests

Oone constantly returns an error, the other one returns an error with a
probability of 50%. They are to test the internal error handling.

Signed-off-by: Michael Tremer <michael.tremer@ipfire.org>