Willy Tarreau [Sun, 9 May 2021 04:25:16 +0000 (06:25 +0200)]
DOC: remove last occurrences of "HA-Proxy" syntax
There were only a few more used as output examples and comments in a few
docs, it was the right moment to get rid of them. The file intro.txt
which explains how to parse the version also got a hint about the possible
presence of a hyphen in the name in older versions.
Willy Tarreau [Sun, 9 May 2021 04:14:25 +0000 (06:14 +0200)]
MINOR: version: report "HAProxy" not "HA-Proxy" in the version output
When running "haproxy -v", we still get "HA-Proxy" which is the last
place where this confusing oddity happens. Being so used to it I didn't
even notice it until it was reported to me just after 2.2 but it never
got fixed, despite the PRODUCT_NAME macro that is used to report the
name in the stats page and in "show info" being already set to "HAProxy"
15 years ago in 1.2.14 with commit e03312613. It's about time to
uniformize everything.
Daniel Corbett [Sat, 8 May 2021 14:50:37 +0000 (10:50 -0400)]
DOC: Fix a few grammar/spelling issues and casing of HAProxy
This patch fixes a few grammar and spelling issues in configuration.txt.
It was also noted that there was a wide range of case usage
(i.e. haproxy, HAproxy, HAProxy, etc... ). This patch updates them
all to be consistently "HAProxy" except where a binary is mentioned.
Willy Tarreau [Sat, 8 May 2021 21:33:25 +0000 (23:33 +0200)]
BUILD: makefile: reorder object files by build time
This is the per-release reordering to improve build parallelism.
It didn't change much, mostly dns+resolvers inflated this time.
Nowadays build times are mostly dominated by the long dependencies
of include files, no less than 170MB of preprocessed code has to be
built, and half of this is SSL support is disabled. Includes should
likely be reworked to be smaller with less dependencies each,
possibly splitting what's the core of each of them and what is used
to interface with other ones. Each split of a .C file in two adds
0.3s of build time just because of this.
Willy Tarreau [Sat, 8 May 2021 18:14:39 +0000 (20:14 +0200)]
BUILD: connection: stop including listener-t.h
listener-t comes with openssl just due to the SSL_CTX type that is
declred as a typedef in openssl hence cannot be abstracted at this
level. However connection-t.h doen't need all that just to know that
bind_conf is a struct. Let's declare it with other external types
instead..
Willy Tarreau [Sat, 8 May 2021 18:10:13 +0000 (20:10 +0200)]
MINOR: task: stop including stream.h from task.c
This one comes with a very deep dependency hell, only to know that
process_stream() is a function. Dropping it reduces the preprocessed
output from 1.5MB to 640kB.
Willy Tarreau [Sat, 8 May 2021 17:58:37 +0000 (19:58 +0200)]
REORG: config: uninline warnifnotcap() and failifnotcap()
These ones are used by virtually every config parser. Not only they
provide no benefit in being inlined, but they imply a very deep
dependency starting at proxy.h, which results for example in task.c
including openssl.
Willy Tarreau [Sat, 8 May 2021 12:12:47 +0000 (14:12 +0200)]
REORG: stick-table: move composite address functions to stick_table.h
These caddr_* functions were once placed into tools.h in the hope they
would be useful but nobody knows they exist. They could deserve being
moved to their own file with other pointer manipulation functions maybe,
but for now they're the only reason left for stick_table.h to include
tools.h, so let's move them directly there since it's its only user.
This allows to remove tools.h from stick_table.h and slightly reduce
the overall build time.
This function has no business being inlined in stick_table.h since it's
only used at boot time by the config parser. In addition it causes an
undesired dependency on tools.h because it uses parse_time_err(). Let's
move it to stick_table.c.
Willy Tarreau [Sat, 8 May 2021 12:06:09 +0000 (14:06 +0200)]
BUILD: connection: move list_mux_proto() to connection.c
No idea why this was put inlined into connection.h, it's used only once
for haproxy -vv, and requires tools.h, causing an undesired dependency
from connection.h. Let's move it to connection.c instead where it ought
to have been.
Willy Tarreau [Sat, 8 May 2021 10:30:50 +0000 (12:30 +0200)]
REORG: mworker: move proc_self from global to mworker
Only mworker uses proc_self, and it was declared in global.h, forcing
users of global.h to include mworker and its dependencies.
Moving it to mworker reduces the preprocessed size of version.c from
170 to 125kB by shrinking the number of local includes from 30 to 16
and the number of system includes from 147 to 132.
Willy Tarreau [Sat, 8 May 2021 09:41:28 +0000 (11:41 +0200)]
REORG: vars: move the "proc" scope variables out of the global struct
The presence of this field causes a long dependency chain because almost
everyone includes global-t.h, and vars include sample_data which include
some system includes as well as HTTP parts.
There is absolutely no reason for having the process-wide variables in
the global struct, let's just move them into vars.c and vars.h. This
reduces from ~190k to ~170k the preprocessed output of version.c.
Willy Tarreau [Sat, 8 May 2021 09:06:32 +0000 (11:06 +0200)]
MINOR: config: mark tune.fd.edge-triggered as experimental
This one is stated as experimental in the doc but could still be used
by accidental copy-paste. Let's mark it with KWF_EXPERIMENTAL so that
users have to opt-in to use it.
Willy Tarreau [Sat, 8 May 2021 06:14:04 +0000 (08:14 +0200)]
MINOR: stats: make "show info" able to report rates as floats when asked
Now "show info float" will also report SSL rates, connection rates and
key reuse ratios as floats. This can be convenient at very low rates.
Note that the SSL reuse ratio which used to commonly oscillate between
0 and 1 under load is now more often above zero with small values. It
indicates that for better stability we shouldn't be comparing a key rate
with a connection rate but instead we should measure the reuse rate at
its source.
Willy Tarreau [Sat, 8 May 2021 05:40:52 +0000 (07:40 +0200)]
MINOR: stats: use tv_remain() to precisely compute the uptime
We'll have to support reporting sub-second uptimes, so let's use the
appropriate function which will automatically adjust the tv_usec field.
In addition to this, it will also report a more accurate uptime thanks
to considering the sub-second part in the result.
Willy Tarreau [Sat, 8 May 2021 05:54:24 +0000 (07:54 +0200)]
MINOR: stats: support an optional "float" option to "show info"
This will allow some fields to be produced with a higher accuracy when
the requester indicates being able to parse floats. Rates and times are
among the elements which can make sense.
Willy Tarreau [Sat, 8 May 2021 05:43:53 +0000 (07:43 +0200)]
MINOR: stats: pass the appctx flags to stats_fill_info()
Currently the stats filling function knows nothing about the caller's
needs, so let's pass the STAT_* flags so that it can adapt to the
requester's constraints.
Willy Tarreau [Sat, 8 May 2021 05:37:38 +0000 (07:37 +0200)]
MINOR: stats: add the HTML conversion for float types
For the prometheus exporter, a new float type was added for the fields
and its conversion was added everywhere except for the HTML output.
Now that we have F2H() we can implement it for consistency.
Willy Tarreau [Sat, 8 May 2021 08:38:20 +0000 (10:38 +0200)]
MINOR: stats: avoid excessive padding of float values with trailing zeroes
When emitting stats, we don't need to have 6 zeroes after the decimal point
for each value, so let's trim floating point numbers to the longest needed
only.
Willy Tarreau [Sat, 8 May 2021 06:12:37 +0000 (08:12 +0200)]
MINOR: freq_ctr: add new functions to report float measurements
For stats reporting it can be convenient to report floats at low rates
instead of discrete integers. We do have quite some precision since we
currently divide counters by number of milliseconds, so we can usually
add 3 digits after the decimal point.
Willy Tarreau [Sat, 8 May 2021 05:35:00 +0000 (07:35 +0200)]
MINOR: tools: add a float-to-ascii conversion function
We already had ultoa_r() and friends but nothing to emit inline floats.
This is now done with ftoa_r() and F2A/F2H. Note that the latter both use
the itoa_str[] as temporary storage and that the HTML format currently is
the exact same as the ASCII one. The trailing zeroes are always timmed so
these outputs are usable in user-visible output.
Willy Tarreau [Sat, 8 May 2021 08:28:53 +0000 (10:28 +0200)]
MINOR: tools: implement trimming of floating point numbers
When using "%f" to print a float, it automatically gets 6 digits after
the decimal point and there's no way to automatically adjust to the
required ones by dropping trailing zeroes. This function does exactly
this and automatically drops the decimal point if all digits after it
were zeroes. This will make numbers more friendly in stats and makes
outputs shorter (e.g. JSON where everything is just a "number").
The function is designed to be easy to use with snprint() and chunks:
Willy Tarreau [Sat, 8 May 2021 04:50:28 +0000 (06:50 +0200)]
MINOR: sample: improve error reporting on missing arg to strcmp() converter
Calling the strcmp() converter with no argument yields this strange error:
[ALERT] (31439) : parsing [test.cfg:3] : error detected in frontend 'f' while parsing 'http-request redirect' rule : failed to parse sample expression <src,strcmp]> : invalid args in converter 'strcmp' : failed to register variable name ''.
This is because the vars name check tries to see if it can create such a
variable having an empty name. Let's at least make a special case of the
missing argument. Now we can read a more explicit:
[ALERT] (31655) : parsing [test.cfg:3] : error detected in frontend 'f' while parsing 'http-request redirect' rule : failed to parse sample expression <src,strcmp]> : invalid args in converter 'strcmp' : missing variable name.
When using the crl-file option with multiple Certificate Authority
levels in the CA chain, there must be one CRL per CA or the verify
function on the backend side will raise an "unagle to get certificate
CRL" error (error code 3).
DOC: ssl: Extra files loading now works for backends too
When implementing the server side certificate hot update, the ckch
mechanism was used on the backend side in order to mimic the frontend
certificate management and to enable server line certificate update via
the CLI (see GitHub issue #427). As an unexpected side effect, we now
also look for ssl extra files (cert.pem.key, cert.pem.ocsp ...) for the
backend side.
This patch updates the documentation accordingly.
Add a new proxy capability for proxy with load-balancing capabilities.
This help to differentiate listen/frontend/backend with special proxies
such as peer proxies.
MINOR: http_act: mark normalize-uri as experimental
normalize-uri http rule is marked as experimental, so it cannot be
activated without the global 'expose-experimental-directives'. The
associated vtc is updated to be able to use it.
Support experimental actions. It is mandatory to use
'expose-experimental-directives' before to be able to use them.
If such action is present in the config file, the tainted status of the
process is updated. Another tainted status is set when an experimental
action is executed.
Add a new flag to mark a keyword as experimental. An experimental
keyword cannot be used if the global 'expose-experimental-directives' is
not present first.
Only keywords parsed through a standard cfg_keywords lists in
global/proxies section will be automatically detected if declared
experimental. To support a keyword outside of these lists,
check_kw_experimental must be called manually during its parsing.
If an experimental keyword is present in the config, the tainted flag is
updated.
For the moment, no keyword is marked as experimental.
MINOR: cfgparse: add a new field flags in cfg_keyword
This field will be used to add various mechanism to config parsing.
Currently no flag value is implemented. The following commit will
implement experimental keywords.
Add a global flag named 'tainted'. Its purpose is to report various
status about experimental features used for the current process
lifetime.
By default it is initialized to 0. It can be set/retrieve by a couple of
new functions mark_tainted()/get_tainted(). Once a flag is set, it
cannot be resetted.
Currently, no tainted status is implemented, it will be the subject of
the following commits.
BUG/MINOR: checks: Reschedule check on observe mode only if fastinter is set
On observe mode, if a server is marked as DOWN, the server's health-check is
rescheduled using the fastinter timeout if the new expiration date is newer
that the current one. But this must only be performed if the fastinter
timeout is defined.
Internally, tick_is_lt() function only checks the date and does not perform any
verification on the provided args. Thus, we must take care of it. However, it is
possible to disable the server health-check by setting its task expiration date
to TICK_ETERNITY.
This patch must be backported as far as 2.2. It is related to
BUG/MINOR: checks: Handle synchronous connect when a tcpcheck is started
A connection may be synchronously established. In the tcpcheck context, it
may be a problem if several connections come one after another. In this
case, there is no event to close the very first connection before starting
the next one. The checks is thus blocked and timed out, a L7 timeout error
is reported.
To fix the bug, when a tcpcheck is started, we immediately evaluate its
state. Most of time, nothing is performed and we must wait. But it is thus
possible to handle the result of a successfull connection.
This patch should fix the issue #1234. It must be backported as far as 2.2.
BUG/MINOR: stream: Reset stream final state and si error type on L7 retry
Thanks to a previous fix, the stream error mask is now cleared on L7
retry. But the stream final state (SF_FINST_*) and the stream-interface
error type must also be reset to properly restart a new connection and be
sure to not inherit errors from the previous connection attempt.
In addition, SF_ADDR_SET flag is not systematically removed.
stream_choose_redispatch() already takes care to unset it if necessary. When
the connection is not redispatch, the server address can be preserved.
Willy Tarreau [Fri, 7 May 2021 09:38:37 +0000 (11:38 +0200)]
CLEANUP: cli/tree-wide: properly re-align the CLI commands' help messages
There were 102 CLI commands whose help were zig-zagging all along the dump
making them unreadable. This patch realigns all these messages so that the
command now uses up to 40 characters before the delimiting colon. About a
third of the commands did not correctly list their arguments which were
added after the first version, so they were all updated. Some abuses of
the term "id" were fixed to use a more explanatory term. The
"set ssl ocsp-response" command was not listed because it lacked a help
message, this was fixed as well. The deprecated enable/disable commands
for agent/health/server were prominently written as deprecated. Whenever
possible, clearer explanations were provided.
Willy Tarreau [Fri, 7 May 2021 06:59:50 +0000 (08:59 +0200)]
MINOR: config: add a new message directive: .diag
This one works just like .notice/.warning/.alert except that it prints
the message at level "DIAG" only when haproxy runs in diagnostic mode
(-dD). This can be convenient for example to pass a few hints to help
locate certain config parts or to leave messages about certain temporary
workarounds.
Example:
.diag "WTA/2021-05-07: $.LINE: replace 'redirect' with 'return' after final switch to 2.4"
http-request redirect location /goaway if ABUSE
Willy Tarreau [Fri, 7 May 2021 06:42:39 +0000 (08:42 +0200)]
MEDIUM: log: slightly refine the output format of alerts/warnings/etc
For about 20 years we've been emitting cryptic messages on warnings and
alerts, that nobody knows how to parse:
[NOTICE] 126/080118 (3115) : haproxy version is 2.4-dev18-0b7c78-49
[NOTICE] 126/080118 (3115) : path to executable is ./haproxy
[WARNING] 126/080119 (3115) : Server default/srv1 is DOWN via static/srv1. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] 126/080119 (3115) : backend 'default' has no server available!
Hint: the first 3-digit number is the day of year, and the 6 digits
after it represent the time of day in format HHMMSS, then the pid in
parenthesis. These are not quite user-friendly and such cryptic into
are not useful at all.
This patch slightly adjusts the output by performing these minimal changes:
- removing the date/time, as they were added very early when haproxy
was meant to be used in foreground as a debugging tool, and they're
provided in more details in logs nowadays ;
- better aligning the fields by padding the severity tag to 10 chars.
The diag output was renamed to "DIAG" only.
Now the output provides this:
[NOTICE] (4563) : haproxy version is 2.4-dev18-75a428-51
[NOTICE] (4563) : path to executable is ./haproxy
[WARNING] (4563) : Server default/srv1 is DOWN via static/srv1. 0 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.
[ALERT] (4563) : backend 'default' has no server available!
The useless space before the colon was kept so as not to confuse any
possible output parser.
The few entries in the doc referring to this format were adjusted to
reflect the new one.
The change was tagged "MEDIUM" as it may have visible consequences on
home-grown monitoring tools, though it is extremely unlikely due to the
limited extent of these changes.
Willy Tarreau [Fri, 7 May 2021 06:19:30 +0000 (08:19 +0200)]
BUG/MINOR: stream: properly clear the previous error mask on L7 retries
The cleanup of the previous error was incorrect on L7 retries, it would
OR two values while they're part of an enum, leaving some bits set.
Depending on the errors it was possible to occasionally see an internal
error ("I" flag) being logged.
This should be backported as far as 2.0, though the do_l7_retry() function
in in proto_htx.c in older versions.
Willy Tarreau [Fri, 7 May 2021 06:01:35 +0000 (08:01 +0200)]
BUG/MINOR: activity: use the new pointer to calculate the new size in realloc()
When memory profiling is enabled, realloc() can occasionally get the area
size wrong due to the wrong pointer being used to check the new size. When
the old area gets unmapped in the operation, this may even result in a
crash. There's no impact without memory profiling though.
No backport is needed as this is exclusively 2.4-dev.
Willy Tarreau [Thu, 6 May 2021 14:53:26 +0000 (16:53 +0200)]
MINOR: config: add predicates "version_atleast" and "version_before" to cond blocks
These predicates respectively verify that the current version is at least
a given version or is before a specific one. The syntax is exactly the one
reported by "haproxy -v", though each component is optional, so both "1.5"
and "2.4-dev18-88910-48" are supported. Missing components equal zero, and
"dev" is below "pre" or "rc", which are both inferior to no such mention
(i.e. they are negative). Thus "2.4-dev18" is older than "2.4-rc1" which
is older than "2.4".
Willy Tarreau [Thu, 6 May 2021 14:34:23 +0000 (16:34 +0200)]
MINOR: config: add predicate "feature" to detect certain built-in features
The "feature(name)" predicate will return true if <name> corresponds to
a name listed after a '+' in the features list, that is it was enabled at
build time with USE_<name>=1. Typical use cases will include OPENSSL, LUA
and LINUX_SPLICE. But maybe it will also be convenient to use with optional
addons such as PROMEX and the device detection modules to help keeping the
same configs across various deployments.
Willy Tarreau [Thu, 6 May 2021 14:10:09 +0000 (16:10 +0200)]
MINOR: config: add predicates "streq()" and "strneq()" to conditional expressions
"streq(str1,str2)" will return true if the two strings match while
"strneq(str1,str2)" will return true only if they differ. This is
convenient to match an environment variable against a predefined value.
Willy Tarreau [Thu, 6 May 2021 13:49:04 +0000 (15:49 +0200)]
MINOR: config: make cfg_eval_condition() support predicates with arguments
Now we can look up a list of known predicates and pre-parse their
arguments. For now the list is empty. The code needed to be arranged with
a common exit point to release all arguments because there's no default
argument freeing function (it likely only used to exist in the deinit
code). Since we only support simple arguments for now it's no big deal,
only a 2-liner loop.
Let's return the position of the first unparsable character on error,
so that instead of just saying "unparsable conditional expression blah"
we can have:
[ALERT] 125/150618 (13995) : parsing [test-conds2.cfg:1]: unparsable conditional expression '12/blah' in '.if' at position 1:
.if 12/blah
^
This is important because conditions will be made from environment
variables or later from more complex expressions where the error will
not always be easy to locate.
Willy Tarreau [Thu, 6 May 2021 05:43:35 +0000 (07:43 +0200)]
MINOR: global: add version comparison functions
The new function split_version() converts a parsable haproxy version to
an array of integers. The function compare_current_version() compares an
arbitrary version to the current one. These two functions were written
by Thierry Fournier in 2013, and are still usable as-is. They will be
used to write config language predicates.
Willy Tarreau [Thu, 6 May 2021 14:30:32 +0000 (16:30 +0200)]
MINOR: global: export the build features string list
Till now it was only presented in the version output but could not be
consulted outside of haproxy.c, let's export it as a variable, and set
it to an empty string if not defined.
Willy Tarreau [Thu, 6 May 2021 12:50:30 +0000 (14:50 +0200)]
MINOR: arg: improve the error message on missing closing parenthesis
When the closing brace is missing after an argument (acl, ...), the
error may report something like "expected ')' before ''". Let's just
drop "before ''" when the final word is empty to make the message a
bit clearer.
Willy Tarreau [Thu, 6 May 2021 08:25:11 +0000 (10:25 +0200)]
MINOR: config: support some pseudo-variables for file/line/section
The new pseudo-variables ".FILE", ".LINE" and ".SECTION" will be resolved
on the fly by the config parser and will respectively retrieve the current
configuration file name, the current line number and the current section
being parsed. This may help emit logs, errors, and debugging information
(e.g. which rule matched).
The '.' in the first char was reserved for such pseudo-variables and no
other variable is permitted. This will allow to add support for new ones
in the future if they prove to be useful (e.g. randoms/uuid for secret
keying or automatic naming of configuration objects).