MINOR: haload: support HTTP status code by version
Add support for tracking and displaying HTTP status code distribution
broken down by HTTP version (h0, h1, h2, h3) via the new -hsv option.
Building upon -hs, -hsv retains the global status code distribution and
appends additional columns detailed by HTTP version.
Each URL configuration (struct hld_url_cfg) stores its targeted HTTP version in
->http_ver. Thread statistics track status code counts per HTTP
version in thrs_info.vtot_sc[][], which are then aggregated and displayed
alongside global status codes in hld_summary().
Also remove deprecated --show-status-codes, which is a too long long option,
in favor of -hs / -hsv.
BUG/MINOR: haload: fix stale global variables affecting URL allocations
hld_alloc_url() relied on global variables (alpn, h2c) set during
command-line parsing. When processing multiple URLs, values retained in
these globals by previous URLs persisted and affected subsequent
allocations. This corrupted the configuration of newly allocated
URLs—especially QUIC URLs, which require "h3" ALPN and no h2c—causing
connection failures.
Fix this by passing is_quic, ALPN, and h2c explicitly as parameters
from haproxy_init_args() to hld_alloc_url(), ensuring each URL is
constructed with its appropriate parameters instead of inheriting
stale global state.
BUG/MEDIUM: filter: Disable auto-close on channel during TCP payload filtering
While a filter is registered on the payload filter in TCP, it is important
to disable the auto-cloes on the corresponding channel. It is already
performed in HTTP. It is only an issue when the ->tcp_payload callback
function is defined.
Without this patch, the data blocked in a filter may be lost when the
producer shut its connection. In that case, we must take care to wait the
filter flushes pending data.
This patch should fix the issue #3457. It must be backported to all
supported versions.
Add a section for the new 'clear counters server <backend>/<server>
[force]' CLI command in doc/management.txt, alongside 'clear counters'
and 'clear counters all'.
The entry documents the admin-only permission level (it clears
accumulated counters, like 'clear counters all') and the fact that both
accumulated counters and max values are reset, along with the clearable
module-registered extra counters, while the server's runtime state
(address, weight, admin state) and health checks are left untouched. It
describes the optional 'force' argument required to clear counters
stored in a shared-memory stats file, and calls out the Kubernetes
slot-recycle use case that motivated the command (per-entity
attribution after 'set server <b>/<s> name').
REGTESTS: stats: add test for 'clear counters server'
Cover the new 'clear counters server <backend>/<server>' CLI command:
- error cases:
* unknown backend
* unknown server
- success case: drive two requests pinned to a single server (the
second server is a backup, so it stays idle), verify via 'show stat'
that the served server accumulated sessions while the backup stayed
at zero, clear only the served server's counters, then verify its
cumulative sessions dropped back to zero while the backup is still
untouched.
- the optional 'force' argument is accepted (a no-op override here,
since these process-local counters are not backed by a shared-memory
stats file) and the command is discoverable via 'help'.
The success path also implicitly exercises the counters_be_reset()
helper: if the reset were unsafe against the shared.tg pointer, the
vtest run would segfault on the next scheduled scrape or health check.
MINOR: server: add 'clear counters server <backend>/<server>' CLI command
Add a CLI command to reset the statistics counters of a single server:
clear counters server <backend>/<server> [force]
Motivation: 'clear counters all' resets counters across every proxy and
server in the process, which is too blunt for common operational needs.
In particular, when a server slot is being reused to represent a
different logical entity (e.g. a different Kubernetes pod occupying the
same slot after 'set server <b>/<s> name'), the operator needs per-slot
counter attribution and cannot afford to wipe the entire process' stats.
The command requires ACCESS_LVL_ADMIN, like 'clear counters all'. It
resets cumulative counters (not just the max gauges cleared by the base
'clear counters'), so restricting it to admin prevents an operator-level
socket from hiding accumulated activity. Like 'clear counters' / 'clear
counters all', it is not gated by the server's administrative state: the
reset only zeroes counter values and does not touch the server object or
its runtime state, so it is safe to issue on a live server (a concurrent
counter increment races on a value exactly as it already does for 'clear
counters all').
When the server's counters are registered in a shared-memory stats file
object (COUNTERS_SHARED_F_LOCAL not set), clearing them breaks the
monotonicity that monitoring tools consuming the shared stats rely on
and affects every process attached to the object. Such a clear is
therefore refused unless the optional 'force' argument is given.
The reset covers both the native server counters, via counters_be_reset()
introduced earlier in this series (which safely zeroes the per-tgroup
accumulated counter contents while preserving the shared.tg pointer
array), and the module-registered extra counters, via the new
srv_stats_clear_extra_counters() helper.
Note on the dispatch: 'clear counters' is a two-word CLI keyword, and
cli_find_kw() matches the first keyword whose tokens are all consumed,
so a separate three-word 'clear counters server' keyword would be
shadowed (or would shadow) depending on registration order. To avoid
that ambiguity the new sub-command is dispatched from within
cli_parse_clear_counters() when args[2] == "server", mirroring how the
existing "all" argument is handled. The per-server worker lives in
server.c (cli_clear_counters_server()) where the server lookup and
lock helpers are available.
MINOR: counters: add max-only reset helpers and use them for clear counters
The plain "clear counters" command (OPER level) resets only the max/peak
gauges of each object's counters, leaving cumulative counters intact,
while "clear counters all" (ADMIN) performs a full reset via the
counters_{fe,be}_reset() helpers introduced earlier in this series.
The max-only reset was open-coded inline in proxy_stats_clear_counters()
with a hand-maintained list of fields, duplicated across the backend,
server and listener branches. This is fragile: a new max gauge added to
struct {fe,be}_counters can silently be forgotten here.
Factor the max-only reset into counters_fe_reset_max() and
counters_be_reset_max(), living next to the existing reset helpers, and
call them from proxy_stats_clear_counters(). Unlike the full-reset
helpers these cannot use a blanket memset because the max fields are
interleaved with live cumulative fields, so the gauges are zeroed
individually.
While consolidating, the two open-coded lists turned out to disagree:
the backend branch cleared conn_max / cps_max / rps_max but not
cur_sess_max, while the server branch cleared cur_sess_max but none of
the former. Both used the same struct be_counters, so these were latent
gaps rather than intentional differences. counters_be_reset_max() now
clears the union of all peak gauges, so plain "clear counters"
consistently resets every max gauge for both backends and servers.
No functional change for the full-reset ("all") path.
BUG/MEDIUM: counters: preserve shared.tg pointer on 'clear counters all'
proxy_stats_clear_counters() did a blanket memset() on the be_counters /
fe_counters / listener counter structs when clearing all counters. This
zeroed the embedded shared.tg pointer array, which every hot path
dereferences with no NULL check (for example
_HA_ATOMIC_INC(&srv->counters.shared.tg[tgid-1]->cum_lbconn) in
process_srv_queue()). After 'clear counters all' the next request on
that server would segfault.
The memset pattern predates the switch to per-thread-group shared
counters (commit 5495c8844 "MEDIUM: counters: Dynamically allocate
per-thread group counters") and wasn't updated when shared.tg gained
pointer-array semantics. Additionally, memset'ing only the outer struct
never actually reset the accumulated counters in the pointed-at
per-tgroup structs, so 'clear counters all' silently failed to match
its documented "same effect as restarting" behaviour for cumulative
counters (bytes, sessions, requests, ...).
Both iterate shared.tg[0 .. nbtgroups-1] and zero the *contents* of each
per-tgroup struct via memset, then zero the local (non-shared) fields of
the outer struct. The shared.tg pointer array, each shared.tg[it]
pointer, and shared.flags (COUNTERS_SHARED_F_LOCAL is set at boot and
reflects allocation ownership, not counter state) are preserved.
Refactor proxy_stats_clear_counters() to call these helpers instead of
the inline memset. This fixes both the segfault after 'clear counters
all' and the silent no-op on cumulative counters, and provides a shared
primitive for a subsequent 'clear counters server <b>/<s>' command.
In SHM stats-file mode, zeroing the per-tgroup structs affects every
process attached to the same object; this matches the intended
"reset everywhere" semantic of clear counters and is unchanged from
the outer-memset intent.
This bug should be backported wherever the per-thread-group shared
counter refactor is present.
BUG/MINOR: ech: reject an ECH store with no usable private key
OSSL_ECHSTORE_read_pem() accepts a PEM file containing only an
ECHConfig with no private key. OSSL_ECHSTORE_num_keys() returns 1
on success regardless of the count it writes back, so the existing
"!= 1" check only ever caught the call itself failing, never a
resulting count of zero: a directory containing only such
config-only ".ech" files loaded "successfully" and got installed via
SSL_CTX_set1_echstore(), while unable to decrypt a single
ECH-protected ClientHello, with nothing pointing at the cause.
BUG/MINOR: ech: propagate error from load_echkeys()
load_echkeys() was not emitting any error messagw, so a failed load only
ever produced "failed to load ECH keys". Add a char **err parameter to
the function so it can emit TLS library errors or system errors.
BUILD: listener: Fix the build on platforms without MSG_CMSG_CLOEXEC
Some OSes, such as macos, do not provide MSG_CMSG_CLOEXEC, so provide a
fallback mechanism using fdcntl(FD_CLOEXEC) instead.
It doesn't matter right now, that code will never be used on such OS,
because you can't use per-thread-group fd tables in there, but maybe it
will change one day.
Add a new experimental option, tune.fd.tables, that can be set to
either "shared" (the default) or "per-thread-group", to enable
per-thread group file descriptor tables.
This can give a nice performance boost when running with a lot of
threads, as we observe a lot of contention on the file descriptor table
lock in the kernel.
MINOR: connection: Do not retrieve src/dst on another thread group's FD
conn_get_src() and conn_get_dst() lazily fetch the connection's
addresses with getsockname()/getpeername() when they were not known
yet. They may be called on a foreign connection by observability code
running on any thread, for example "show peers", which dumps the peers
sessions' connections from whatever thread serves the CLI. With
per-thread-group FD tables, the FD of a connection owned by another
group is not usable from the calling thread.
So just fail if we attempt to access a connection that is owned by
another thread group.
MEDIUM: quic: Do not use another thread group's listener FD
A datagram may be read by a thread of another group than the one owning
the connection it is for, and with per-thread-group file descriptor
tables, we can't assume that we can use that file descriptor, so find a
more fitting one if needed.
MEDIUM: resolvers: Do not close another thread group's socket at deinit
resolvers_destroy() runs at deinit, on the last remaining thread, and
closes the UDP nameserver sockets, which were created by the thread
their resolvers section's task was pinned to. With per-thread-group FD
tables, when that thread belongs to another group, its group's FD
table died with its last thread and already closed the socket, and the
stored number is only meaningful in that table, closing it from here
would actually close whatever FD carries the same number in this
thread's own table. So in that case, leave the FD alone.
MEDIUM: server: Do not close other thread groups' connections at deinit
srv_close_idle_conns() runs at deinit, on the last remaining thread,
and closes the FDs of every thread's idle connections. With
per-thread-group FD tables, the connections of the other thread groups
were already closed by the kernel when the last thread of their group
exited, and their numbers are only meaningful in those groups' tables,
closing them from here would in fact close whatever FD carries the
same number in this thread's own table, and let sock_conn_ctrl_close()
rewrite an unrelated entry of this group's FD table. So in that case,
only release the memory and leave the FDs alone.
MEDIUM: dns: Stick the TCP nameserver tasks to the resolvers' thread
The tasks handling the TCP nameservers (task_req, task_rsp, task_idle)
are created with no thread affinity, but with thread groups having their
own file descriptors table, it has to run on the same thread as the
one who created the socket, or maybe the other thread will not be able
to use the socket.
In practice, it was not a problem, because the scheduler will wake the
task on the running thread. But that may change in the future, so make
sure we always run on the current thread from start to finish.
MINOR: cli: Report the tgid along the FD in "show sess"
With per-thread-group FD tables, an FD number alone is ambiguous: it is
only meaningful within the thread group of the stream that uses it. So
display the FDs of the "show sess" output in the "tgid/fd" form already
used by "show fd", both in the one-line format and in the detailed one.
In the detailed output, the FD state, update and thread masks are also
read from the owner group's table rather than from the calling
thread's one, which would report an unrelated entry when the tables
are not shared.
MINOR: debug: Report the current tgid in "debug dev fd"
"debug dev fd" scans the FDs of the process as the calling thread sees
them, and reports the ones unknown to fdtab. With per-thread-group FD
tables, both that scan and the fdtab lookups only reflect the view of
the calling thread's group: the same command may report different FDs
depending on the thread that serves it. Let's print the current tgid in
a header line so the output states which view it shows.
MINOR: cli: Make "show fd" aware of per-thread-group FD tables
The <tgid>/<fd> argument forms of "show fd" were parsed but the tgid was
ignored so far, the dump always reading the local thread's FD table.
Now that each thread group can have its own table, give the tgid its
intended meaning: a specific tgid dumps that group's table only, while
the wildcard and unspecified forms visit every group in turn, so that
"show fd /<fd>" shows what each group knows about a given FD number.
Each entry's group remains visible in the gid= field. The polled_mask
words are read from the dumped group's slice as well, and the dump can
resume from the right group when the output buffer fills up.
When the tables are shared, all the groups alias the same table, so a
single pass is performed as before and the tgid makes no difference.
The tgid is now also validated against the number of configured thread
groups rather than the maximum supported.
MEDIUM: pipes: Have one pool of free pipes per thread group
The pipes used for splicing are kept in a shared pool of free pipes, so
that they can be reused by any thread. However, with per-thread-group
FD tables, a pipe created by one group only exists in that group's
kernel table, so its FDs must never be handed over to a thread of
another group.
To fix that, replace the global list of free pipes with one list per
thread group if the fd tables are not shared across thread groups.
MEDIUM: pollers: Only allow epoll when each tgroup has its fd table
Getting pollers such as select and poll to work when thread groups
unshared their file descriptors would require to have per-thread groups
arrays of file descriptors. Given making each thread group its own file
descriptor table is done for performance reasons, and those pollers will
never perform very well, this does not seem to be worth it.
kqueue and evports would work, except that they are only implemented on
OSes that do not provide the equivalent of the unshare() system call.
So effectively, only epoll can be used as a poller when fds are not
shared across thread groups.
MEDIUM: cli: Transfer sockets with unshared file descriptor tables
Teach the _getsocks command how to use the new rx agent facilities to
obtain the file descriptors from other thread groups when each one has
its own file descriptor table.
MEDIUM: listener: Properly handle unshared fd tables between tgroups
With different FD tables for each thread group, a new mechanism is
needed to get each thread group to close/open their fds, and to transfer
fds from one thread group to another.
So we add the concept of rx agents. Each thread group has a tasklet
dedicated to that, bound to the first thread of each thread group, and
are used to get those operations done.
Each receiver contains context bits so that the tasklet will know which
action to do when woken up.
A new socketpair is also created per group, to be able to send fds to other
thread groups.
MEDIUM: backend: Do not always allow takeover across thread groups
When we're running with separate file descriptor tables across thread
groups, do not allow connection takeover across thread groups, as the
connection would not be usable for that thread, anyway.
MEDIUM: listeners: Don't always balance connections across thread groups
Do not try to dispatch new connections to threads from other thread
groups if there is no file descriptor sharing across thread groups, as
they could not access that new fd.
MEDIUM: pollers: Create the poller pipes before we create the thread
As we may now unshare the file descriptor tables, make sure we create
the pipes used to wake the pollers up before we create the threads, so
that they will be shared by all threads.
MEDIUM: pollers: Allow one polled_mask per thread group
Similarly to what's been done for fdtab, create one polled_mask per
thread group if GTUNE_NO_TG_FD_SHARING is set.
As the flag can't be set yet, this commit should have no impact.
Olivier Houchard [Mon, 29 Jun 2026 12:32:19 +0000 (14:32 +0200)]
MEDIUM: fd: Make it possible to have one fdtab per thread-group
Add a new global flag, GTUNE_NO_TG_FD_SHARING. When set, the goal is to
have to a different file descriptor table for each thread group, if
the platform supports separate file descriptor tables for each thread
(so right now, only linux with the unshare() syscall), as with many
threads, the kernel lock for the file descriptor table is a huge source
of contention.
This commit just makes it so if the flag is set, we will have one fdtab
per thread group, and we will call unshare() for each thread group, so
that each has its own file descriptor table.
Note that for now there is no way to set the flag, as a lot more work is
needed before the feature is usable. As such, this commit should have no
noticeable impact.
fdinfo is a global table, indexed by the file descriptor, whose purpose
is just to remember if a port was allocated in a port range, so that
that port may be released when the file descriptor is closed. But that
would no longer work if we split file descriptors per thread-group, and
while we may just have one fdinfo table per thread group, this sounds
like wasted memory with little benefits.
So instead, abuse fdtab to store the relevant information. "state" now
has two more flags, FD_HAS_PORT and FD_OWNER_PR. FD_HAS_PORT means a
port has been explicitly allocated for that file descriptor.
FD_OWNER_PR means the owner field now points to the struct port_range
from which the port was allocated, and where it should be released.
The local port is now obtained from getsockname(), instead of storing it
anywhere.
For TCP, as fd_delete() is called from sock_conn_ctrl_close(), which
itself is called when the connection is about to be destroyed, we can
get the port range used as it is the one provided by the server, which
is available as conn->target.
For QUIC, unfortunately, the connection may be gone already, so instead,
we just store it in the quic_conn.
We have to wait until _fd_delete_orphan() is called in order to release
the port, as in some rare cases, fd_delete() will not call
_fd_delete_orphan(), and we want to make sure we only release the port
just as we're closing the file descriptor.
BUG/MINOR: haload: fix rate limit bypass during stream errors
In hld_strm_task(), stream errors triggered an unconditional task_wakeup()
on the user task. This bypassed arg_rate and caused haload to send connection
retries as fast as possible instead of respecting the configured request rate.
Call hld_usr_schedule() when arg_rate is set to guarantee that
rate pacing is preserved even when streams fail.
CLEANUP: haload: embed rate_task into hld_thr_info structure
Store the rate task directly inside the per-thread <hld_thr_info>
struct instead of keeping a separate global <hld_rate_tasks> array.
This simplifies memory management by grouping thread-specific tasks and
data together:
- move rate task creation into <hld_alloc_thrs_info>
- clean up rate tasks directly inside <hld_dealloc_thrs_info>
- remove the separate <hld_dealloc_rate_task> helper and global array
BUG/MINOR: cli: do not reject the "/<fd>" form of "show fd"
Commit da554b7ef added the tgid/fd argument form, where tgid is
optional, and a missing tgid would mean "in each thread group".
However the code considered everything that was not a digit a filter
type, and as / was not a valid filter type, we'd just get an output of
"Invalid FD type".
Fix that by not considering / a filter character.
BUG/MINOR: cli: use the current argument to parse the FD spec in "show fd"
Commit 1cb041a6e added filtering to show fd, which means the number of
arguments can now change, however args[2] was still used from time to
time. Fix this by using args|arg] instead when relevant.
Note that before 3.4, there is only one instance of that, atoi(args[2])
should be atoi(args[arg]). That could lead to the wrong fd being used.
IMPORT: cebtree: private: fix the duplicate detection in the lookup shortcut
Commit 6b5cd28 ("OPTIM: descent: stop at the node when the path to the
leaf is not used") made _ceb_descend() stop as soon as a match is found
on the key when the caller requests none of the ret_* pointers needed by
insert and delete. This overlooked a special case needed for duplicates,
which require to descend to the leaf. The detection performed at the end
of the function is:
It needs <is_leaf>, i.e. the knowledge that the node was reached through
a leaf pointer which is not its own, because a node's role (node/leaf)
is only figured by the path used to reach it, and the state of the leaf
itself is not sufficient. For example, inserting "s1", "s2", or "s2",
"s2" both produce a second node whose b[0] is a tagged pointer to the
first node and whose b[1] is a tagged pointer to itself. Only the tag
of the pointer leading to it differs.
Two independent problems follow:
- the shortcut to the leaf from the commit above does not update
<is_leaf>. For arrays and strings it assigns "node = ln"/"node = rn"
then breaks, so is_leaf still describes the node we came from, and
a duplicate is reported as a regular leaf ;
- the shortcut may stop on an inner node instead of the leaf, and an
inner node carrying the searched key says nothing about the possible
presence of a list of duplicates hanging below it. For example, in
(0,2a,3,2b), the descent will visit 0,3,2b and this is the last 2
that was inserted and not the one we want to return.
As a result *_prev_dup() returned NULL instead of the previous duplicate
as soon as the tree contained another key, i.e. exactly when the tree is
deep enough for the descent to enter the loop. *_next_dup() is immune to
the first problem since it never looks at is_dup, but not to the second
one. Less visibly, *_lookup() also uses is_dup, to walk back to the
first element of a list, and with is_dup wrongly zero it returned the
last one.
The fix keeps the shortcut everywhere it is harmless, as the whole point
of that commit was to avoid useless operations. Only three call sites
both reach this block and request is_dup: _ceb_lookup() with CEB_WM_KEQ,
and _ceb_next_dup() and _ceb_prev_dup() with CEB_WM_KNX/CEB_WM_KPR. All
the others either pass a blocking ret_* pointer (insert, delete,
next/prev, lookup_le/lt/ge/gt) or use a method below CEB_WM_KEQ which
never enters the block. Thus we have this:
- for arrays and strings, the shortcut jumps to the matching branch,
which for a list of duplicates is its last element, so it is taken
whenever that branch is already a leaf, and <is_leaf> is updated
accordingly. Only the inner node case is given up ;
- for ints, the shortcut stops on the matching node itself, which is
the dual-role node, i.e. the *first* element of the list. That is
exactly what _ceb_lookup() must return, so it is kept there, and
excluded only for _next_dup()/_prev_dup() which need the last
element in order to walk the list.
More importantly, performance-wise, no operation that was already correct
descends further than before. Int lookups in multi-key trees are now
slightly slower since they need to reach a leaf or a dup tree, but that
is the price to pay to find the real first element. On the other hand,
string lookups are now slightly faster (~10%).
Now tests/testdups does not return any failure out of 6298 sequences.
[wt: this could sometimes miss duplicate server names since 3.3 and
commit 413e903a22 ("MEDIUM: server: switch conf.name to cebis_tree"),
as Amaury found. This must be backported to 3.3]
BUG/MINOR: server: fix check reuse-pool in srv_settings_cpy()
<reuse_pool> may be set on check to instruct that connection reuse may
be performed for server health checks.
As expected, this field is duplicated in srv_settings_cpy(). However,
there is an exception for rHTTP servers as in this case check-reuse-pool
must be forcefully activated, as performed in srv_settings_init(). Thus,
the value was not copied to avoid changing it to 0.
This causes a configuration issue with rHTTP on server-template though.
In this case, check-reuse-pool is not set for duplicated server
instances as srv_settings_init() is not called for them. This causes
checks to always fail for such entries.
Fix this by forcing check-reuse-pool for rHTTP in srv_settings_cpy(),
and duplicate the values for other servers.
MINOR: server: ensure check-reuse-pool is init in srv_settings_init()
For rHTTP servers, check-reuse-pool is forcefully set on them, else
checks would always fail. This initialization was performed in
_srv_parse_init().
Move this force init in srv_settings_init(). This is a better place as
_srv_parse_init() should only perform minimal configuration, mostly
related to server address. Then srv_settings_init() is called with the
task to set every fields to suitable default values before keywords
parsing.
Also, this ensures that check-reuse-pool is set to 0 for non rHTTP
servers. This is not strictly required though as server structure is
calloc'ed.
BUG/MINOR: server: duplicate server alt_proto in srv_settings_cpy()
Server <alt_proto> may be set to select an alternative protocol to the
default one depending on the server address type. Currently, the main
usage is for MPTCP on the backend side.
This field was not duplicated in srv_settings_cpy(). This has no impact
with default-server as such entry do not have any address configured.
However, this causes an issue with server-template, as duplicated
entries will remain with an unset <alt_proto> field. This results in an
incorrect TCP protocol selected on connect instead of MPTCP.
Fix this by ensuring that <alt_proto> is copied in srv_settings_cpy().
This is only performed for server-templates, as with other settings
relative to the server address.
For QUIC backend side support, server xprt has to be set to XPRT_QUIC
when a quic4/6 address is parsed.
Prior to this patch, this was performed in _srv_parse_init(). However
this is not functional with server-template as this function is not
called for the duplicated entries. Xprt field was not updated, which
prevented any communication on these servers.
To properly initialize a QUIC server, uses the same method as SSL.
Server xprt is now set later during ssl_sock_prepare_srv_ctx(). This
callback is invoked through XPRT prepare_srv during proxy_finalize(),
called for any servers with SSL or QUIC settings.
MINOR: server: improve parsing error for server-template
When parsing a server configuration line, a global context is set to the
current server instance. This allows to automatically prefix any
warning/error messages by the server name. This prefix is generated via
generate_usermsgs_ctx_str().
The output was not useful for server-template lines as server <id> is
NULL for them. Thus, this patch improves generate_usermsgs_ctx_str() to
identify a server-template. Now, a dedicated 'server-template' prefix is
generated, with <tmpl_info.prefix> used instead of NULL <id> field.
* Example of a previous error message :
[ALERT] (39934) : config : [./hap.cfg:54] : 'server-template be/(null)' : error detected while parsing [...]
* Example of a new error message :
[ALERT] (40205) : config : [./hap.cfg:54] : 'server-template be/local' : error detected while parsing [...]
Matt Suiche [Tue, 28 Jul 2026 13:51:09 +0000 (15:51 +0200)]
BUG/MAJOR: ssl/ocsp: lock the OCSP response around reads in the stapling callback
ssl_sock_ocsp_stapling_cbk() reads ocsp->response.area and
ocsp->response.data without any lock, while
ssl_sock_load_ocsp_response() -- called from the CLI "set ssl
ocsp-response" handler, the ocsp-update task and the reload path --
frees and replaces that buffer via chunk_dup(), also without any
synchronization:
A concurrent update frees the old area between the allocation and the
copy (heap-use-after-free read, confirmed under ASan as a READ of size
12548 in the callback's memcpy), or yields a torn read pairing the new
larger length with the old smaller area (linear over-read). Because the
copied bytes are handed to SSL_set_tlsext_status_ocsp_resp() and sent
to the TLS client in the status_request extension, freed or reused heap
contents can be disclosed to any remote client asking for a stapled
response during an update window; updates are periodic by default when
ocsp-update is enabled. A crash is the more likely practical outcome,
but the disclosure variant makes this heartbleed-class.
Let's take ocsp_tree_lock on both sides, which is the file's existing
idiom for accessing OCSP response contents (see
ssl_get_ocspresponse_detail()). The lock declaration is moved to the
top of the !OPENSSL_NO_OCSP section so that the callback can use it.
The critical section on the handshake path stays short (a validity
check plus a copy of a few KB), and allocating memory under this
spinlock is consistent with the existing "show ssl ocsp-response"
handler, which base64 encodes the response into a growable chunk while
holding the same lock.
This must be backported to all supported versions.
Matt Suiche [Tue, 28 Jul 2026 13:51:09 +0000 (15:51 +0200)]
BUG/MEDIUM: sample: reject the deprecated protobuf group wire types
protobuf_field_lookup() dispatches on the field's wire type through
protobuf_parser_defs[], but the entries for wire types 3 and 4
(START_GROUP/STOP_GROUP, deprecated) are zero-initialized:
while the only validation is the upper bound on the table:
if (wire_type >= sizeof(protobuf_parser_defs) / sizeof(protobuf_parser_defs[0]))
return 0;
A field using wire type 3 or 4 therefore passes the check and reaches a
call through the NULL .skip/.smp_store pointer: a request body
consisting of the single byte 0x0b (field 1, wire type 3) crashes the
process at pc=0. Any configuration routing a client body through the
"protobuf" converter (e.g. "http-request set-var(txn.f)
req.body,protobuf(1)") makes this remotely and unauthenticatedly
triggerable.
Let's reject wire types whose parser definition is missing and return 0
("field not found"), in line with the function's existing failure
contract.
The protobuf converter first appeared in 2.2; this should be backported
to all supported versions.
Matt Suiche [Tue, 28 Jul 2026 13:51:09 +0000 (15:51 +0200)]
BUG/MEDIUM: peers: check the available room before encoding dict values
peer_prepare_updatemsg() encodes stick-table entries into an update
message of <size> bytes but never checks that the encoded data actually
fits (the function still carries its "TODO: check size" comment). For
entries holding a server_key, the dictionary value, up to ~16 KB, is
copied unconditionally:
/* Encode the length of the dictionary entry data */
value_len = de->len;
intencode(value_len, &end);
/* Copy the data */
memcpy(end, de->value.key, value_len);
The peers protocol is plaintext and unauthenticated, so a remote peer
can plant an entry whose server_key is larger than the room left in the
buffer. When the victim later teaches that entry, the memcpy() above
writes past the end of the 16384-byte trash buffer; this was confirmed
under ASan as a heap-buffer-overflow WRITE of size 16360, and it fires
again on every teach retry at the same address.
Let's verify that the value fits in the buffer before copying it, and
return 0 ("unable to encode the message") when it does not, which is
how the function already reports its other encoding failures.
This must be backported to all supported versions (server_key in
stick-tables dates back to 2.4).
CLEANUP: slz: clarify that the size promise applies to the stream, not to a call
The output size guarantee of slz_rfc1951_encode() reads as if it applied
to every call, but up to 31 bits are retained in the queue from one call
to the next (on 64-bit systems), so a call may emit a few bytes that
belong to the data of the previous ones, and a single call may emit up to
5 bytes more than expected. Let's just clarify this to avoid future
surprises.
Unexpectedly, an unsigned char is promoted to a *signed* int when
shifting, so shifting a value >= 128 by 24 places can overflow it, which
is undefined behaviour depending on build options. It happens to produce
the expected result with the usual compilers, but ubsan on an i386 build
reports it for about half of input bytes:
src/slz.c:482:107: runtime error: left shift of 220 by 24 places cannot
be represented in type 'int'
Let's just properly cast the uchars to u32 before shifting (this does
not change the produced code at all).
Note: in practice it doesn't happen with the compilers and build options
we support in haproxy but the fix is trivial so better fix this to
clean up the code base and make ubsan happy.
BUG/MINOR: slz: fix the adler32 accumulators signedness on 32-bit
slz_adler32_block() unfortunately uses a signed long as the crc accumulator
instead of an unsigned one, meaning that for CRC values where the 32th bit
is set on 32-bit machines, the right shift will drag sign bits and corrupt
it. This only affects zlib streams on 32-bit systems (rfc1950) and has
been there for a very long time, showing that the zlib format is really
not much used in target environments.
The fix is trivial, just change the accumulators to unsigned long.
This must be backported to 1.2. It was in slz.c prior to 1.3.0.
Note: the impact in haproxy is the "zlib" compression algorithm often
causing CRC errors on clients when haproxy runs on a 32-bit
system. Since "zlib" has long been avoided due to incompatibilities
with certain clients in the past, the impact should be almost
inexistent (this issue was never reported).
BUG/MINOR: slz: do not append a block to an already finished stream
slz_rfc1951_flush() terminates the pending block then emits an empty
stored block to byte-align the output. But encode() called with <more>
cleared may leave the stream in SLZ_ST_LAST, that is, inside a block whose
BFINAL bit has already been sent. Flushing there terminated that block,
which completes the deflate stream, and then emitted one more block with
BFINAL set again. Those 5 bytes sit past the end of the stream, so the
gzip or zlib trailer that finish() appends right after is no longer where
the peer expects it: it reads the empty block as the checksum.
The data itself decodes correctly, only the check fails, which makes it
particularly difficult to diagnose. Raw deflate is not affected as it has
no trailer to shift, and a stream that ends in EOB state is not affected
either since its queue is empty and flush() returns early. Fuzzing random
sequences of encode/flush/finish showed ~2% corrupt streams for gzip and
zlib.
When the terminated block was the final one there is nothing left to align
to, and nothing may be appended, so let's just flush the pending bits and
return. This also makes the flush 4 bytes shorter in that case, and the
BFINAL bit of the empty block is now always zero, which is what the state
guarantees at that point.
BUG/MEDIUM: slz: bound the bits wasted by the 9-bit literals
Commit 002e838 ("bug: always make sure to limit fixed output to less than
worst case literals") made sure that switching from EOB to FIXED to emit
a reference is only done when the reference pays for the way back, based
on the fact that once in FIXED state a reference is always smaller than
the bytes it replaces.
This was unfortunately not enough: the literals interleaved with those
references can still fail. Indeed, octets 144 to 255 cost 9 bits instead
of the 8 they would cost in a stored block. The <bit9> counter measures
exactly that, but it was reset after every reference, so a stream
alternating just under 52 such literals with a cheap reference never
reached the threshold that sends them as a stored block, and kept
inflating for as long as the pattern lasted. 51 random literals >= 144
followed by 4 bytes copied from 8 bytes earlier (i.e. almost the exact
same pattern as previously tested) produce 1073161 bytes out of 1048576
(+2.34%) where the API promises at most 1048663, i.e. 24 kB more than a
caller sizing its output buffer from that promise would expect for a
single call.
Bit9 isn't sufficient to track the debt cross references, so let's add a
second <debt> counter to the stream's unused space. It accumulates the
bits actually wasted by the literals emitted in huffman mode, and each
reference records what it saved over the same bytes sent as literals,
bounded to zero. Above SLZ_MAX_DEBT (200 bits, i.e. 25 bytes) the encoder
stops trusting bit9: pending literals are stored and references have to
compensate for the full round trip, which lets the literal runs merge into
a full 65535-byte block and stop the growth.
The crafted stream now produces 1048677 bytes (+14 bytes over the
documented maximum instead of +24509). Other inputs such as text or
silesia corpus do not show any change since it's quite hard to fall
into this case.
Note that the threshold is deliberately much larger than the 52 bits of
a switch to amortize oscillations without needlessly sending literals.
Note: the impact in haproxy will start with tune.bufsize above 43kB
for the default 1kB reserve. A simple workaround consists in
always keeping the reserve (tune.maxrewrite) at least 1/32 of
bufsize.
BUG/MINOR: slz: use the exact switch cost for the last literals of a block
The decision to send the pending literals as a stored block rather than in
fixed huffman mode is taken when the 9-bit literals wasted more than the
52 bits it costs to leave the fixed huffman encoding and to come back to
it. But for the last literals of a block, nothing comes after the stored
block, so there is no need to pay for the block type of a next block nor
for the EOB, while the huffman variant still has to send an EOB. The
switch is thus 10 bits cheaper, and 10 more when the stream is still in
EOB state, since then the block type is needed in both cases and no EOB
has to be terminated.
Using 52 there made the encoder prefer huffman for data that was cheaper
to store, and the output could exceed the documented maximum. The smallest
case found by fuzzing is a 47-byte input entirely made of bytes >= 144
which produced 55 bytes (3 bits of block type + 47*9 bits + 7 bits of EOB)
where the stored block only needs 52, for a documented maximum of 54.
With these correct costs, we no longer see outputs exceed the documented
maximum, wether it's with small inputs (tested with ~3 million random
small inputs as small as 47 bytes), or usual files found in tests/ and
bash, gcc, libc, and silesia. No performance change was observed either.
Note that a stream can still exceed the documented maximum by a few bytes
(17 bytes were observed on a 390000-byte crafted input) because each
reference emitted between two stored blocks forces them out and adds a
5-byte block header that the accounting attributes to the reference. This
is for a future fix.
Note: the impact in haproxy is practically inexistent since we have a 1kB
reserve and build by default with the tag at the end of pools, even
if two extra bytes were to be emitted, it would have no effect.
Better backport it to avoid triggering ASAN though.
CLEANUP: slz: fix the documented worst case size of flush() and finish()
The documented output buffer requirements of the flush() and finish()
functions date back to the 32-bit queue, where at most 7 bits could be
pending. Since the 64-bit queue was introduced (used on x86_64 and armv8),
up to 31 bits may be pending, and the accounting also forgot the EOB that
may have to be emitted before the empty block. As a result a caller
strictly sizing its output buffer from the documentation could be short
by one to two bytes and see the encoder write past the end of its buffer.
Let's update the document worst cases for these functions depending on
what they still have to emit: 31 pending bits + 7 for EOB + 3 for
BFINAL/BTYPE + 7 for EOB or 32 for LEN+NLEN, rounded up to the next
byte (easily forgotten):
Note that all values are at least as large as the previously claimed ones
and that 32-bit systems never consume more than what was claimed, so the
new documented values are valid both for 32 and 64 bits.
Even though this patch only touches comments, it's marked as a bug so
that it is backported where it matters and users have a chance to spot
the new values.
BUG/MINOR: slz: do not read past the end of the input around the match loop
slz_rfc1951_encode() pre-loads the first 3 bytes of the input into <word>
before entering the main loop on architectures which do not define
UNALIGNED_FASTER (e.g. i386, or big endian ones). This was done
unconditionally, thus inputs shorter than 3 bytes (including empty ones)
caused up to 3 bytes to be read past the end of the input buffer, which
may segfault if the buffer ends on the last page of a mapping. This is
easily reproduced on i386 by placing a zero-length input right before an
unmapped page.
The pre-loaded word is only ever used if the main loop is entered, which
requires at least 4 remaining bytes, so let's simply condition the load
on this.
The exact same case exists at the end of the loop where we can go beyond
end-3 and try to read 3 or 4 bytes before getting back to the beggining
of the loop, so we're using the same condition here, which helps the
compiler perform the test only once and use unconditional branches from
there.
The code is unchanged on x86_64 and armv8 (out of ifdef) and no
measurable change is observed on other archs.
This should be backported to 1.2. The patch is easier consulted with
git show -b.
Note: the impact in haproxy is practically inexistent since we build by
default with the tag at the end of pools, even if an extra byte
were to be accessed on slower architectures, it would have no
effect. Better backport it to avoid triggering ASAN though.
BUG/MINOR: http-rules: fix release of a failed "set-cookie-fmt" redirect rule
<redirect_rule.cookie> is a union holding either an ist (set-cookie,
clear-cookie) or a log-format expression (set-cookie-fmt), and
http_free_redirect_rule() picks the member to release from the
REDIRECT_FLAG_COOKIE_FMT flag:
if ((rdr->flags & REDIRECT_FLAG_COOKIE_FMT))
lf_expr_deinit(&rdr->cookie.fmt);
else
istfree(&rdr->cookie.str);
But http_parse_redirect_rule() accumulates the flags in a local variable and
only assigns them with "rule->flags = flags" at the very end, once
everything succeeded. So when the log-format string of a "set-cookie-fmt"
fails to parse, the "goto err" runs http_free_redirect_rule() on a rule
whose ->flags is still 0, and istfree() is called on the <str> member while
<fmt> is the live one, leading the process to crash.
Let's init the rule members (code, type and flags) as soon as possible, so
after the rule allocation. And REDIRECT_FLAG_COOKIE_FMT flag is not set
directly on the rule's flags, before the log-format string parsing.
This only happens while parsing an invalid configuration, hence no security
impact, but a configuration checker must report errors rather than abort.
This should be backported to all versions having "set-cookie-fmt", so 2.9 and
above.
BUG/MINOR: htx: Transfer HTX_FL_EOM flag on success in htx_append_msg()
htx_append_msg() function copy all blocks from a source message to a
destination one. But it never take care to also transfer HTX_FL_EOM flag if
necessary on success. It is important because this function is used to copy
error messages during HTTP analysis.
It seems to be harmless because when an error is triggered the stream is
also closed and most of time a raw copy is performed instead of a
block-per-block copy. But this could lead to prematurely close the
connection at the end of the response.
This patch should be backported to all supported versions.
but it never verifies that the destination buffer is at least as large as
<msg->size>. The only caller, http_reply_to_htx(), copies an error message
buffer, which http_str_to_htx() always allocates with a size of
global.tune.bufsize, into the response channel buffer. Two things follow:
- if the destination were smaller, this would be a plain heap overflow. It is
not reachable today because the response channel buffer is never a small
one: only the request channel may be moved to a small buffer, by the
PR_O2_USE_SBUF_QUEUE code in stream.c, and the L7-retry buffer is not a
channel. But nothing states nor checks that invariant, and small buffers
are recent, so this is a landmine.
- if the destination is larger, which does happen since "http-response
wait-for-body <time> use-large-buffer" moves the response channel to a large
buffer, the copy also installs the source's ->size, so the destination HTX
message ends up believing it is only bufsize-sized. That is harmless (it
only under-uses the buffer and heals on the next reset) but wrong.
Let's restrict the raw copy to the case where both underlying buffers have
exactly the same size, and fall back to the existing block-per-block append
otherwise.
This should be backported to all supported versions.
BUG/MEDIUM: tools: make string encoding possible to fail instead of truncating
encode_string() and encode_chunk() functions were silently truncated the
output string if it is too small. encode_string() is not used but
encode_chunk() is used to encode urls (url_enc converter and ocsp). In this
context it is not expected to have an url partially encoded.
So let's slightly change the API to add an extra argument to these functions
to be able to fail when the output string is too small. <truncate> must be
set to 0 to return an error (NULL) instead of truncate the output string.
url_enc converter and ocsp were also update to trigger an error in that
case.
The issue is pretty minor but it remains an API change, so it is tagged as
MEDIUM. It could be backported to 3.4 and probably as far as 3.2 or 3.0.
while every sibling argument does check it ("string", and "lf-file" through its
combined "!obj || read(...)" test). <obj> is later handed over to
parse_logformat_string(), which starts with "lf_expr->str = strdup(fmt)", so a
NULL would be passed to strdup() and dereferenced.
This only happens if an allocation fails while parsing the configuration, so the
impact is limited, but the check is missing where all the others are present.
This should be backported to all supported versions.
CLEANUP: http-conv: index the captures array with hdr->index in the converters
smp_conv_req_capture() and smp_conv_res_capture() look the capture slot up by
walking the proxy's capture list backwards until the decreasing counter matches
the requested id, then allocate the storage using <hdr->index> but write it
using <idx>:
Both are in fact always equal, because every place that appends to the list
(cfgparse-listen.c, proxy.c, tcp_rules.c and http_act.c) does "hdr->next = px->
req_cap; hdr->index = px->nb_req_cap++; px->req_cap = hdr;", so the head always
carries the highest index and the walk keeps hdr->index == i. But relying on
that invariant to index an array while the neighbouring lines use the field that
actually describes the slot is confusing, and it silently ties these two
functions to the way the list happens to be built.
Let's use hdr->index everywhere. No functional change.
BUG/MINOR: http-act: reject a negative capture id in the capture actions
"http-request capture <expr> id <idx>" and "http-response capture <expr> id
<idx>" parse their identifier with strtol() and only reject trailing garbage:
id = strtol(args[cur_arg], &error, 10);
if (*error != '\0') {
memprintf(err, "cannot parse id '%s'", args[cur_arg]);
A negative identifier is therefore accepted at boot. The check functions only
verify the upper bound ("idx >= px->nb_req_cap"), and even that only for a proxy
with the frontend capability, so nothing complains. At run time,
http_action_req_capture_by_id() looks the slot up by walking the capture list
backwards:
for (h = fe->req_cap, i = fe->nb_req_cap - 1;
h != NULL && i != rule->arg.capid.idx ;
i--, h = h->next);
if (!h)
return ACT_RET_CONT;
<i> never matches a negative <idx> so the walk ends on a NULL <h> and the action
silently does nothing. There is no out-of-bounds access here, unlike in the
"capture.req.hdr" sample fetch, but a configuration that can only ever be a
no-op must be rejected rather than silently ignored.
Let's reject negative ids where they are parsed, which also covers the proxies
that have no frontend capability and are thus not checked at all.
This should be backported to all supported versions.
BUG/MINOR: http-act: work on a copy of the sample in del-headers-bin
http_action_del_headers_bin() walks the varint-encoded list of header names
directly in the sample expression result:
p = b_orig(&hdrs_bin->data.u.str);
end = b_tail(&hdrs_bin->data.u.str);
and uses the names it decodes there while calling http_remove_header() on the
HTX message in between. But the sample may perfectly well point into that very
message: a string sample is cast to a binary one in place, so an expression such
as "req.hdr(x-list)" hands out a pointer inside the header block it describes.
http_remove_header() memmoves the payload of the block it shortens and marks the
blocks it removes as unused, so <p>, <end> and <n> may then designate stale or
recycled bytes and the loop goes on deleting names decoded from garbage. The
header value must be a valid varint-encoded list for this to happen, which is
possible since only NUL, CR and LF are rejected in an H1 header value.
This is the exact same problem as the one fixed for the sibling actions by
commit 43932db85 ("BUG/MEDIUM: http-act: Make a copy of the sample expr in
(set/add)-headers-bin"), which this action was left out of. Let's copy the
sample into a private chunk the same way before decoding it.
This should be backported to 3.4, like the commit above.
BUG/MINOR: http-act: restore the response buffer state in the early-hint action
http_action_early_hint() starts with:
struct htx *htx = htx_from_buf(&res->buf);
htx_from_buf() marks the underlying buffer as full (b_data = b_size) and, as
documented, it is the caller's responsibility to call htx_to_buf() to update it
back. The function never does it. On the success path this is harmless because
the HTX message is not empty, which is exactly what a full buffer represents,
and on the error path channel_htx_truncate() takes care of it. But the very
first test of the function is:
if (!(s->txn.http->req.flags & HTTP_MSGF_VER_11))
goto leave;
so for an HTTP/1.0 client, where no 103 response may be emitted, the response
buffer is left flagged as containing b_size bytes of data while the HTX message
is empty, until some other code path happens to call htx_to_buf() on it. Any
code looking at b_data(&res->buf) in between sees a full response buffer.
No misbehaviour could be observed (the HTX-aware helpers all work on the HTX
message, not on b_data), but leaving the buffer in a state that contradicts its
contents is an accident waiting to happen.
Let's simply call htx_to_buf() on the leave path. It is a no-op for the other
two paths since the message is either non-empty or was already truncated.
This should be backported to all supported versions.
BUG/MINOR: http-act: fix a double free of the map reference on a parsing error
parse_http_set_map() extracts the map/acl file name into <rule->arg.map.ref>,
then parses one or two log-format strings. On a parsing failure it releases the
reference before reporting the error:
if (!parse_logformat_string(args[cur_arg], px, &rule->arg.map.key, ...)) {
free(rule->arg.map.ref);
return ACT_RET_PRS_ERR;
}
but <rule->release_ptr> has already been set to release_http_map(), which starts
with "free(rule->arg.map.ref)". Since the caller calls free_act_rule() upon
ACT_RET_PRS_ERR, the same pointer is freed twice. Both error paths of the
function (the key and the value patterns) are affected, and they cover the
"add-acl", "del-acl", "set-map" and "del-map" actions.
BUG/MINOR: http-act: fix a double free of the regex on a rule parsing error
parse_replace_uri() and parse_http_replace_header() compile their regex into
<rule->arg.http.re>, and when the log-format argument that follows fails to
parse they release it before reporting the error:
The pointer is left dangling in the rule while <rule->release_ptr> has already
been set to release_http_action(), which does exactly the same:
if (rule->arg.http.re)
regex_free(rule->arg.http.re);
On ACT_RET_PRS_ERR the caller (parse_http_req_cond() & friends) calls
free_act_rule(), which invokes release_ptr, so regex_free() runs twice on the
same object. It ends up calling regfree()/pcre*_free() on freed memory and
free() on an already freed pointer.
Both abort under MALLOC_CHECK_=3, and the second one even segfaults with the
libc regex backend, so "haproxy -c" dies instead of reporting the configuration
error (and the remaining errors of the file are never reported).
This only happens on an invalid configuration during parsing, so it has no
security impact, but a configuration checker must not crash.
Let's reset the pointer after releasing it, as done for <arg.http_reply> in
release_act_http_reply().
This should be backported to all supported versions.
The first call can never do anything: st->comp_algo is only set from the
"Accept-Encoding" loop above, whose exit condition is http_find_header()
returning 0, which resets ctx.blk to NULL, and http_remove_header() returns
immediately for a NULL ctx.blk. The loop that follows removes all the
occurrences of the header anyway, so the call is dead code, and it would remove
the wrong header if ctx were ever to carry another context.
CLEANUP: htx: remove the unreachable "append_data" label in htx_reserve_max_data()
htx_reserve_max_data() carries an "append_data:" label which is never the target
of a goto: the function simply falls through it. It looks like a leftover from
htx_add_data(), which does use such a label. Let's drop it, the code is reached
by fallthrough anyway and an unused label is only confusing.
BUG/MINOR: http-ana: fix a one-byte over-read in the client-side cookie parser
In http_manage_client_side_cookies(), each iteration of the cookie loop skips
the blanks in front of the attribute name, then tests it against '$':
while (att_beg < hdr_end && HTTP_IS_SPHT(*att_beg))
att_beg++;
...
if (*att_beg == '$')
continue;
<att_beg> may legitimately have reached <hdr_end>, in which case the test reads
one byte past the end of the Cookie header value. This happens for any value
ending with a delimiter, optionally followed by blanks, e.g. "Cookie: a=b; ":
the last iteration starts on the ';', skips it and the trailing space, and lands
exactly on <hdr_end>.
The extra byte always belongs to the HTX buffer (the payload area is followed by
other blocks, the free space or the block table), so there is no out-of-bounds
access at the allocation level, and the only functional consequence is that a
neighbour byte holding a '$' makes the parser skip the end of the header as if
it were an attribute instead of marking the header as to be preserved. But the
value must not be read past its end, and the check is free since <hdr_end> is
already at hand.
Note that http_manage_server_side_cookies() does not have this issue, it uses
"equal == val_end" to detect the empty trailing element.
This has been there since the HTX rewrite of the analysers by commit f4eb75d17
("MINOR: htx: Add proto_htx.c file") in 1.9-dev7, and the pre-HTX code had the
same construct, so it should be backported to all supported versions.
BUG/MINOR: h3: don't use a block pointer to roll back a partial HTX conversion
h3_resp_headers_to_htx() and h3_trailers_to_htx() save a pointer on the tail
HTX block of the destination message and use it to remove whatever they added
when the conversion fails:
tailblk = htx_get_tail_blk(htx);
...
out:
if (appbuf) {
if ((ssize_t)len < 0)
htx_truncate_blk(htx, tailblk);
An HTX block pointer encodes a block *position* (the table is indexed backwards
from the end of the storage area), so it stops designating the same block as
soon as the table is compacted. htx_add_trailer() may reach
htx_reserve_nxblk(), which calls htx_defrag_blks() when the block table has
grown down to the payload while htx->head > 0, i.e. for a nearly full buffer
whose head was already consumed. All blocks then move down by the old value of
htx->head while the saved pointer does not follow, and htx_truncate_blk()
truncates at the wrong place, leaving partially converted trailers in the
message or removing valid blocks.
Only the trailers are really concerned: h3_resp_headers_to_htx() refuses to
work on a non-empty message, so no defragmentation can happen there, but it is
fixed the same way for consistency.
Let's save the amount of data present before the conversion and use
htx_truncate(), which works on a byte offset and is thus insensitive to any
block move, as htx_append_msg() already does.
This should be backported to all versions where the H3 trailers are supported,
so 2.8 and above.
BUG/MINOR: h2: don't use a block pointer to roll back a partial HTX conversion
h2_make_htx_request(), h2_make_htx_response() and h2_make_htx_trailers() save a
pointer on the tail HTX block on entry and use it on their error path to remove
everything they added:
An HTX block pointer is not a stable reference: it is computed from a block
*position* ("htx->blocks + htx->size - (pos + 1) * sizeof(struct htx_blk)"), so
it only designates the same block as long as the block table is not compacted.
htx_add_header()/htx_add_trailer() end up in htx_reserve_nxblk(), which calls
htx_defrag_blks() when the table has grown down to the payload while
htx->head > 0. After such a compaction all the blocks move down by the old
value of htx->head, but <tailblk> still points at the same address, hence at a
block located <head> positions further in the message. htx_truncate_blk() would
then truncate at the wrong place, either leaving partially converted headers or
trailers in the message, or dropping valid blocks that were there before.
This only concerns the trailers, and possibly the response on the backend side,
because the destination message may already hold payload with a partially
consumed head there, while a request is always converted into an empty buffer
which cannot defragment.
Let's save the amount of data present on entry and use htx_truncate() instead,
which relies on a byte offset and is therefore immune to any block move. This
is the same pattern as the one already used by htx_append_msg().
This has been there since the H2 to HTX conversion was introduced, so it should
be backported to all supported versions.
BUG/MINOR: h1: report the right error position on authority/host mismatch
When an absolute-form request target does not match the Host header value,
h1_headers_to_hdr_list() reports the error at two places depending on whether
the message must be blocked or only captured:
if (h1m->err_pos < -1) {
state = H1_MSG_LAST_LF;
ptr = host.ptr; /* Set ptr on the error */
goto http_msg_invalid;
}
if (h1m->err_pos == -1) /* capture the error pointer */
h1m->err_pos = v.ptr - start + skip; /* >= 0 now */
The strict path correctly points at the Host header value while the tolerant
path uses <v>, which at this point still holds the value of the *last* parsed
header field, whatever it was. So with "option
accept-unsafe-violations-in-http-request" enabled, the offset stored in
h1m->err_pos, later used by h1_capture_bad_message() and reported by "show
errors", designates an unrelated part of the message.
Let's use host.ptr in both paths.
This was introduced by commit 25bcdb1d9 ("BUG/MAJOR: h1: Be stricter on request
target validation during message parsing") in 3.0-dev12, so it should be
backported to 3.0 and above.
BUG/MINOR: http-htx: check the trash allocation in http_scheme_based_normalize()
http_scheme_based_normalize() uses a trash chunk to rebuild the target URI when
the default port must be dropped or when an empty path must be replaced by "/",
but it does not test the allocation:
struct buffer *temp = alloc_trash_chunk();
struct ist meth, vsn;
alloc_trash_chunk() takes its memory from pool_head_trash and returns NULL when
the pool is exhausted, in which case chunk_memcat() dereferences NULL and the
worker crashes. All the other users of alloc_trash_chunk() in the HTTP code
(http_act.c, http_ana.c) do test the result.
Normalization is performed on every absolute-form request URI (from H1, H2 and
H3), so this is reachable under memory pressure. Let's simply report a failure,
which the callers already handle as a rewrite error.
This was introduced by commit 4c0882b1b ("MEDIUM: http: implement scheme-based
normalization") in 2.5-dev2, so it should be backported to all supported
versions.
BUG/MINOR: http: fix an out-of-bounds read in http_get_host_port() on empty host
http_get_host_port() walks backwards from the end of the host looking for the
first non-digit, then checks whether it is a colon:
start = istptr(host);
end = istend(host);
for (ptr = end; ptr > start && isdigit((unsigned char)*--ptr););
/* no port found */
if (likely(*ptr != ':'))
return IST_NULL;
When <host> is empty, the loop condition fails immediately, the pre-decrement
is never evaluated and <ptr> is left equal to <end>, so *ptr reads one byte
past the end of the string. With an IST_NULL argument this is a NULL
dereference.
Both cases are reachable with an empty host: h1_validate_mismatch_authority()
calls it on the Host header value, which may be empty ("Host:\r\n") while an
absolute-form request URI is used, and http_scheme_based_normalize() calls it
on the authority extracted from the URI, which is empty for a request like
"GET http:///x HTTP/1.1". In practice the extra byte always lies inside the
request buffer, so the observable effect is limited to possibly mistaking a
neighbour byte for a colon and returning a bogus port, but it remains an
out-of-bounds read and the helper must be usable with an unset ist.
Let's return IST_NULL right away for an empty host.
This was introduced by commit 658f97162 ("MINOR: http: Add function to get port
part of a host") in 2.7-dev2, so it should be backported to 2.8 and above.
BUG/MINOR: http-fetch: fix a NULL channel dereference in smp_fetch_body()
smp_fetch_body() is also used from the health-check context, where the HTX
message comes from <check->bi> and where there is no channel at all: for
"res.body", SMP_RES_CHN(smp) evaluates to NULL because smp->strm is NULL. The
function is aware of this and guards the channel a few lines below:
if (!finished && (check || (chn && !channel_full(chn, global.tune.maxrewrite) &&
but the trash chunk retrieval isn't:
chk = get_best_trash_chunk(&chn->buf, htx->data);
so as soon as a check response is made of more than one HTX DATA block, a
bogus address close to NULL is dereferenced and the worker crashes. Let's use
the check input buffer when there is no channel, as done elsewhere.
Note that no reproducer could be built: with the h1 mux the successive DATA
blocks of a check response always end up appended to the same HTX block, so
the multi-block case was not reachable in practice. The dereference is
nevertheless plainly invalid.
This is a regression introduced by commit ac37158a6 ("BUG/MEDIUM: chunk:
Review chunks usage to not retrieve a large buffer by error") in 3.5-dev2,
which replaced get_trash_chunk_sz(htx->data), that did not look at the
channel, with get_best_trash_chunk(&chn->buf, htx->data). smp_fetch_body_param()
received the same change but is only reachable through "req.body_param", for
which <check> is always NULL and thus <chn> always set, so it is not affected.
It must be backported wherever the commit above was backported, so 3.4 and
above.
Only the upper bound is verified, and unlike "req.hdr" and friends, which rely
on val_hdr() to enforce the lower bound of the occurrence number, these two
keywords are declared with no argument checker at all. A negative identifier is
therefore accepted at boot, and at runtime req_cap[-1] is read; if the pointer
found there is not NULL it is then passed to strlen() and returned as a string.
This is trivially reproduced with a frontend containing:
capture request header Host len 32
http-request return status 200 hdr X-Cap "%[capture.req.hdr(-1)]"
which segfaults the worker on the very first request.
Since a negative capture identifier is meaningless, the cleanest fix is to
reject it at configuration parsing time, as is done for the header occurrence.
Let's add a val_cap_id() checker and reference it from both keywords.
Note that the "capture-req"/"capture-res" converters in http_conv.c index the
same array with an unchecked value too, but they are saved by the list walk
that precedes the access and which stops on a NULL <hdr>, so they only fail to
capture. They are left untouched.
This bug has been there since the keywords were introduced, so this should be
backported to all supported versions.
BUG/MEDIUM: http-fetch: don't parse a non-HTTP check buffer as an HTX message
In the health-check context, smp_prefetch_htx() unconditionally treats
<check->bi> as an HTX message:
if (!s || !chn) {
if (check) {
htx = htxbuf(&check->bi);
This is only true for an HTTP check, where the h1 mux fills the buffer with
HTX blocks. For any other check ruleset (plain "tcp-check", or the default TCP
connect check), the buffer holds the raw bytes received from the server.
All the response-side HTTP sample fetches (res.body, res.hdr, res.hdrs,
res.ver, status, res.cook...) are declared with SMP_SRC_HRSHV/HRSHP/HRSBO and
the validity table in sample.c makes them usable at the SMP_CKP_BE_CHK_RUL
checkpoint, so referencing one of them from a "tcp-check" rule (typically in an
"on-error" log-format string, or in a "status-code" expression) is accepted at
boot without any warning. When the check then runs, the first bytes of the
server's answer are used as "struct htx" fields: ->size, ->head, ->tail and
->first are entirely provided by the peer. htx_get_first_blk() computes
"htx->blocks + htx->size - (first + 1) * sizeof(struct htx_blk)" and the
resulting block is dereferenced, which is a wild read that crashes the worker,
and which may otherwise return arbitrary process memory as the fetched value.
and a server answering with 24 bytes crafted as a "struct htx" header with a
large ->size (e.g. 0x40000000), ->head = 0, ->tail = 1 and ->first = 0,
followed by any padding: the worker segfaults at the first check.
Let's simply refuse to look at the buffer when the check is not relying on
HTX. HTTP checks are unaffected, the fetches now simply return no sample on
other checks.
Note that this requires a hostile server, which is a trusted component for a
reverse proxy, so this is not a security issue, but the parser must not be fed
a buffer whose format it cannot assume.
This should be backported to all supported versions.
BUG/MINOR: http-htx: fix the length moved when removing a header value
http_remove_header() removes <len> bytes at the offset <off> of the header
value, <off> being "start - v.ptr" once <start> has been adjusted to also eat
the comma surrounding the removed value. The number of bytes that remain to be
moved down is therefore "v.len - off - len", but the function passes
"v.len - len" to memmove(), i.e. <off> bytes too many.
As a result, as soon as the removed value is not the first one of the
comma-delimited list (<off> != 0), memmove() reads <off> bytes past the end of
the header value and, when <off> is larger than <len>, writes (<off> - <len>)
bytes past the new end of the HTX block payload, silently corrupting whatever
follows it in the HTX buffer (typically the payload of the next blocks). For
instance with "x-test: aaaaaaaaaa,b,c", removing the second value gives
off = 10 and len = 2, hence 10 bytes read and 8 bytes written past the 20-byte
block payload. The resulting header value itself is not affected, which is
probably why this went unnoticed.
All the current callers happen to be safe: they either use <full> = 1 when
looking the header up, in which case the whole value matches and the block is
removed as a whole by the early return, or they only ever remove the first
value of the list ("Expect: 100-continue" in http_ana.c and "Age" in cache.c).
So this is only a latent out-of-bounds write for now, but the function is a
generic helper and the next caller removing a non-first value would corrupt
the message.
Let's simply compute the moved length from the end of the value. This can be
verified with the "http_htx" unit test added in the previous commit
(DEBUG_UNIT=1, then "haproxy -U http_htx").
This bug was introduced with the HTX conversion by commit 47596d378 ("MINOR:
http_htx: Add functions to manipulate HTX messages in http_htx.c") in 1.9-dev7
so it should be backported to all supported versions.
CLEANUP: haload: use <arg_thrd> instead of <global.nbthread> where applicable
Replace references to <global.nbthread> with <arg_thrd> across loops,
memory allocations, and thread ID distributions. Having two variables
sharing the same value and meaning is confusing, so it is cleaner to
rely exclusively on <arg_thrd>.
BUG/MINOR: haload: fix CPU topology detection by omitting forced "nbthread"
Always writing "nbthread" in the generated global configuration forced a
static thread count, preventing HAProxy from using its automatic CPU
topology detection and correct core binding. This caused severe performance
degradation on large multi-core machines. Fix this by omitting the "nbthread"
directive when -t is not explicitly specified, while ensuring <arg_thrd>
is properly initialized to global.nbthread for internal calculations such as
connection and request rates.
Also ensure <arg_thrd> <= <arg_usr> when -R is specified for internal
calculations.
MINOR: mux-h1: Use htx version to send default low-level errors
When an error was returned by the H1 multiplexer, the raw version was used
for default low-level errors. It is not really an issue, but it is more
consitent to use the HTX version. This way, by default, header names for
such errors are now sent in lower case. And the case of header names can
still be adjusted if necessary, thanks to the previous fix.
It is not really a bug, there is no reason to backport it, except if someone
ask for it.
MINOR: mux-h1: Lower the case for Sec-Websocket-* headers when manually added
During the message formatting, if websocket handshake key must be added by
the mux itself, the corresponding headers were not inserted in lowercase as
expected. It is not really an issue but it is not consistant with processing
on other headers. So let's fix it.
BUG/MEDIUM: mux-h1: Always adjust case for all outgoing headers as expected
In HTX, all header names are converted to lowercase. However, some legacy H1
applications are still sensitive to the header names case. For this
purpose, it is possible to provide a map to automatically adjust the case of
header names. While it is performed for most responses, it is not true for
the low-level errors triggered during requests parsing. It the same ways,
the case of "Sec-Websocket-Key" a "Sec-Websocket-Accept" headers were
adjusted as expected.
To fix this issue the h1-htx API was slightly changed. Now the map used to
adjust the case of header names, if any, is passed to the function
responsible to format the headers. In the H1 multiplexer, the map is first
retrieved then passed as argument to h1_format_htx_hdr() and
h1_format_htx_msg() functions. A NULL pointer is passed if no rewrite must
be performed.
In the H1 multiplexer, h1_adjust_case_outgoing_hdr() was replaced by
h1_get_hdrs_map().
All other calls to h1_format_htx_hdr() were adapted to use a NULL pointer
(httpclient, http-fetch).
This patch relies on "REORG: h1-htx: Move h1 headers map in h1-htx". Both
commits should be backported to all supported versions.
The map used to adjust the H1 headers case was moved in h1-htx part. For
this purpose h1_htx-t.h file was added and h1_hdrs_map and h1_hdr_entry
structures were moved into this file.
Tim Duesterhus [Wed, 15 Jul 2026 15:14:00 +0000 (17:14 +0200)]
MINOR: halog: Add support filtering on header capture values using -hdr-match
This patch extends the existing support for printing captured header fields
(`-hdr`) by a new filter (`-hdr-match`) that only processes lines where the
given capture has a specific value. It works together with all existing filters
and output formats.
The full syntax is `-hdr-match <block>:<field>=<value>`, where <block> and <field>
work just like `-hdr` and `<value>` is an exact string match:
Example:
capture request header a len 50
capture request header b len 50
capture request header c len 50
capture response header d len 50
capture response header e len 50
capture response header f len 50
- `-hdr-match 1:1=foo` will filter for requests where `a` is equal to "foo".
- `-hdr-match "2:3=foo bar"` will filter for requests where `f` is equal to
"foo bar".
The chosen syntax leaves future scope for allowing `<block>:<field>*<value>`
for substring matches and `<block>:<field>^<value>` for prefix matches without
introducing a breaking change.
Tim Duesterhus [Wed, 15 Jul 2026 15:10:36 +0000 (17:10 +0200)]
CLEANUP: halog: Clean up naming for variables related to `-hdr` processing
This is in preparation of allowing to filter based on the values of a header
capture by having a common "capture" prefix for variables related to extracting
header captures.
Certain rare bugs may cause an fd_want_recv() or any of the other
operations being done on a closed FD. This triggers a BUG_ON() on the
next call trying to insert the FD but it's too late to figure when
this happened. Let's just add some BUG_ON_HOT() to detect an attempt
to modify the state of a closed FD so that the culprit is detected.
It will only be enabled with DEBUG_STRICT=2, since by design this may
never happen so it's not needed to enable it in default builds. It was
verified not to trigger on various tests.
OPTIM: tools: keep a cache of recent localtime() and gmtime()
The log subsystem already keeps a cache of the latest generated time
header to avoid paying the price of snprintf() notably. However, as
reported in GH issue #3444 by @zino7825, localtime() and gmtime() are
affected (at least in glibc) by a tzset_lock held during the call to
__tz_convert(), which ruins performance when logging time such as the
accept date. Even just "option httplog" sees its performance divided
by two on a 64-core machine, from 1.8M to 910k req/s.
The following config even goes down to 388k req/s:
defaults
mode http
frontend fe
bind :8001
log 127.0.0.1:5514 len 8192 local0
log-format '{"t":"%t","tr":"%tr","ci":"%ci"}'
http-request set-var(txn.now) date(),ltime("%Y-%m-%dT%H:%M:%S")
option httplog
default_backend be
backend be
server s1 198.18.0.31:8000
With native_queued_spin_lock_slowpath() taking 80% of the CPU, showing
that it's also involved in futexes.
This patch, suggested by @zino7825, implements a very simple thread-
local cache for the 4 previously seen values for both localtime() and
gmtime(). The cache is visited in reverse order so that most recently
updated values are visited first (the most likely ones to be used). A
test with the config above and 12.8M requests showed 38.4M lookups with
only 12k total misses. A more naive scan from 0 to N caused 59M misses.
With this patch, all variants of the tests above remain at native speed
without native_queued_spin_lock_slowpath() being noticeable at all. If
for any reason a log format was so complex that it needed more stored
local times, it would be easy to change the cache size by redefining
TIME_CACHE_BITS.
The functions are no longer inlined, they were moved to tools.c since
we'd rather avoid loops and complex constructs in inlined functions.
Co-authored-by: Jinho Kong <zino7825@users.noreply.github.com>
OPTIM/MEDIUM: proxy/server: avoid server list reordering on startup
Prior to this patch, each server parsed from the configuration was added
at the front of the proxy list. The list was then reversed once parsing
is finished to reflect the configuration order.
Now that proxy servers list has been converted to a doubly linked list,
there is no more a reason for this. Thus, this patch changes the server
insertion order on configuration parsing : this is now performed
directly at the end of the proxy list. Reversal is unnecessary and has
been removed, so post-config performance may be slightly improved.
Peers parsing is the only module which relies on the order insertion.
Thus it has been adapted to now use the last server in its proxy.
MAJOR: proxy: convert server list to a doubly linked struct list
Servers are stored in a list in their parent proxy. Prior to this patch,
this list was singly linked.
This patch converts the proxy server list to a doubly linked struct
list. Server <next> pointer is replaced by a struct list <el_px> attach
point.
The main benefit from this patch is that it removes the bottleneck
performance for add and delete server operations at runtime. As with
main proxies list conversion, this is labelled as major as it is an API
change.
Most of the changes are straightforward : for/while statements are
replaced by list_for_each_entry() macros. LIST_ISEMPTY() is now used to
detect if a proxy does not contain any server.
Server insertion at the front position during config parsing is kept at
the moment, with reordering on post parsing. With the current patch,
this is not strictly necessary so this will be removed in a next change.
MINOR: proxy: define server list iteration functions
Define wrappers function to iterate over a proxy servers list. These
functions are used when a standard for loop over the whole list is not
desirable.
This patch will ease the conversion of the proxy servers list into a
doubly linked struct list type, as the final conversion patch changes
will be smaller.
MINOR: server: do not return next server on srv_drop()
Previously, srv_drop() returned the next server entry in the parent
proxy. This was used as convenience during server iteration. However,
the code has evolved several times to better deal with the server
deletion risk. Currently, srv_drop() return value is only used in a
single place.
Thus, this patch simplifies srv_drop() by removing its return value. As
a side effect, this will also reduce the modification required to
convert the proxy server list to a doubly linked struct list.
MINOR: server: rename global servers_list to all_servers
Rename global <servers_list> to <all_servers>. This name better reflects
that it contains all the servers and servers a similar purpose to
<all_proxies> list.
BUILD: ssl: Do not use SSL3_MT_KEY_UPDATE, hardcode 24 instead
Not every SSL lib provides SSL3_MT_KEY_UPDATE, so just hardcode 24
instead.
This should be backported up to 2.8, when 91004114fe8816f848025fe71def4ea23e72a5f6 will be backported.
BUG/MEDIUM: ssl: Put CO_ER_SSL_KEYUPDATE at the right place
FOr some reason, CO_ER_SSL_KEYUPDATE has been put in the middle of the
enum containing all the different possible connection errors, but
reg-tests depend on their numeric value, so it broke them. Put it at the
end of the enum instead.
MEDIUM: ssl: Add a way to rate-limit TLSv1.3 KeyUpdate
Processing TLSv1.3 KeyUpdate is expensive in term of CPU, and in normal
usage there is very few reason to get a lot of them. So add a new
keyword, tune.ssl.keyupdate-rate-limit, that gives the maximum number of
KeyUpdate we're okay with receiving per second. The default is 100,
which should be enough. 0 means no rate-limiting at all.
This should mitigate the problem reported in Github issue #3450.
BUG/MEDIUM: ssl: Handle non-application data record while splicing
When using splicing with kTLS, if we receive a record that is not an
application data record, such as a KeyUpdate, then splicing will fail.
If that happens, temporarily disable splicing and go the regular way so that
recvmsg() is used, we get the record, and we can resume splicing.
Please not that KeyUpdate is still not handled with AWS-LC. Only recent
Linux kernels support it, and the code hasn't been written for that yet.
projects / thirdparty / haproxy.git / log
