Emmanuel Hocdet [Tue, 18 Feb 2020 14:27:32 +0000 (15:27 +0100)]
MINOR: ssl: resolve issuers chain later
The goal is to use the ckch to store data from a loaded PEM file or a
<payload> and only for that. This patch addresses the ckch->chain case.
Looking for the issuers chain, if no chain is present in the ckch, can
be done in ssl_sock_put_ckch_into_ctx(). This way it is possible to know
the origin of the certificate chain without an extra state.
Willy Tarreau [Wed, 26 Feb 2020 09:39:36 +0000 (10:39 +0100)]
MEDIUM: buffer: remove the buffer_wq lock
This lock was only needed to protect the buffer_wq list, but now we have
the mt_list for this. This patch simply turns the buffer_wq list to an
mt_list and gets rid of the lock.
It's worth noting that the whole buffer_wait thing still looks totally
wrong especially in a threaded context: the wakeup_cb() callback is
called synchronously from any thread and may end up calling some
connection code that was not expected to run on a given thread. The
whole thing should probably be reworked to use tasklets instead and be
a bit more centralized.
Willy Tarreau [Tue, 25 Feb 2020 17:14:02 +0000 (18:14 +0100)]
[RELEASE] Released version 2.2-dev3
Released version 2.2-dev3 with the following main changes :
- SCRIPTS: announce-release: place the send command in the mail's header
- SCRIPTS: announce-release: allow the user to force to overwrite old files
- SCRIPTS: backport: fix the master branch detection
- BUG/MINOR: http-act: Set stream error flag before returning an error
- BUG/MINOR: http-act: Fix bugs on error path during parsing of return actions
- BUG/MEDIUM: ssl/cli: 'commit ssl cert' wrong SSL_CTX init
- BUG/MEDIUM: tcp-rules: Fix track-sc* actions for L4/L5 TCP rules
- DOC: schematic of the SSL certificates architecture
- BUG/MAJOR: mux-h2: don't wake streams after connection was destroyed
- BUG/MINOR: unix: better catch situations where the unix socket path length is close to the limit
- BUILD: cirrus-ci: switch to "snap" images to unify openssl naming
- BUILD: cirrus-ci: workaround "pkg install" bug
- BUILD: cirrus-ci: add ERR=1 to freebsd builds
- BUG/MINOR: connection: correctly retry I/O on signals
- CLEANUP: mini-clist: simplify nested do { while(1) {} } while (0)
- BUILD: http_act: cast file sizes when reporting file size error
- BUG/MEDIUM: listener: only consider running threads when resuming listeners
- BUG/MINOR: listener: enforce all_threads_mask on bind_thread on init
- BUG/MINOR: tcp: avoid closing fd when socket failed in tcp_bind_listener
- MINOR: build: add aix72-gcc build TARGET and power{8,9} CPUs
- BUILD: travis-ci: no more allowed failures for openssl-1.0.2
- BUILD: travis-ci: harden builds, add ERR=1 (warning ought to be errors)
- BUILD: scripts/build-ssl.sh: use "uname" instead of ${TRAVIS_OS_NAME}
- BUG/MINOR: tcp: don't try to set defaultmss when value is negative
- SCRIPTS: make announce-release executable again
- BUG/MINOR: namespace: avoid closing fd when socket failed in my_socketat
- BUG/MEDIUM: muxes: Use the right argument when calling the destroy method.
- BUG/MINOR: mux-fcgi: Forbid special characters when matching PATH_INFO param
- CLEANUP: ssl: remove unused functions in openssl-compat.h
- MINOR: mux-fcgi: Make the capture of the path-info optional in pathinfo regex
- MINOR: tools: add is_idchar() to tell if a char may belong to an identifier
- MINOR: chunk: implement chunk_strncpy() to copy partial strings
- MINOR: sample/acl: use is_idchar() to locate the fetch/conv name
- MEDIUM: arg: make make_arg_list() stop after its own arguments
- MEDIUM: arg: copy parsed arguments into the trash instead of allocating them
- MEDIUM: arg: make make_arg_list() support quotes in arguments
- MINOR: sample: make sample_parse_expr() able to return an end pointer
- MEDIUM: log-format: make the LF parser aware of sample expressions' end
- BUG/MINOR: arg: report an error if an argument is larger than bufsize
- SCRIPTS: announce-release: use mutt -H instead of -i to include the draft
- BUILD: enable ERR=1 in github cygwin builds
- BUG/MINOR: arg: fix again incorrect argument length check
- MINOR: sample: regsub now supports backreferences
- BUG/MINOR: tools: also accept '+' as a valid character in an identifier
- MINOR: http-htx: Add a function to retrieve the headers size of an HTX message
- MINOR: filters: Forward data only if the last filter forwards something
- BUG/MINOR: filters: Count HTTP headers as filtered data but don't forward them
- BUG/MINOR: http-htx: Don't return error if authority is updated without changes
- BUG/MINOR: stream: Don't incr frontend cum_req counter when stream is closed
- BUG/MINOR: sample: exit regsub() in case of trash allocation error
- MINOR: ssl: add "issuers-chain-path" directive.
- REGTESTS: use "command -v" instead of "which"
- BUG/MINOR: http-ana: Matching on monitor-uri should be case-sensitive
- MINOR: http-ana: Match on the path if the monitor-uri starts by a /
- BUG/MINOR: ssl: Stop passing dynamic strings as format arguments
- BUG/MAJOR: http-ana: Always abort the request when a tarpit is triggered
- BUG/MINOR: mux: do not call conn_xprt_stop_recv() on buffer shortage
- MINOR: checks: do not call conn_xprt_stop_send() anymore
- CLEANUP: epoll: place the struct epoll_event in the stack
- MEDIUM: connection: remove the intermediary polling state from the connection
- MINOR: raw_sock: directly call fd_stop_send() and not conn_xprt_stop_send()
- MINOR: tcp/uxst/sockpair: use fd_want_send() instead of conn_xprt_want_send()
- MINOR: connection: remove the last calls to conn_xprt_{want,stop}_*
- CLEANUP: connection: remove the definitions of conn_xprt_{stop,want}_{send,recv}
- MINOR: connection: introduce a new receive flag: CO_RFL_READ_ONCE
- MINOR: mux-h1: pass CO_RFL_READ_ONCE to the lower layers when relevant
- MINOR: ist: add an iststop() function
- BUG/MINOR: http: http-request replace-path duplicates the query string
- CLEANUP: sample: use iststop instead of a for loop
- BUG/MEDIUM: shctx: make sure to keep all blocks aligned
- MINOR: compiler: move CPU capabilities definition from config.h and complete them
- BUG/MEDIUM: ebtree: don't set attribute packed without unaligned access support
- CLEANUP: http/h1: rely on HA_UNALIGNED_LE instead of checking for CPU families
- BUILD: fix recent build failure on unaligned archs
- MINOR: ssl: load the key from a dedicated file
- BUG/MINOR: ssl: load .key in a directory only after PEM
- MINOR: compiler: drop special cases of likely/unlikely for older compilers
- CLEANUP: conn: Do not pass a pointer to likely
- CLEANUP: net_helper: Do not negate the result of unlikely
- BUILD: remove obsolete support for -mregparm / USE_REGPARM
- CLEANUP: cfgparse: Fix type of second calloc() parameter
- BUILD: ssl: only pass unsigned chars to isspace()
- BUILD: general: always pass unsigned chars to is* functions
- BUG/MINOR: sample: fix the json converter's endian-sensitivity
- BUG/MEDIUM: ssl: fix several bad pointer aliases in a few sample fetch functions
- CLEANUP: fd: use a union in fd_rm_from_fd_list() to shut aliasing warnings
- CLEANUP: cache: use read_u32/write_u32 to access the cache entry's hash
- CLEANUP: stick-tables: use read_u32() to display a node's key
- CLEANUP: sample: use read_u64() in ipmask() to apply an IPv6 mask
- MINOR: pattern: fix all remaining strict aliasing issues
- CLEANUP: lua: fix aliasing issues in the address matching code
- CLEANUP: connection: use read_u32() instead of a cast in the netscaler parser
- BUILD: makefile: re-enable strict aliasing
- BUG/MINOR: connection: make sure to correctly tag local PROXY connections
- MINOR: compiler: add new alignment macros
- BUILD: ebtree: improve architecture-specific alignment
- MINOR: config: mark global.debug as deprecated
- BUILD: travis-ci: enable s390x builds
- MINOR: ssl/cli: 'show ssl cert' displays the chain
- MINOR: ssl/cli: 'show ssl cert'displays the issuer in the chain
- MINOR: ssl/cli: reorder 'show ssl cert' output
- CLEANUP: ssl: move issuer_chain tree and definition
- DOC: proxy-protocol: clarify IPv6 address representation in the spec
Willy Tarreau [Tue, 25 Feb 2020 17:04:39 +0000 (18:04 +0100)]
DOC: proxy-protocol: clarify IPv6 address representation in the spec
Daniel Barclay reported that the wording around "IPv6 addresses must be
indicated as series of 4 hex digits" is confusing and can be interpreted
two ways (only 4 digits or series of sets of 4 digits), so let's adjust
the wording to resolve this ambiguity.
CLEANUP: ssl: move issuer_chain tree and definition
Move the cert_issuer_tree outside the global_ssl structure since it's
not a configuration variable. And move the declaration of the
issuer_chain structure in types/ssl_sock.h
Reorder the 'show ssl cert' output so it's easier to see if the whole
chain is correct.
For a chain to be correct, an "Issuer" line must have the same
content as the next "Subject" line.
Example:
Subject: /C=FR/ST=Paris/O=HAProxy Test Certificate/CN=test.haproxy.local
Issuer: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 2/CN=ca2.haproxy.local
Chain Subject: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 2/CN=ca2.haproxy.local
Chain Issuer: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 1/CN=ca1.haproxy.local
Chain Subject: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 1/CN=ca1.haproxy.local
Chain Issuer: /C=FR/ST=Paris/O=HAProxy Test Root CA/CN=root.haproxy.local
MINOR: ssl/cli: 'show ssl cert'displays the issuer in the chain
For each certificate in the chain, displays the issuer, so it's easy to
know if the chain is right.
Also rename "Chain" to "Chain Subject".
Example:
Chain Subject: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 2/CN=ca2.haproxy.local
Chain Issuer: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 1/CN=ca1.haproxy.local
Chain Subject: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 1/CN=ca1.haproxy.local
Chain Issuer: /C=FR/ST=Paris/O=HAProxy Test Root CA/CN=root.haproxy.local
MINOR: ssl/cli: 'show ssl cert' displays the chain
Display the subject of each certificate contained in the chain in the
output of "show ssl cert <filename>".
Each subjects are on a unique line prefixed by "Chain: "
Example:
Chain: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 2/CN=ca2.haproxy.local
Chain: /C=FR/ST=Paris/O=HAProxy Test Intermediate CA 1/CN=ca1.haproxy.local
Willy Tarreau [Tue, 25 Feb 2020 10:27:22 +0000 (11:27 +0100)]
MINOR: config: mark global.debug as deprecated
This directive has never made any sense and has already caused trouble
by forcing the process to stay in foreground during the boot process.
Let's emit a warning mentioning it's deprecated and will be removed in
2.3.
Commit 2c315ee75e ("BUG/MEDIUM: ebtree: don't set attribute packed
without unaligned access support") addressed alignment issues in
ebtrees in a way that is not really optimal since it will leave holes
in eb32trees for example.
This fix is better in that it restores the packed attribute on ebnode
but enforces proper alignment on the carrying nodes where necessary.
This also has the benefit of closing holes wherever possible and to
align data to the minimally required size.
The only thing it cannot close is the 32-bit hole at the end of ebmbnode
due to the required 64-bit on certain archs but at least it guarantees
that the key correctly points to the end of the node and that there is
never a hole after it.
This is a better fix than the one above and should be backported to
branches where the one above will be backported.
Willy Tarreau [Sat, 22 Feb 2020 14:51:39 +0000 (15:51 +0100)]
MINOR: compiler: add new alignment macros
This commit adds ALWAYS_ALIGN(), MAYBE_ALIGN() and ATOMIC_ALIGN() to
be placed as delimitors inside structures to force alignment to a
given size. These depend on the architecture's capabilities so that
it is possible to always align, align only on archs not supporting
unaligned accesses at all, or only on those not supporting them for
atomic accesses (e.g. before a lock).
Willy Tarreau [Wed, 19 Feb 2020 14:10:00 +0000 (15:10 +0100)]
BUG/MINOR: connection: make sure to correctly tag local PROXY connections
As reported in issue #511, when sending an outgoing local connection
(e.g. health check) we must set the "local" tag and not a "proxy" tag.
The issue comes from historic support on v1 which required to steal the
address on the outgoing connection for such ones, creating confusion in
the v2 code which believes it sees the incoming connection.
In order not to risk to break existing setups which might rely on seeing
the LB's address in the connection's source field, let's just change the
connection type from proxy to local and keep the addresses. The protocol
spec states that for local, the addresses must be ignored anyway.
This problem has always existed, this can be backported as far as 1.5,
though it's probably not a good idea to change such setups, thus maybe
2.0 would be more reasonable.
Willy Tarreau [Tue, 25 Feb 2020 09:10:47 +0000 (10:10 +0100)]
BUILD: makefile: re-enable strict aliasing
For a very long time we've used to build without strict aliasing due to
very few places in the stick-tables code mostly, that initially we didn't
know how to deal with. The problem of doing this is that it encourages
to write possibly incorrect code such as the few SSL sample fetch functions
that were recently fixed.
All places causing aliasing errors on x86_64, i586, armv8, armv7 and
mips were fixed so it's about time to re-enable the warning hoping to
catch such errors early in the development cycle. As a bonus, this
removed about 5kB of code.
Willy Tarreau [Tue, 25 Feb 2020 08:43:22 +0000 (09:43 +0100)]
CLEANUP: sample: use read_u64() in ipmask() to apply an IPv6 mask
There were 8 strict aliasing warnings there due to the dereferences
casting to uint32_t of input and output. We can achieve the same using
two write_u64() and four read_u64() which do not cause this issue and
even let the compiler use 64-bit operations.
Willy Tarreau [Tue, 25 Feb 2020 08:35:07 +0000 (09:35 +0100)]
CLEANUP: cache: use read_u32/write_u32 to access the cache entry's hash
Enabling strict aliasing fails on the cache's hash which is a series of
20 bytes cast as u32. And in practice it could even fail on some archs
if the http_txn didn't guarantee the hash was properly aligned. Let's
use read_u32() to read the value and write_u32() to set it, this makes
sure the compiler emits the correct code to access these and knows about
the intentional aliasing.
Willy Tarreau [Tue, 25 Feb 2020 08:25:53 +0000 (09:25 +0100)]
CLEANUP: fd: use a union in fd_rm_from_fd_list() to shut aliasing warnings
Enabling strict aliasing fails in fd.c when using the double-word CAS,
let's get rid of the (void**)(void*)&cur_list junk and use a union
instead. This way the compiler knows they do alias.
Willy Tarreau [Tue, 25 Feb 2020 07:59:23 +0000 (08:59 +0100)]
BUG/MEDIUM: ssl: fix several bad pointer aliases in a few sample fetch functions
Sample fetch functions ssl_x_sha1(), ssl_fc_npn(), ssl_fc_alpn(),
ssl_fc_session_id(), as well as the CLI's "show cert details" handler
used to dereference the output buffer's <data> field by casting it to
"unsigned *". But while doing this could work before 1.9, it broke
starting with commit 843b7cbe9d ("MEDIUM: chunks: make the chunk struct's
fields match the buffer struct") which merged chunks and buffers, causing
the <data> field to become a size_t. The impact is only on 64-bit platform
and depends on the endianness: on little endian, there should never be any
non-zero bits in the field as it is supposed to have been zeroed before the
call, so it shouldbe harmless; on big endian, the high part of the value
only is written instead of the lower one, often making the result appear
4 billion times larger, and making such values dropped everywhere due to
being larger than a buffer.
It seems that it would be wise to try to re-enable strict-aliasing to
catch such errors.
Willy Tarreau [Tue, 25 Feb 2020 07:37:37 +0000 (08:37 +0100)]
BUG/MINOR: sample: fix the json converter's endian-sensitivity
About every time there's a pointer cast in the code, there's a hidden
bug, and this one was no exception, as it passes the first octet of the
native representation of an integer as a single-character string, which
obviously only works on little endian machines. On big-endian machines,
something as simple as "str(foo),json" only returns zeroes.
This bug was introduced with the JSON converter in 1.6-dev1 by commit 317e1c4f1e ("MINOR: sample: add "json" converter"), the fix may be
backported to all stable branches.
Willy Tarreau [Tue, 25 Feb 2020 07:16:33 +0000 (08:16 +0100)]
BUILD: general: always pass unsigned chars to is* functions
The isalnum(), isalpha(), isdigit() etc functions from ctype.h are
supposed to take an int in argument which must either reflect an
unsigned char or EOF. In practice on some platforms they're implemented
as macros referencing an array, and when passed a char, they either cause
a warning "array subscript has type 'char'" when lucky, or cause random
segfaults when unlucky. It's quite unconvenient by the way since none of
them may return true for negative values. The recent introduction of
cygwin to the list of regularly tested build platforms revealed a lot
of breakage there due to the same issues again.
So this patch addresses the problem all over the code at once. It adds
unsigned char casts to every valid use case, and also drops the unneeded
double cast to int that was sometimes added on top of it.
It may be backported by dropping irrelevant changes if that helps better
support uncommon platforms. It's unlikely to fix bugs on platforms which
would already not emit any warning though.
It's caused by a signed char being passed to isspace(), and this one
being implemented as a macro instead of a function as the man page
suggests. It's the same issue that regularly pops up on Solaris. This
comes from commit 98263291cc3 which was merged in 1.8-dev1. A backport
is possible though not incredibly useful.
Tim Duesterhus [Sat, 22 Feb 2020 15:39:05 +0000 (16:39 +0100)]
CLEANUP: cfgparse: Fix type of second calloc() parameter
`curr_idle_thr` is of type `unsigned int`, not `int`. Fix this issue by
taking the size of the dereferenced `curr_idle_thr` array.
This issue was introduced when adding the `curr_idle_thr` struct member
in commit f131481a0af79037bc6616edf450ae81d80084d7. This commit is first
tagged in 2.0-dev1 and marked for backport to 1.9.
Willy Tarreau [Tue, 25 Feb 2020 06:38:05 +0000 (07:38 +0100)]
BUILD: remove obsolete support for -mregparm / USE_REGPARM
This used to be a minor optimization on ix86 where registers are scarce
and the calling convention not very efficient, but this platform is not
relevant enough anymore to warrant all this dirt in the code for the sake
of saving 1 or 2% of performance. Modern platforms don't use this at all
since their calling convention already defaults to using several registers
so better get rid of this once for all.
Tim Duesterhus [Fri, 21 Feb 2020 12:02:04 +0000 (13:02 +0100)]
CLEANUP: net_helper: Do not negate the result of unlikely
This patch turns the double negation of 'not unlikely' into 'likely'
and then turns the negation of 'not smaller' into 'greater or equal'
in an attempt to improve readability of the condition.
[wt: this was not a bug but purposely written like this to improve code
generation on older compilers but not needed anymore as described here:
https://www.mail-archive.com/haproxy@formilux.org/msg36392.html ]
Tim Duesterhus [Fri, 21 Feb 2020 12:02:03 +0000 (13:02 +0100)]
CLEANUP: conn: Do not pass a pointer to likely
Move the `!` inside the likely and negate it to unlikely.
The previous version should not have caused issues, because it is converted
to a boolean / integral value before being passed to __builtin_expect(), but
it's certainly unusual.
[wt: this was not a bug but purposely written like this to improve code
generation on older compilers but not needed anymore as described here:
https://www.mail-archive.com/haproxy@formilux.org/msg36392.html ]
Willy Tarreau [Tue, 25 Feb 2020 06:14:43 +0000 (07:14 +0100)]
MINOR: compiler: drop special cases of likely/unlikely for older compilers
We used to special-case the likely()/unlikely() macros for a series of
early gcc 4.x compilers which used to produce very bad code when using
__builtin_expect(x,1), which basically used to build an integer (0 or 1)
from a condition then compare it to integer 1. This was already fixed in
5.x, but even now, looking at the code produced by various flavors of 4.x
this bad behavior couldn't be witnessed anymore. So let's consider it as
fixed by now, which will allow to get rid of some ugly tricks at some
specific places. A test on 4.7.4 shows that the code shrinks by about 3kB
now, thanks to some tests being inlined closer to the call place and the
unlikely case being moved to real functions. See the link below for more
background on this.
Willy Tarreau [Fri, 21 Feb 2020 16:40:25 +0000 (17:40 +0100)]
BUILD: fix recent build failure on unaligned archs
Last commit 2c315ee ("BUG/MEDIUM: ebtree: don't set attribute packed
without unaligned access support") accidently enclosed the semicolon
in the ifdef so it will fail when HA_UNALIGNED is not set.
Willy Tarreau [Fri, 21 Feb 2020 15:31:22 +0000 (16:31 +0100)]
CLEANUP: http/h1: rely on HA_UNALIGNED_LE instead of checking for CPU families
Now that we have flags indicating the CPU's capabilities, better use
them instead of missing some updates for new CPU families (ARMv8 was
missing there).
Willy Tarreau [Fri, 21 Feb 2020 14:47:36 +0000 (15:47 +0100)]
BUG/MEDIUM: ebtree: don't set attribute packed without unaligned access support
An alignment issue on Sparc64 with ebtrees was reported in early 2017
here https://www.mail-archive.com/haproxy@formilux.org/msg25937.html and
a similar one was finally reported in issue #512.
The problem has its roots in the fact that 64-bit keys will end up being
unaligned on such archs which do not support unaligned accesses. But on
most platforms supporting unaligned accesses, dealing with smaller nodes
results in better performance.
One of the possible problems caused by attribute packed there is that it
promotes a structure both to be unaligned and unpadded, which may come
with fun if some fields of the struct itself are accessed and such a node
is placed at an unaligned location. It's not a problem for regular unaligned
accesses but may become one for atomic operations such as the CAS on leaf_p
that's used in struct task. In practice we know that this struct is properly
aligned and is a very edge case so this patch adds comments there to remind
to be careful about it.
This patch depends on previous patch "MINOR: compiler: move CPU capabilities
definition from config.h and complete them" and could be backported to all
stable branches. It fixes issues #512 and #9.
Willy Tarreau [Fri, 21 Feb 2020 14:40:58 +0000 (15:40 +0100)]
MINOR: compiler: move CPU capabilities definition from config.h and complete them
These ones are irrelevant to the config but rather to the platform, and
as such are better placed in compiler.h.
Here we take the opportunity for declaring a few extra capabilities:
- HA_UNALIGNED : CPU supports unaligned accesses
- HA_UNALIGNED_LE : CPU supports unaligned accesses in little endian
- HA_UNALIGNED_FAST : CPU supports fast unaligned accesses
- HA_UNALIGNED_ATOMIC : CPU supports unaligned accesses in atomics
This will help remove a number of #ifdefs with arch-specific statements.
Willy Tarreau [Fri, 21 Feb 2020 12:45:58 +0000 (13:45 +0100)]
BUG/MEDIUM: shctx: make sure to keep all blocks aligned
The blocksize and the extra field are not necessarily aligned on a
machine word. This can result in crashing an align-sensitive machine
when initializing the shctx area. Let's round both sizes up to a pointer
size to make this safe everywhere.
This fixes issue #512. This should be backported as far as 1.8.
Jerome Magnin [Fri, 21 Feb 2020 09:49:12 +0000 (10:49 +0100)]
CLEANUP: sample: use iststop instead of a for loop
In sample_fetch_path we can use iststop() instead of a for loop
to find the '?' and return the correct length. This requires commit
"MINOR: ist: add an iststop() function".
Jerome Magnin [Fri, 21 Feb 2020 09:37:48 +0000 (10:37 +0100)]
BUG/MINOR: http: http-request replace-path duplicates the query string
In http_action_replace_uri() we call http_get_path() in the case of
a replace-path rule. http_get_path() will return an ist pointing to
the start of the path, but uri.ptr + uri.len points to the end of the
uri. As as result, we are matching against a string containing the
query, which we append to the "path" later, effectively duplicating
the query string.
This patch uses the iststop() function introduced in "MINOR: ist: add
an iststop() function" to find the '?' character and update the ist
length when needed.
This fixes issue #510.
The bug was introduced by commit 262c3f1a ("MINOR: http: add a new
"replace-path" action"), which was backported to 2.1 and 2.0.
Jerome Magnin [Fri, 21 Feb 2020 09:33:12 +0000 (10:33 +0100)]
MINOR: ist: add an iststop() function
Add a function that finds a character in an ist and returns an
updated ist with the length of the portion of the original string
that doesn't contain the char.
Willy Tarreau [Thu, 20 Feb 2020 10:11:50 +0000 (11:11 +0100)]
MINOR: mux-h1: pass CO_RFL_READ_ONCE to the lower layers when relevant
When we're in H1_MSG_RQBEFORE or H1_MSG_RPBEFORE, we know that the
first message is highly likely the only one and that it's pointless
to try to perform a second recvfrom() to complete a first partial
read. This is similar to what used to be done in the older I/O methods
with the CF_READ_DONTWAIT flag on the channel. So let's pass
CO_RFL_READ_ONCE to the transport layer during rcv_buf() in this case.
By doing so, in a test involving keep-alive connections with a non-null
client think time, we remove 20% of the recvfrom() calls, all of which
used to systematically fail. More precisely, we observe a drop from 5.0
recvfrom() per request with 60% failure to 4.0 per request with 50%
failure.
Willy Tarreau [Thu, 20 Feb 2020 10:04:40 +0000 (11:04 +0100)]
MINOR: connection: introduce a new receive flag: CO_RFL_READ_ONCE
This flag is currently supported by raw_sock to perform a single recv()
attempt and avoid subscribing. Typically on the request and response
paths with keep-alive, with short messages we know that it's very likely
that the first message is enough.
Willy Tarreau [Fri, 21 Feb 2020 08:58:29 +0000 (09:58 +0100)]
CLEANUP: connection: remove the definitions of conn_xprt_{stop,want}_{send,recv}
This marks the end of the transition from the connection polling states
introduced in 1.5-dev12 and the subscriptions in that arrived in 1.9.
The socket layer can now safely use its FD while all upper layers rely
exclusively on subscriptions. These old functions were removed. Some may
deserve some renaming to improved clarty though. The single call to
conn_xprt_stop_both() was dropped in favor of conn_cond_update_polling()
which already does the same.
Willy Tarreau [Fri, 21 Feb 2020 09:34:19 +0000 (10:34 +0100)]
MINOR: connection: remove the last calls to conn_xprt_{want,stop}_*
The last few calls to conn_xprt_{want,stop}_{recv,send} in the central
connection code were replaced with their strictly exact equivalent fd_*,
adding the call to conn_ctrl_ready() when it was missing.
Willy Tarreau [Fri, 21 Feb 2020 09:24:51 +0000 (10:24 +0100)]
MINOR: tcp/uxst/sockpair: use fd_want_send() instead of conn_xprt_want_send()
Just like previous commit, we don't need to pass through the connection
layer anymore to enable polling during a connect(), we know the FD, so
let's simply call fd_want_send().
Willy Tarreau [Fri, 21 Feb 2020 09:21:46 +0000 (10:21 +0100)]
MINOR: raw_sock: directly call fd_stop_send() and not conn_xprt_stop_send()
Now that we know that the connection layer is transparent for polling
changes, we have no reason for hiding behind conn_xprt_stop_send() and
can safely call fd_stop_send() on the FD once the buffer is empty.
Willy Tarreau [Fri, 21 Feb 2020 07:46:19 +0000 (08:46 +0100)]
MEDIUM: connection: remove the intermediary polling state from the connection
Historically we used to require that the connections held the desired
polling states for the data layer and the socket layer. Then with muxes
these were more or less merged into the transport layer, and now it
happens that with all transport layers having their own state, the
"transport layer state" as we have it in the connection (XPRT_RD_ENA,
XPRT_WR_ENA) is only an exact copy of the undelying file descriptor
state, but with a delay. All of this is causing some difficulties at
many places in the code because there are still some locations which
use the conn_want_* API to remain clean and only rely on connection,
and count on a later collection call to conn_cond_update_polling(),
while others need an immediate action and directly use the FD updates.
Since our updates are now much cheaper, most of them being only an
atomic test-and-set operation, and since our I/O callbacks are deferred,
there's no benefit anymore in trying to "cache" the transient state
change in the connection flags hoping to cancel them before they
become an FD event. Better make such calls transparent indirections
to the FD layer instead and get rid of the deferred operations which
needlessly complicate the logic inside.
This removes flags CO_FL_XPRT_{RD,WR}_ENA and CO_FL_WILL_UPDATE.
A number of functions related to polling updates were either greatly
simplified or removed.
Two places were using CO_FL_XPRT_WR_ENA as a hint to know if more data
were expected to be sent after a PROXY protocol or SOCKSv4 header. These
ones were simply replaced with a check on the subscription which is
where we ought to get the autoritative information from.
Now the __conn_xprt_want_* and their conn_xprt_want_* counterparts
are the same. conn_stop_polling() and conn_xprt_stop_both() are the
same as well. conn_cond_update_polling() only causes errors to stop
polling. It also becomes way more obvious that muxes should not at
all employ conn_xprt_{want|stop}_{recv,send}(), and that the call
to __conn_xprt_stop_recv() in case a mux failed to allocate a buffer
is inappropriate, it ought to unsubscribe from reads instead. All of
this definitely requires a serious cleanup.
Willy Tarreau [Thu, 20 Feb 2020 10:23:43 +0000 (11:23 +0100)]
CLEANUP: epoll: place the struct epoll_event in the stack
Historically we used to have a global epoll_event for various
manipulations involving epoll_ctl() and when threads were added,
this was turned to a thread_local, which is needlessly expensive
since it's just a temporary variable. Let's move it to a local
variable wherever it's called instead.
Willy Tarreau [Fri, 21 Feb 2020 09:13:03 +0000 (10:13 +0100)]
MINOR: checks: do not call conn_xprt_stop_send() anymore
While trying to address issue #253, Commit 5909380c ("BUG/MINOR: checks:
stop polling for write when we have nothing left to send") made sure that
we stop polling for writes when the buffer is empty. This was actually
more a workaround than a bug fix because by doing so we may be stopping
polling for an intermediary transport layer without acting on the check
itself in case there's SSL or send-proxy in the chain for example, thus
the approach is wrong. In practice due to the small size of check
requests, this will not have any impact. At best, we ought to unsubscribe
for sending, but that's already the case when we arrive in this function.
But given that the root cause of the issue was addressed later in commits cc705a6b, c5940392 and ccf3f6d1, we can now safely revert this change.
It was confirmed on the faulty config that this change doesn't have any
effect anymore on the test.
Willy Tarreau [Fri, 21 Feb 2020 08:44:39 +0000 (09:44 +0100)]
BUG/MINOR: mux: do not call conn_xprt_stop_recv() on buffer shortage
In H1/H2/FCGI, the *_get_buf() functions try to disable receipt of data
when there's no buffer available. But they do so at the lowest possible
level, which is unrelated to the upper transport layers which may still
be trying to feed data based on subscription. The correct approach here
would theorically be to only disable subscription, though when we get
there, the subscription will already have been dropped, so we can safely
just remove that call.
It's unlikely that this could have had any practical impact, as the upper
xprt layer would call this callback which would fail an not resubscribe.
Having the lowest layer disabled would just be temporary since when
re-enabling reading, a subscribe at the end of data would re-enable it.
Backport should not harm but seems useless at this point.
BUG/MAJOR: http-ana: Always abort the request when a tarpit is triggered
If an client error is reported on the request channel (CF_READ_ERROR) while a
session is tarpitted, no error is returned to the client. Concretly,
http_reply_and_close() function is not called. This function is reponsible to
forward the error to the client. But not only. It is also responsible to abort
the request. Because this function is not called when a read error is reported
on the request channel, and because the tarpit analyzer is the last one, there
is nothing preventing a connection attempt on a server while it is totally
unexpected.
So, a useless connexion on a backend server may be performed because of this
bug. If an HTTP load-balancing algorithm is used on the backend side, it leads
to a crash of HAProxy because the request was already erased.
If you have tarpit rules and if you use an HTTP load-balancing algorithm on your
backends, you must apply this patch. Otherwise a simple TCP reset on a tarpitted
connexion will most likely crash your HAProxy. A safe workaround is to use a
silent-drop rule or a deny rule instead of a tarpit.
This bug also affect the legacy code. It is in fact an very old hidden bug. But
the refactoring of process_stream() in the 1.9 makes it visible. And,
unfortunately, with the HTX, it is easier to hit it because many processing has
been moved in lower layers, in the muxes.
It must be backported as far as 1.9. For the 2.0 and the 1.9, the legacy HTTP
code must also be patched the same way. For older versions, it may be backported
but the bug seems to not impact them.
Thanks to Olivier D <webmaster@ajeux.com> to have reported the bug and provided
all the infos to analyze it.
Tim Duesterhus [Wed, 19 Feb 2020 10:41:13 +0000 (11:41 +0100)]
BUG/MINOR: ssl: Stop passing dynamic strings as format arguments
gcc complains rightfully:
src/ssl_sock.c: In function ‘ssl_load_global_issuers_from_path’:
src/ssl_sock.c:9860:4: warning: format not a string literal and no format arguments [-Wformat-security]
ha_warning(warn);
^
MINOR: http-ana: Match on the path if the monitor-uri starts by a /
if the monitor-uri starts by a slash ('/'), the matching is performed against
the request's path instead of the request's uri. It is a workaround to let the
HTTP/2 requests match the monitor-uri. Indeed, in HTTP/2, clients are encouraged
to send absolute URIs only.
This patch is not tagged as a bug, because the previous behavior matched exactly
what the doc describes. But it may surprise that HTTP/2 requests don't match the
monitor-uri.
This patch may be backported to 2.1 because URIs of HTTP/2 are stored using the
absolute-form starting this version. For previous versions, this patch will only
helps explicitely absolute HTTP/1 requests (and only the HTX part because on the
legacy HTTP, all the URI is matched).
BUG/MINOR: http-ana: Matching on monitor-uri should be case-sensitive
The monitor-uri should be case-sensitive. In reality, the scheme and the host
part are case-insensitives and only the path is case-sensive. But concretely,
since the start, the matching on the monitor-uri is case-sensitive. And it is
probably the expected behavior of almost all users.
This patch must be backported as far as 1.9. For HAProxy 2.0 and 1.9, it must be
applied on src/proto_htx.c.
Willy Tarreau [Tue, 18 Feb 2020 13:42:33 +0000 (14:42 +0100)]
REGTESTS: use "command -v" instead of "which"
Ilya reported that the "which" utility is not that much portable and is
absent from Fedora. "type -p" is not portable either, and the correct
solution appears to be "command -v", so let's use this for now, we can
change it again in the future in case of problems.
Emmanuel Hocdet [Fri, 4 Jan 2019 10:08:20 +0000 (11:08 +0100)]
MINOR: ssl: add "issuers-chain-path" directive.
Certificates loaded with "crt" and "crt-list" commonly share the same
intermediate certificate in PEM file. "issuers-chain-path" is a global
directive to share intermediate chain certificates in a directory. If
certificates chain is not included in certificate PEM file, haproxy
will complete chain if issuer match the first certificate of the chain
stored via "issuers-chain-path" directive. Such chains will be shared
in memory.
Willy Tarreau [Tue, 18 Feb 2020 13:27:44 +0000 (14:27 +0100)]
BUG/MINOR: sample: exit regsub() in case of trash allocation error
As reported in issue #507, since commiy 07e1e3c93e ("MINOR: sample:
regsub now supports backreferences") we must not proceed in regsub()
if we fali to allocate a trash (which in practice never happens). No
backport needed.
BUG/MINOR: stream: Don't incr frontend cum_req counter when stream is closed
This counter is already incremented when a new request is received (or if an
error occurred waiting it). So it must not be incremented when the stream is
terminated, at the end of process_strem(). This bug was introduced by the commit cff0f739e ("MINOR: counters: Review conditions to increment counters from
analysers").
BUG/MINOR: http-htx: Don't return error if authority is updated without changes
When an Host header is updated, the autority part, if any, is also updated to
keep the both syncrhonized. But, when the update is performed while there is no
change, a failure is reported while, in reality, no update is necessary. This
bug was introduced by the commit d7b7a1ce5 ("MEDIUM: http-htx: Keep the Host
header and the request start-line synchronized").
This commit was pushed in the 2.1. But on this version, the bug is hidden
because rewrite errors are silently ignored. And because it happens when there
is no change, if the rewrite fails, noone notices it. But since the 2.2, rewrite
errors are now fatals by default. So when the bug is hit, a 500 error is
returned to the client. Without this fix, a workaround is to disable the strict
rewriting mode (see the "strict-mode" HTTP rule).
The following HTTP rule is a good way to reproduce the bug if a request with an
authority is received. In HTT2, it is pretty common.
acl host_header_exists req.hdr(host) -m found
http-request set-header host %[req.hdr(host)] if host_header_exists
This patch must be backported to 2.1 and everywhere the commit d7b7a1ce5 is
backported. It should fix the issue #494.
BUG/MINOR: filters: Count HTTP headers as filtered data but don't forward them
In flt_analyze_http_headers() HTTP analyzer, we must not forward systematically
the headers. We must only count them as filtered data (ie. increment the offset
of the right size). It is the http_payload callback responsibility to decide to
forward headers or not by forwarding at least 1 byte of payload. And there is
always at least 1 byte of payload to forward, the EOM block.
This patch depends on following commits:
* MINOR: filters: Forward data only if the last filter forwards something
* MINOR: http-htx: Add a function to retrieve the headers size of an HTX message
This patch must be backported with commits above as far as 1.9. In HAProxy 2.0
and 1.9, the patch must be adapted because of the legacy HTTP code.
MINOR: filters: Forward data only if the last filter forwards something
In flt_tcp_payload() and flt_http_payload(), if the last filter does not
forwarding anything, nothing is forwarded, not even the already filtered
data. For now, this patch is useless because the last filter is always sync with
the stream's offset. But it will be mandatory for a bugfix.
MINOR: http-htx: Add a function to retrieve the headers size of an HTX message
http_get_hdrs_size() function may now be used to get the bytes held by headers
in an HTX message. It only works if the headers were not already
forwarded. Metadata are not counted here.
Willy Tarreau [Mon, 17 Feb 2020 05:34:11 +0000 (06:34 +0100)]
BUG/MINOR: tools: also accept '+' as a valid character in an identifier
The function is_idchar() was added by commit 36f586b ("MINOR: tools:
add is_idchar() to tell if a char may belong to an identifier") to
ease matching of sample fetch/converter names. But it lacked support
for the '+' character used in "base32+src" and "url32+src". A quick
way to figure the list of supported sample fetch+converter names is
to issue the following command:
Jerome Magnin [Sun, 16 Feb 2020 18:20:19 +0000 (19:20 +0100)]
MINOR: sample: regsub now supports backreferences
Now that the configuration parser is more flexible with samples,
converters and their arguments, we can leverage this to enable
support for backreferences in regsub.
Willy Tarreau [Sun, 16 Feb 2020 09:46:37 +0000 (10:46 +0100)]
BUG/MINOR: arg: fix again incorrect argument length check
Recent commit 807aef8a14 ("BUG/MINOR: arg: report an error if an argument
is larger than bufsize") aimed at fixing the argument length check but
relied on the fact that the end of string was not reached, except that
it forgot to consider the delimiters (comma and parenthesis) which are
valid conditions to break out of the loop. This used to break simple
expressions like "hdr(xff,1)". Thanks to Jérôme for reporting this.
Willy Tarreau [Sat, 15 Feb 2020 14:24:28 +0000 (15:24 +0100)]
SCRIPTS: announce-release: use mutt -H instead of -i to include the draft
Commit 0f5ce6014a ("SCRIPTS: announce-release: place the send command
in the mail's header") broke the announce-release script: by not having
to edit the message at all anymore, mutt does nothing when sending, but
it still does if the message is edited (which was the case before). With
some testing, it appears that mutt -H does work when there's no change,
so let's use this instead. This should be backported till 1.7.
Willy Tarreau [Sat, 15 Feb 2020 13:54:28 +0000 (14:54 +0100)]
BUG/MINOR: arg: report an error if an argument is larger than bufsize
Commit ef21facd99 ("MEDIUM: arg: make make_arg_list() support quotes
in arguments") removed the chunk_strncpy() to fill the trash buffer
as the input is being parsed, and accidently dropped the jump to the
error path in case the argument is too large, which is now fixed.
No backport is needed, this is for 2.2. This addresses issue #502.
Willy Tarreau [Fri, 14 Feb 2020 16:33:06 +0000 (17:33 +0100)]
MEDIUM: log-format: make the LF parser aware of sample expressions' end
For a very long time it used to be impossible to pass a closing square
bracket as a valid character in argument to a sample fetch function or
to a converter because the LF parser used to stop on the first such
character found and to pass what was between the first '[' and the first
']' to sample_parse_expr().
This patch addresses this by passing the whole string to sample_parse_expr()
which is the only one authoritative to indicate the first character that
does not belong to the expression. The LF parser then verifies it matches
a ']' or fails. As a result it is finally possible to write rules such as
the following, which is totally valid an unambigous :
Willy Tarreau [Fri, 14 Feb 2020 15:50:14 +0000 (16:50 +0100)]
MINOR: sample: make sample_parse_expr() able to return an end pointer
When an end pointer is passed, instead of complaining that a comma is
missing after a keyword, sample_parse_expr() will silently return the
pointer to the current location into this return pointer so that the
caller can continue its parsing. This will be used by more complex
expressions which embed sample expressions, and may even permit to
embed sample expressions into arguments of other expressions.
Willy Tarreau [Fri, 14 Feb 2020 12:37:20 +0000 (13:37 +0100)]
MEDIUM: arg: make make_arg_list() support quotes in arguments
Now it becomes possible to reuse the quotes within arguments, allowing
the parser to distinguish a ',' or ')' that is part of the value from
one which delimits the argument. In addition, ',' and ')' may be escaped
using a backslash. However, it is also important to keep in mind that
just like in shell, quotes are first resolved by the word tokenizer, so
in order to pass quotes that are visible to the argument parser, a second
level is needed, either using backslash escaping, or by using an alternate
type.
For example, it's possible to write this to append a comma:
Note that due to the wide use of '\' in front of parenthesis in regex,
the backslash character will purposely *not* escape parenthesis, so that
'\)' placed in quotes is passed verbatim to a regex engine.
Willy Tarreau [Fri, 14 Feb 2020 10:34:35 +0000 (11:34 +0100)]
MEDIUM: arg: copy parsed arguments into the trash instead of allocating them
For each and every argument parsed by make_arg_list(), there was an
strndup() call, just so that we have a trailing zero for most functions,
and this temporary buffer is released afterwards except for strings where
it is kept.
Proceeding like this is not convenient because 1) it performs a huge
malloc/free dance, and 2) it forces to decide upfront where the argument
ends, which is what prevents commas and right parenthesis from being used.
This patch makes the function copy the temporary argument into the trash
instead, so that we avoid the malloc/free dance for most all non-string
args (e.g. integers, addresses, time, size etc), and that we can later
produce the contents on the fly while parsing the input. It adds a length
check to make sure that the argument is not longer than the buffer size,
which should obviously never be the case but who knows what people put in
their configuration.
Willy Tarreau [Fri, 14 Feb 2020 07:40:37 +0000 (08:40 +0100)]
MEDIUM: arg: make make_arg_list() stop after its own arguments
The main problem we're having with argument parsing is that at the
moment the caller looks for the first character looking like an end
of arguments (')') and calls make_arg_list() on the sub-string inside
the parenthesis.
Let's first change the way it works so that make_arg_list() also
consumes the parenthesis and returns the pointer to the first char not
consumed. This will later permit to refine each argument parsing.
Willy Tarreau [Fri, 14 Feb 2020 17:27:10 +0000 (18:27 +0100)]
MINOR: sample/acl: use is_idchar() to locate the fetch/conv name
Instead of scanning a string looking for an end of line, ')' or ',',
let's only accept characters which are actually valid identifier
characters. This will let the parser know that in %[src], only "src"
is the sample fetch name, not "src]". This was done both for samples
and ACLs since they are the same here.
Willy Tarreau [Fri, 14 Feb 2020 10:31:41 +0000 (11:31 +0100)]
MINOR: chunk: implement chunk_strncpy() to copy partial strings
This does like chunk_strcpy() except that the maximum string length may
be limited by the caller. A trailing zero is always appended. This is
particularly handy to extract portions of strings to put into the trash
for use with libc functions requiring a nul-terminated string.
MINOR: mux-fcgi: Make the capture of the path-info optional in pathinfo regex
Now, only one capture is mandatory in the path-info regex, the one matching the
script-name. The path-info capture is optional. Of couse, it must be defined to
fill the PATH_INFO parameter. But it is not mandatory. This way, it is possible
to get the script-name part from the path, excluding the path-info.
This patch is small enough to be backported to 2.1.
BUG/MINOR: mux-fcgi: Forbid special characters when matching PATH_INFO param
If a regex to match the PATH_INFO parameter is configured, it systematically
fails if a newline or a null character is present in the URL-decoded path. So,
from the moment there is at least a "%0a" or a "%00" in the request path, we
always fail to get the PATH_INFO parameter and all the decoded path is used for
the SCRIPT_NAME parameter.
It is probably not the expected behavior. Because, most of time, these
characters are not expected at all in a path, an error is now triggered when one
of these characters is found in the URL-decoded path before trying to execute
the path_info regex. However, this test is not performed if there is no regex
configured.
Note that in reality, the newline character is only a problem when HAProxy is
complied with pcre or pcre2 library and conversely, the null character is only a
problem for the libc's regex library. But both are always excluded to avoid any
inconsistency depending on compile options.
An alternative, not implemented yet, is to replace these characters by another
one. If someone complains about this behavior, it will be re-evaluated.
This patch must be backported to all versions supporting the FastCGI
applications, so to 2.1 for now.
Olivier Houchard [Fri, 14 Feb 2020 12:23:45 +0000 (13:23 +0100)]
BUG/MEDIUM: muxes: Use the right argument when calling the destroy method.
When calling the mux "destroy" method, the argument should be the mux
context, not the connection. In a few instances in the mux code, the
connection was used (mainly when the session wouldn't handle the idle
connection, and the server pool was fool), and that could lead to random
segfaults.
Willy Tarreau [Wed, 12 Feb 2020 17:21:11 +0000 (18:21 +0100)]
SCRIPTS: make announce-release executable again
I managed to mess up with the file's permission while using a temporary
one during last release, and to backport the non-exec version everywhere.
This can be backported as far as 1.7 now.
MINOR: build: add aix72-gcc build TARGET and power{8,9} CPUs
As haproxy wont build on AIX 7.2 using the old "aix52" TARGET a new
TARGET was introduced which adds two special CFLAGS to prevent the
loading of AIXs xmem.h and var.h. This is done by defining the
corresponding include-guards _H_XMEM and _H_VAR. Without excluding
those headers-files the build fails because of redefinition errors:
1)
CC src/mux_fcgi.o
In file included from /usr/include/sys/uio.h:90,
from /opt/freeware/lib/gcc/powerpc-ibm-aix7.1.0.0/8.3.0/include-fixed/sys/socket.h:104,
from include/common/compat.h:32,
from include/common/cfgparse.h:25,
from src/mux_fcgi.c:13:
src/mux_fcgi.c:204:13: error: expected ':', ',', ';', '}' or '__attribute__' before '.' token
struct ist rem_addr;
^~~~~~~~
2)
CC src/cfgparse-listen.o
In file included from include/types/arg.h:31,
from include/types/acl.h:29,
from include/types/proxy.h:41,
from include/proto/log.h:34,
from include/common/cfgparse.h:30,
from src/mux_h2.c:13:
include/types/vars.h:30:8: error: redefinition of 'struct var'
struct var {
^~~
Futhermore, to enable multithreading via USE_THREAD, the atomic
library was added to the LDFLAGS. Finally, two new CPUs were added
to simplify the usage of power8 and power9 optimizations.
This TARGET was only tested on GCC 8.3 and may or may not work on
IBM's native C-compiler (XLC).
Willy Tarreau [Wed, 12 Feb 2020 09:15:34 +0000 (10:15 +0100)]
BUG/MINOR: listener: enforce all_threads_mask on bind_thread on init
When intializing a listener, let's make sure the bind_thread mask is
always limited to all_threads_mask when inserting the FD. This will
avoid seeing listening FDs with bits corresponding to threads that are
not active (e.g. when using "bind ... process 1/even"). The side effect
is very limited, all that was identified is that atomic operations are
used in fd_update_events() when not necessary. It's more a matter of
long-term correctness in practice.
This fix might be backported as far as 1.8 (then proto_sockpair must
be dropped).
Willy Tarreau [Wed, 12 Feb 2020 09:01:29 +0000 (10:01 +0100)]
BUG/MEDIUM: listener: only consider running threads when resuming listeners
In bug #495 we found that it is possible to resume a listener on an
inexistent thread. This happens when a bind's thread_mask contains bits
out of the active threads mask, such as when using "1/odd" or "1/even".
The thread_mask was used as-is to pick a thread number to re-enable the
listener, and given that the highest number is used, 1/odd or 1/even can
produce quite high thread numbers and crash the process by queuing some
entries into non-existent lists.
This bug is an incomplete fix of commit 413e926ba ("BUG/MAJOR: listener:
fix thread safety in resume_listener()") though it will only trigger if
some bind lines are explicitly bound to thread numbers higher than the
thread count. The fix must be backported to all branches having the fix
above (as far as 1.8, though the code is different there, see the commit
message in 1.8 for changes).
There are a few other places where bind_thread is used without
enforcing all_thread_mask, namely when doing fd_insert() while creating
listeners. It seems harmless but would probably deserve another fix.
Willy Tarreau [Tue, 11 Feb 2020 09:17:52 +0000 (10:17 +0100)]
CLEANUP: mini-clist: simplify nested do { while(1) {} } while (0)
While looking for other occurrences of do { continue; } while (0) I
found these few leftovers in mini-clist where an outer loop was made
around "do { } while (0)" then another loop was placed inside just to
handle the continue. Let's clean this up by just removing the outer
one. Most of the patch is only the inner part of the loop that is
reindented. It was verified that the resulting code is the same.
Willy Tarreau [Tue, 11 Feb 2020 09:08:05 +0000 (10:08 +0100)]
BUG/MINOR: connection: correctly retry I/O on signals
Issue #490 reports that there are a few bogus constructs of the famous
"do { if (cond) continue; } while (0)" in the connection code, that are
used to retry on I/O failures caused by receipt of a signal. Let's turn
them into the more correct "while (1) { if (cond) continue; break }"
instead. This may or may not be backported, it shouldn't have any
visible effect.
Ilya Shipitsin [Tue, 11 Feb 2020 08:16:02 +0000 (13:16 +0500)]
BUILD: cirrus-ci: switch to "snap" images to unify openssl naming
"snap" images are updated frequently, while regular images are updated quarterly.
so, mixing "snap" and regular images lead to package naming mismatch, which will occur every
quarter. we cannot use 11.3 release image, it is broken, so we have to use 11.3 "snap" image.
Thus let us use all "snap" images. 13-snap is first introduced with this commit.
Willy Tarreau [Tue, 11 Feb 2020 05:43:37 +0000 (06:43 +0100)]
BUG/MINOR: unix: better catch situations where the unix socket path length is close to the limit
We do have some checks for the UNIX socket path length to validate the
full pathname of a unix socket but the pathname extension is only taken
into account when using a bind_prefix. The second check only matches
against MAXPATHLEN. So this means that path names between 98 and 108
might successfully parse but fail to bind. Let's adjust the check in
the address parser and refine the error checking at the bind() step.
Willy Tarreau [Tue, 11 Feb 2020 03:38:56 +0000 (04:38 +0100)]
BUG/MAJOR: mux-h2: don't wake streams after connection was destroyed
In commit 477902b ("MEDIUM: connections: Get ride of the xprt_done
callback.") we added an inconditional call to h2_wake_some_streams()
in h2_wake(), though we must not do it if the connection is destroyed
or we end up with a use-after-free. In this case it's already done in
h2_process() before destroying the connection anyway.
Let's just add this test for now. A cleaner approach might consist in
doing it in the h2_process() function itself when a connection status
change is detected.
projects / thirdparty / haproxy.git / log
