This one dumps the fdtab for all active FDs with some quickly interpretable
characters to read the flags (like upper case=set, lower case=unset). It
can probably be improved to report fdupdt[] and/or fdinfo[] but at least it
provides a good start and allows to see how FDs are seen. When the fd owner
is a connection, its flags are also reported as it can help compare with the
polling status, and the target (fe/px/sv) as well. When it's a listener, the
listener's state is reported as well as the frontend it belongs to.
BUG/MEDIUM: stream: don't retry SSL connections which fail the SNI name check
Commits 2ab8867 ("MINOR: ssl: compare server certificate names to the
SNI on outgoing connections") and 96c7b8d ("BUG/MINOR: ssl: Fix check
against SNI during server certificate verification") made it possible
to check that the server's certificate matches the name presented in
the SNI field. While it solves a class of problems, it opens another
one which is that by failing such a connection, we'll retry it and put
more load on the server. It can be a real problem if a user can trigger
this issue, which is what will very often happen when the SNI is forwarded
from the client to the server.
This patch solves this by detecting that this very specific hostname
verification failed and that the hostname was provided using SNI, and
then it simply disables retries and the failure is immediate.
At the time of writing this patch, the previous patches were not backported
(yet), so no backport is needed for this one unless the aforementionned
patches are backported as well. This patch requires previous patches
"BUG/MINOR: ssl: make use of the name in SNI before verifyhost" and
"MINOR: ssl: add a new error code for wrong server certificates".
MINOR: ssl: add a new error codes for wrong server certificates
If a server presents an unexpected certificate to haproxy, that is, a
certificate that doesn't match the expected name as configured in
verifyhost or as requested using SNI, we want to store that precious
information. Fortunately we have access to the connection in the
verification callback so it's possible to store an error code there.
For this purpose we use CO_ER_SSL_MISMATCH_SNI (for when the cert name
didn't match the one requested using SNI) and CO_ER_SSL_MISMATCH for
when it doesn't match verifyhost.
BUG/MINOR: ssl: make use of the name in SNI before verifyhost
Commit 2ab8867 ("MINOR: ssl: compare server certificate names to the SNI
on outgoing connections") introduced the ability to check server cert
names against the name provided with in the SNI, but verifyhost was kept
as a way to force the name to check against. This was a mistake, because :
- if an SNI is used, any static hostname in verifyhost will be wrong ;
worse, if it matches and doesn't match the SNI, the server presented
the wrong certificate ;
- there's no way to have a default name to check against for health
checks anymore because the point above mandates the removal of the
verifyhost directive
This patch reverses the ordering of the check : whenever SNI is used, the
name provided always has precedence (ie the server must always present a
certificate that matches the requested name). And if no SNI is provided,
then verifyhost is used, and will be configured to match the server's
default certificate name. This will work both when SNI is not used and
for health checks.
If the commit 2ab8867 is backported in 1.7 and/or 1.6, this one must be
backported too.
BUG/MINOR: ssl: Fix check against SNI during server certificate verification
This patch fixes the commit 2ab8867 ("MINOR: ssl: compare server certificate
names to the SNI on outgoing connections")
When we check the certificate sent by a server, in the verify callback, we get
the SNI from the session (SSL_SESSION object). In OpenSSL, tlsext_hostname value
for this session is copied from the ssl connection (SSL object). But the copy is
done only if the "server_name" extension is found in the server hello
message. This means the server has found a certificate matching the client's
SNI.
When the server returns a default certificate not matching the client's SNI, it
doesn't set any "server_name" extension in the server hello message. So no SNI
is set on the SSL session and SSL_SESSION_get0_hostname always returns NULL.
To fix the problemn, we get the SNI directly from the SSL connection. It is
always defined with the value set by the client.
If the commit 2ab8867 is backported in 1.7 and/or 1.6, this one must be
backported too.
Note: it's worth mentionning that by making the SNI check work, we
introduce another problem by which failed SNI checks can cause
long connection retries on the server, and in certain cases the
SNI value used comes from the client. So this patch series must
not be backported until this issue is resolved.
While playing with Lua API I've noticed that core.proxies attribute
doesn't return all the proxies, more precisely the ones with same names
(e.g. for frontend and backend with the same name it would only return
the latter one).
So, this patch fixes this problem without breaking the actual behaviour.
We have two case of proxies with frontend/backend capabilities:
The first case is the listen. This case is not a problem because the
proxy object process these two entities as only one and it is the
expected behavior. With these case the "proxies" list works fine.
The second case is the frontend and backend with the same name. i think
that this case is possible for compatibility with 'listen' declaration.
These two proxes with same name and different capabilities must not
processed with the same object (different statitics, differents orders).
In fact, one the the two object crush the other one whoch is no longer
accessible.
To fix this problem, this patch adds two lists which are "frontends" and
"backends", each of these list contains specialized proxy, but warning
the "listen" proxy are declare in each list.
This is just for convenience and uniformity, Proxy.servers/listeners
returns a table/hash of objects with names as keys, but for example when
I want to pass such object to some other Lua function I have to manually
copy the name (or wrap the object), since the object itself doesn't
expose name info.
This patch simply adds the proxy name as member of the proxy object.
BUG/MAJOR: lua: properly dequeue hlua_applet_wakeup() for new scheduler
The recent scheduler change broke the Lua co-sockets due to
hlua_applet_wakeup() returning NULL after waking the applet up. With the
previous scheduler, returning NULL was a way to do nothing on return.
With the new one it keeps TASK_RUNNING set, causing all new notifications
to end up into t->pending_state instead of t->state, and prevents the
task from being added into the run queue again, so and it's never woken
up anymore.
The applet keeps waking up, causing hlua_socket_handler() to do nothing
new, then si_applet_wake_cb() calling stream_int_notify() to try to wake
the task up, which it can't do due to the TASK_RUNNING flag, then decide
that since the associated task is not in the run queue, it needs to call
stream_int_update_applet() to propagate the update. This last one finds
that the applet needs to be woken up to deal with the last reported events
and calling appctx_wakeup() again. Previously, this situation didn't exist
because the task was always added in the run queue despite the TASK_RUNNING
flag.
By returning the task instead in hlua_applet_wakeup(), we can ensure its
flag is properly cleared and the task is requeued if needed or just sits
waiting for new events to happen.
This fix requires the previous ones ("BUG/MINOR: lua: always detach the
tcp/http tasks before freeing them") and MINOR: task: always preinitialize
the task's timeout in task_init().
Thanks to Thierry, Christopher and Emeric for the long head-scratching
session!
No backport is needed as the bug doesn't appear in older versions and
it's unsure whether we'll not break something by backporting it.
MINOR: task: always preinitialize the task's timeout in task_init()
task_init() is called exclusively by task_new() which is the only way
to create a task. Most callers set t->expire to TICK_ETERNITY, some set
it to another value and a few like Lua don't set it at all as they don't
need a timeout, causing random values to be used in case the task gets
queued.
Let's always set t->expire to TICK_ETERNITY in task_init() so that all
tasks are now initialized in a clean state.
This patch can be backported as it will definitely make the code more
robust (at least the Lua code, possibly other places).
BUG/MINOR: lua: always detach the tcp/http tasks before freeing them
In hlua_{http,tcp}_applet_release(), a call to task_free() is performed
to release the task, but no task_delete() is made on these tasks. Till
now it wasn't much of a problem because this was normally not done with
the task in the run queue, and the task was never put into the wait queue
since it doesn't have any timer. But with threading it will become an
issue. And not having this already prevents another bug from being fixed.
Thanks to Christopher for spotting this one. A backport to 1.7 and 1.6 is
preferred for safety.
MINOR: samples: Don't allocate memory for SMP_T_METH sample when method is known
For known methods (GET,POST...), in samples, an enum is used instead of a chunk
to reference the method. So there is no needs to allocate memory when a variable
is stored with this kind of sample.
MINOR: samples: Handle the type SMP_T_METH when we duplicate a sample in smp_dup
First, the type SMP_T_METH was not handled by smp_dup function. It was never
called with this kind of samples, so it's not really a problem. But, this could
be useful in future.
For all known HTTP methods (GET, POST...), there is no extra space allocated for
a sample of type SMP_T_METH. But for unkown methods, it uses a chunk. So, like
for strings, we duplicate data, using a trash chunk.
BUG/MINOR: lua: Correctly use INET6_ADDRSTRLEN in Server.get_addr()
The get_addr() method of the Lua Server class incorrectly used
INET_ADDRSTRLEN for IPv6 addresses resulting in failing to convert
longer IPv6 addresses to strings.
The get_addr() method of the Lua Server class was using the
'sockaddr_storage addr' member to get the port value. HAProxy does not
store ports in this member as it uses a separate member, called
'svc_port'.
BUG/MAJOR: http: Fix possible infinity loop in http_sync_(req|res)_state
In commit "MINOR: http: Switch requests/responses in TUNNEL mode only by
checking txn flags", it is possible to have an infinite loop on HTTP_MSG_CLOSING
state.
DOC: Updated 51Degrees git URL to point to a stable version.
The previously documented location doesn't work anymore and must not be
used. Warning for backports, different branches are in use depending on
the version (v3.2.10 for 1.7, v3.2.5 for 1.6).
BUILD: lua: replace timegm() with my_timegm() to fix build on Solaris 10
Akhnin Nikita reported that Lua doesn't build on Solaris 10 because
the code uses timegm() to parse a date, which is not provided there.
The recommended way to implement timegm() is broken in the man page,
as it is based on a change of the TZ environment variable at run time
before calling the function (which is obviously not thread safe, and
terribly inefficient).
Here instead we rely on the new my_timegm() function, it should be
sufficient for all known use cases.
timegm() is not provided everywhere and the documentation on how to
replace it is bogus as it proposes an inefficient and non-thread safe
alternative.
Here we reimplement everything needed to compute the number of seconds
since Epoch based on the broken down fields in struct tm. It is only
guaranteed to return correct values for correct inputs. It was successfully
tested with all possible 32-bit values of time_t converted to struct tm
using gmtime() and back to time_t using the legacy timegm() and this
function, and both functions always produced the same result.
Thanks to Benoît Garnier for an instructive discussion and detailed
explanations of the various time functions, leading to this solution.
BUG/MINOR: ssl: remove haproxy SSLv3 support when ssl lib have no SSLv3
The commit 5db33cbd "MEDIUM: ssl: ssl_methods implementation is reworked and
factored for min/max tlsxx" drop the case when ssl lib have removed SSLv3.
The commit 1e59fcc5 "BUG/MINOR: ssl: Be sure that SSLv3 connection methods
exist for openssl < 1.1.0" fix build but it's false because haproxy think
that ssl lib support SSLv3.
SSL_OP_NO_* are flags to set in ssl_options and is the way haproxy do the
link between ssl capabilities and haproxy configuration. (The mapping table
is done via methodVersions). SSL_OP_NO_* is set to 0 when ssl lib doesn't
support a new TLS version. Older version (like SSLv3) can be removed at
build or unsupported (like libressl). In all case OPENSSL_NO_SSL3 is define.
To keep the same logic, this patch alter SSL_OP_NO_SSLv3 to 0 when SSLv3 is
not supported by ssl lib (when OPENSSL_NO_SSL3 is define).
MINOR: http: Rely on analyzers mask to end processing in forward_body functions
Instead of relying on request or response state, we use "chn->analysers" mask as
all other analyzers. So now, http_resync_states does not return anything
anymore.
The debug message in http_resync_states has been improved.
BUG/MEDIUM: http: Switch HTTP responses in TUNNEL mode when body length is undefined
When the body length of a HTTP response is undefined, the HTTP parser is blocked
in the body parsing. Before HAProxy 1.7, in this case, because
AN_RES_HTTP_XFER_BODY is never set, there is no visible effect. When the server
closes its connection to terminate the response, HAProxy catches it as a normal
closure. Since 1.7, we always set this analyzer to enter at least once in
http_response_forward_body. But, in the present case, when the server connection
is closed, http_response_forward_body is called one time too many. The response
is correctly sent to the client, but an error is catched and logged with "SD--"
flags.
To reproduce the bug, you can use the configuration "tests/test-fsm.cfg". The
tests 3 and 21 hit the bug.
Idea to fix the bug is to switch the response in TUNNEL mode without switching
the request. This is possible because of previous patches.
First, we need to detect responses with undefined body length during states
synchronization. Excluding tunnelled transactions, when the response length is
undefined, TX_CON_WANT_CLO is always set on the transaction. So, when states are
synchronized, if TX_CON_WANT_CLO is set, the response is switched in TUNNEL mode
and the request remains unchanged.
Then, in http_msg_forward_body, we add a specific check to switch the response
in DONE mode if the body length is undefined and if there is no data filter.
This patch depends on following previous commits:
* MINOR: http: Switch requests/responses in TUNNEL mode only by checking txn flags
* MINOR: http: Reorder/rewrite checks in http_resync_states
This patch must be backported in 1.7 with 2 previous ones.
MINOR: http: Switch requests/responses in TUNNEL mode only by checking txn flags
Today, the only way to have a request or a response in HTTP_MSG_TUNNEL state is
to have the flag TX_CON_WANT_TUN set on the transaction. So this is a symmetric
state. Both the request and the response are switch in same time in this
state. This can be done only by checking transaction flags instead of relying on
the other side state. This is the purpose of this patch.
This way, if for any reason we need to switch only one side in TUNNEL mode, it
will be possible. And to prepare asymmetric cases, we check channel flags in
DONE _AND_ TUNNEL states.
WARNING: This patch will be used to fix a bug. The fix will be commited in a
very next commit. So if the fix is backported, this one must be backported too.
MINOR: http: Reorder/rewrite checks in http_resync_states
The previous patch removed the forced symmetry of the TUNNEL mode during the
state synchronization. Here, we take care to remove body analyzer only on the
channel in TUNNEL mode. In fact, today, this change has no effect because both
sides are switched in same time. But this way, with some changes, it will be
possible to keep body analyzer on a side (to finish the states synchronization)
with the other one in TUNNEL mode.
WARNING: This patch will be used to fix a bug. The fix will be commited in a
very next commit. So if the fix is backported, this one must be backported too.
DOC: update CONTRIBUTING regarding optional parts and message format
Make it clear that optional components must not break when disabled,
that openssl is the only officially supported library and its support
must not be broken, and that bug fixes must always be detailed.
BUG/MAJOR: lua/socket: resources not detroyed when the socket is aborted
In some cases, the socket is misused. The user can open socket and never
close it, or open the socket and close it without sending data. This
causes resources leak on all resources associated to the stream (buffer,
spoe, ...)
This is caused by the stream_shutdown function which is called outside
of the stream execution process. Sometimes, the shtudown is required
while the stream is not started, so the cleanup is ignored.
This patch change the shutdown mode of the session. Now if the session is
no longer used and the Lua want to destroy it, it just set a destroy flag
and the session kill itself.
BUG/MINOR: lua: In error case, the safe mode is not removed
Just forgot of reset the safe mode. This have not consequences
the safe mode just set a pointer on fucntion which is called only
and initialises a longjmp.
Out of lua execution, this longjmp is never executed and the
function is never called.
CLEANUP: hdr_idx: make some function arguments const where possible
Functions hdr_idx_first_idx() and hdr_idx_first_pos() were missing a
"const" qualifier on their arguments which are not modified, causing
a warning in some experimental H2 code.
BUG/MINOR: peers: peer synchronization issue (with several peers sections).
When several stick-tables were configured with several peers sections,
only a part of them could be synchronized: the ones attached to the last
parsed 'peers' section. This was due to the fact that, at least, the peer I/O handler
refered to the wrong peer section list, in fact always the same: the last one parsed.
The fact that the global peer section list was named "struct peers *peers"
lead to this issue. This variable name is dangerous ;).
So this patch renames global 'peers' variable to 'cfg_peers' to ensure that
no such wrong references are still in use, then all the functions wich used
old 'peers' variable have been modified to refer to the correct peer list.
OPTIM: ssl: don't consider a small ssl_read() as an indication of end of buffer
In ssl_sock_to_buf(), when we face a small read, we used to consider it
as an indication for the end of incoming data, as is the case with plain
text. The problem is that here it's quite different, SSL records are
returned at once so doing so make us wake all the upper layers for each
and every record. Given that SSL records are 16kB by default, this is
rarely observed unless the protocol employs small records or the buffers
are increased. But with 64kB buffers while trying to deal with HTTP/2
frames, the exchanges are obviously suboptimal as there are two messages
per frame (one for the frame header and another one for the frame payload),
causing the H2 parser to be woken up half of the times without being able
to proceed :
BUG/MINOR: http: properly handle all 1xx informational responses
Only 100 was considered informational instead of all 1xx. This can be
a problem when facing a 102 ("progress") or with the upcoming 103 for
early hints. Let's properly handle all 1xx now, leaving a special case
for 101 which is used for the upgrade.
This fix should be backported to 1.7, 1.6 and 1.5. In 1.4 the code is
different but the backport should be made there as well.
MINOR: peers: Add additional information to stick-table definition messages.
With this patch additional information are added to stick-table definition
messages so that to make external application capable of learning peer
stick-table configurations. First stick-table entries duration is added
followed by the frequency counters type IDs and values.
BUG/MEDIUM: filters: Be sure to call flt_end_analyze for both channels
In the commit 2b553de5 ("BUG/MINOR: filters: Don't force the stream's wakeup
when we wait in flt_end_analyze"), we removed a task_wakeup in flt_end_analyze
to no consume too much CPU by looping in certain circumstances.
But this fix was too drastic. For Keep-Alive transactions, flt_end_analyze is
often called only for the response. Then the stream is paused until a timeout is
hitted or the next request is received. We need first let a chance to both
channels to call flt_end_analyze function. Then if a filter need to wait here,
it is its responsibility to wake up the stream when needed. To fix the bug, and
thanks to previous commits, we set the flag CF_WAKE_ONCE on channels to pretend
there is an activity. On the current channel, the flag will be removed without
any effect, but for the other side the analyzer will be called immediatly.
Thanks for Lukas Tribus for his detailed analysis of the bug.
This patch must be backported in 1.7 with the 2 previous ones:
* a94fda3 ("BUG/MINOR: http: Don't reset the transaction if there are still data to send")
* cdaea89 ("BUG/MINOR: stream: Don't forget to remove CF_WAKE_ONCE flag on response channel")
BUG/MINOR: http: Don't reset the transaction if there are still data to send
To reset an HTTP transaction, we need to be sure all data were sent, for the
request and the response. There are tests on request and response buffers for
that in http_resync_states function. But the return code was wrong. We must
return 0 to wait.
BUG/MINOR: stream: Don't forget to remove CF_WAKE_ONCE flag on response channel
This flag can be set on a channel to pretend there is activity on it. This is a
way to wake-up the corresponding stream and evaluate stream analyzers on the
channel. It is correctly handled on both channels but removed only on the
request channel.
This patch is flagged as a bug but for now, CF_WAKE_ONCE is never set on the
response channel.
MINOR: ssl: compare server certificate names to the SNI on outgoing connections
When support for passing SNI to the server was added in 1.6-dev3, there
was no way to validate that the certificate presented by the server would
really match the name requested in the SNI, which is quite a problem as
it allows other (valid) certificates to be presented instead (when hitting
the wrong server or due to a man in the middle).
This patch adds the missing check against the value passed in the SNI.
The "verifyhost" value keeps precedence if set. If no SNI is used and
no verifyhost directive is specified, then the certificate name is not
checked (this is unchanged).
In order to extract the SNI value, it was necessary to make use of
SSL_SESSION_get0_hostname(), which appeared in openssl 1.1.0. This is
a trivial function which returns the value of s->tlsext_hostname, so
it was provided in the compat layer for older versions. After some
refinements from Emmanuel, it now builds with openssl 1.0.2, openssl
1.1.0 and boringssl. A test file was provided to ease testing all cases.
After some careful observation period it may make sense to backport
this to 1.7 and 1.6 as some users rightfully consider this limitation
as a bug.
Emeric Brun [Fri, 30 Jun 2017 12:11:56 +0000 (14:11 +0200)]
BUG/MAJOR: applet: fix a freeze if data is immedately forwarded.
Introduced regression with 'MAJOR: applet scheduler rework' (1.8-dev only).
The fix consist to re-enable the appctx immediatly from the
applet wake cb if the process_stream is not pending in runqueue
and the applet want perform a put or a get and the WAIT_ROOM
flag was removed by stream_int_notify.
MINOR: compression: Use a memory pool to allocate compression states
Instead of doing a malloc/free to each HTTP transaction to allocate the
compression state (when the HTTP compression is enabled), we use a memory pool.
BUG/MAJOR: compression: Be sure to release the compression state in all cases
This patch fixes an obvious memory leak in the compression filter. The
compression state (comp_state) is allocated when a HTTP transaction starts, in
channel_start_analyze callback, Whether we are able to compression the response
or not. So it must be released when the transaction ends, in channel_end_analyze
callback.
But there is a bug here. The state is released on the response side only. So, if
a transaction ends before the response is started, it is never released. This
happens when a connection is closed before the response is started.
To fix the bug, statistics about the HTTP compression are now updated in
http_end callback, when the response parsing ends. It happens only if no error
is encountered and when the response is compressed. So, it is safe to release
the compression state in channel_end_analyze callback, regardless the
channel's type.
Emeric Brun [Thu, 29 Jun 2017 13:40:33 +0000 (15:40 +0200)]
BUG/MAJOR: map: fix segfault during 'show map/acl' on cli.
The reference of the current map/acl element to dump could
be destroyed if map is updated from an 'http-request del-map'
configuration rule or throught a 'del map/acl' on CLI.
We use a 'back_refs' chaining element to fix this. As it
is done to dump sessions.
This patch needs also fix:
'BUG/MAJOR: cli: fix custom io_release was crushed by NULL.'
To clean the back_ref and avoid a crash on a further
del/clear map operation.
Those fixes should be backported on mainline branches 1.7 and 1.6.
Willy Tarreau [Tue, 27 Jun 2017 13:47:56 +0000 (15:47 +0200)]
BUG/MAJOR: frontend: don't dereference a null conn on outgoing connections
Recently merged commit 0cfe388 ("MINOR: frontend: retrieve the ALPN name when
available") assumed that the connection is always known in frontend_accept()
which is not true for outgoing peers connections for example.
It's more efficient this way, as it allows to flush a send buffer before
receiving data in the other one. This can lead to a slightly faster buffer
recycling, thus slightly less memory and a small performance increase by
using a hotter cache.
Willy Tarreau [Tue, 13 Dec 2016 14:26:56 +0000 (15:26 +0100)]
MINOR: stream: don't set backend's nor response analysers on SF_TUNNEL
In order to implement hot-pluggable applets like we'll need for HTTP/2
which will speak a different protocol than the expected one, it will be
mandatory to be able to clear all analysers from the request and response
channel and/or to keep only the ones the applet initializer installed.
Unfortunately for now in sess_establish() we systematically place a number
of analysers inherited from the frontend, backend and some hard-coded ones.
This patch reuses the now unused SF_TUNNEL flag on the stream to indicate
we're dealing with a tunnel and don't want to add more analysers anymore.
It will be usable to install such a specific applet.
Ideally over the long term it might be nice to be able to set the mode on
the stream instead of the proxy so that we can decide to change a stream's
mode (eg: TCP, HTTP, HTTP/2) at run time. But it would require many more
changes for a gain which is not yet obvious.
Willy Tarreau [Sun, 4 Dec 2016 17:44:29 +0000 (18:44 +0100)]
MINOR: ssl: add a get_alpn() method to ssl_sock
This is used to retrieve the TLS ALPN information from a connection. We
also support a fallback to NPN if ALPN doesn't find anything or is not
available on the existing implementation. It happens that depending on
the library version, either one or the other is available. NPN was
present in openssl 1.0.1 (very common) while ALPN is in 1.0.2 and onwards
(still uncommon at the time of writing). Clients are used to send either
one or the other to ensure a smooth transition.
Willy Tarreau [Sun, 4 Dec 2016 17:42:09 +0000 (18:42 +0100)]
MINOR: connection: add a .get_alpn() method to xprt_ops
This will be used to retrieve the ALPN negociated over SSL (or possibly
via the proxy protocol later). It's likely that this information should
be stored in the connection itself, but it requires adding an extra
pointer and an extra integer. Thus better rely on the transport layer
to pass this info for now.
Willy Tarreau [Sun, 4 Dec 2016 17:39:22 +0000 (18:39 +0100)]
MINOR: frontend: initialize HTTP layer after the debugging code
For HTTP/2 we'll have to choose the upper layer based on the
advertised protocol name here and we want to keep debugging,
so let's move debugging earlier.
Willy Tarreau [Sun, 4 Dec 2016 23:26:31 +0000 (00:26 +0100)]
MEDIUM: stream: make stream_new() always set the target and analysers
It doesn't make sense that stream_new() doesn't sets the target nor
analysers and that the caller has to do it even if it doesn't know
about streams (eg: in session_accept_fd()). This causes trouble for
H2 where the applet handling the protocol cannot properly change
these information during its init phase.
Let's ensure it's always set and that the callers don't set it anymore.
Note: peers and lua don't use analysers and that's properly handled.
Emeric Brun [Mon, 29 May 2017 13:26:51 +0000 (15:26 +0200)]
MINOR: task/stream: tasks related to a stream must be init by the caller.
The task_wakeup was called on stream_new, but the task/stream
wasn't fully initialized yet. The task_wakeup must be called
explicitly by the caller once the task/stream is initialized.
Emeric Brun [Thu, 30 Mar 2017 13:37:25 +0000 (15:37 +0200)]
MAJOR: task: task scheduler rework.
In order to authorize call of task_wakeup on running task:
- from within the task handler itself.
- in futur, from another thread.
The lookups on runqueue and waitqueue are re-worked
to prepare multithread stuff.
If task_wakeup is called on a running task, the woken
message flags are savec in the 'pending_state' attribute of
the state. The real wakeup is postponed at the end of the handler
process and the woken messages are copied from pending_state
to the state attribute of the task.
It's important to note that this change will cause a very minor
(though measurable) performance loss but it is necessary to make
forward progress on a multi-threaded scheduler. Most users won't
ever notice.
Emeric Brun [Fri, 31 Mar 2017 10:04:09 +0000 (12:04 +0200)]
BUG/MINOR: stream: flag TASK_WOKEN_RES not set if task in runqueue
Under certain circumstances, if a stream's task is first woken up
(eg: I/O event) then notified of the availability of a buffer it
was waiting for via stream_res_wakeup(), this second event is lost
because the flags are only merged after seeing that the task is
running. At the moment it seems that the TASK_WOKEN_RES event is
not explicitly checked for, but better fix this before getting
reports of lost events.
This fix removes this "task running" test which is properly
performed in task_wakeup(), while the flags are properly merged.
Willy Tarreau [Fri, 23 Jun 2017 09:23:43 +0000 (11:23 +0200)]
BUG/MINOR: log: pin the front connection when front ip/ports are logged
Mathias Weiersmueller reported an interesting issue with logs which Lukas
diagnosed as dating back from commit 9b061e332 (1.5-dev9). When front
connection information (ip, port) are logged in TCP mode and the log is
emitted at the end of the connection (eg: because %B or any log tag
requiring LW_BYTES is set), the log is emitted after the connection is
closed, so the address and ports cannot be retrieved anymore.
It could be argued that we'd make a special case of these to immediatly
retrieve the source and destination addresses from the connection, but it
seems cleaner to simply pin the front connection, marking it "tracked" by
adding the LW_XPRT flag to mention that we'll need some of these elements
at the last moment. Only LW_FRTIP and LW_CLIP are affected. Note that after
this change, LW_FRTIP could simply be removed as it's not used anywhere.
Note that the problem doesn't happen when using %[src] or %[dst] since
all sample expressions set LW_XPRT.
BUG/MINOR: cfgparse: Check if tune.http.maxhdr is in the range 1..32767
We cannot store more than 32K headers in the structure hdr_idx, because
internaly we use signed short integers. To avoid any bugs (due to an integers
overflow), a check has been added on tune.http.maxhdr to be sure to not set a
value greater than 32767 and lower than 1 (because this is a nonsense to set
this parameter to a value <= 0).
BUG/MINOR: Wrong peer task expiration handling during synchronization processing.
When a peer task has sent a synchronization request to remote peers
its next expiration date was updated based on a resynchronization timeout
value which itself may have already expired leading the underlying
poller to wait for 0ms during a fraction of second (consuming high CPU
resources).
With this patch we update such peer task expiration dates only if
the resynchronization timeout is not already expired.
Thanks to Patrick Hemmer who reported an issue with nice traces
which helped in finding this one.
BUG/MEDIUM: mworker: don't reuse PIDs passed to the master
When starting the master worker with -sf or -st, the PIDs will be reused
on the next reload, which is a problem if new processes on the system
took those PIDs.
This patch ensures that we don't register old PIDs in the reload system
when launching the master worker.
Willy Tarreau [Fri, 16 Jun 2017 08:34:20 +0000 (10:34 +0200)]
BUG/MEDIUM: unix: never unlink a unix socket from the file system
James Brown reported some cases where a race condition happens between
the old and the new processes resulting in the leaving process removing
a newly bound unix socket. Jeff gave all the details he observed here :
The unix socket removal was an attempt at an optimal cleanup, which
almost never works anyway since the process is supposed to be chrooted.
And in the rare cases where it works it occasionally creates trouble.
There was already a workaround in place to avoid removing this socket
when it's been inherited from a parent's file descriptor.
So let's finally kill this useless stuff now to definitely get rid of
this persistent problem.
This fix should be backported to all stable releases.
A peer session which has just been created upon reconnect timeout expirations,
could be right after shutdown (at peer session level) because the remote
side peer could also righ after have connected. In such a case the underlying
TCP session was still running (connect()/accept()) and finally left in CLOSE_WAIT
state after the remote side stopped writting (shutdown(SHUT_WR)).
Now on, with this patch we never shutdown such peer sessions wich have just
been created. We leave them connect to the remote peer which is already
connected and must shutdown its own peer session.
Thanks to Patric Hemmer and Yves Lafon at w3.org for reporting this issue,
and for having tested this patch on the field.
Thanks also to Willy and Yelp blogs which helped me a lot in fixing it
(see https://www.haproxy.com/blog/truly-seamless-reloads-with-haproxy-no-more-hacks/ and
https://engineeringblog.yelp.com/2015/04/true-zero-downtime-haproxy-reloads.htmll).
BUG/MINOR: http/filters: Be sure to wait if a filter loops in HTTP_MSG_ENDING
A filter can choose to loop when a HTTP message is in the state
HTTP_MSG_ENDING. But the transaction is terminated with an error if the input is
closed (CF_SHUTR set on the channel). At this step, we have received all data,
so we can wait.
So now, we also check the parser state before leaving. This fix only affects
configs that use a filter that can wait in http_forward_data or http_end
callbacks, when all data were parsed.
BUG/MINOR: ssl: Be sure that SSLv3 connection methods exist for openssl < 1.1.0
For openssl 1.0.2, SSLv3_server_method and SSLv3_client_method are undefined if
OPENSSL_NO_SSL3_METHOD is set. So we must add a check on this macro before using
these functions.
BUG/MINOR: acls: Set the right refflag when patterns are loaded from a map
For an ACL, we can load patterns from a map using the flag -M. For example:
acl test hdr(host) -M -f hosts.map
The file is parsed as a map et the ACL will be executed as expected. But the
reference flag is wrong. It is set to PAT_REF_ACL. So the map will never be
listed by a "show map" on the stat socket. Setting the reference flag to
PAT_REF_ACL|PAT_REF_MAP fixes the bug.
CONTRIB: plug qdiscs: Plug queuing disciplines mini HOWTO.
Add plug_qdisc.c source file which may help in how to programatically
use plug queueing disciplines with its README file.
Such code may be useful to reproduce painful network application bugs.
BUG/MINOR: buffers: Fix bi/bo_contig_space to handle full buffers
These functions was added in commit 637f8f2c ("BUG/MEDIUM: buffers: Fix how
input/output data are injected into buffers").
This patch fixes hidden bugs. When a buffer is full (buf->i + buf->o ==
buf->size), instead of returning 0, these functions can return buf->size. Today,
this never happens because callers already check if the buffer is full before
calling bi/bo_contig_space. But to avoid possible bugs if calling conditions
changed, we slightly refactored these functions.
Willy Tarreau [Sun, 11 Jun 2017 15:56:27 +0000 (17:56 +0200)]
BUG/MAJOR: http: call manage_client_side_cookies() before erasing the buffer
Jean Lubatti reported a crash on haproxy using a config involving cookies
and tarpit rules. It just happens that since 1.7-dev3 with commit 83a2c3d
("BUG/MINOR : allow to log cookie for tarpit and denied request"), function
manage_client_side_cookies() was called after erasing the request buffer in
case of a tarpit action. The problem is that this function must absolutely
not be called with an empty buffer since it moves parts of it. A typical
reproducer consists in sending :
"GET / HTTP/1.1\r\nCookie: S=1\r\n\r\n"
On such a config :
listen crash
bind :8001
mode http
reqitarpit .
cookie S insert indirect
server s1 127.0.0.1:8000 cookie 1
The fix simply consists in moving the call to the function before the call
to buffer_erase().
Many thanks to Jean for testing instrumented code and providing a usable
core.
This fix must be backported to all stable versions since the fix introducing
this bug was backported as well.
Willy Tarreau [Fri, 9 Jun 2017 13:54:39 +0000 (15:54 +0200)]
BUILD: scripts: add an automatic mode for publish-release
Passing "-a" will make it easier to automatically create archives from
tagged repositories. It doesn't ask any question and doesn't return an
error when the current branch is not tagged nor if the release already
exists.
Willy Tarreau [Fri, 9 Jun 2017 13:36:02 +0000 (15:36 +0200)]
BUILD: scripts: make publish-release support bare repositories
First we must not report an error when "git diff HEAD" fails. Second, we
don't want to "cd" to the home dir when "git rev-parse --show-toplevel"
returns an empty string. Third, we definitely want to check that a master
branch really exists in the current directory to avoid mistakes.
projects / thirdparty / haproxy.git / log
