Willy Tarreau [Fri, 12 Oct 2012 18:17:54 +0000 (20:17 +0200)]
MEDIUM: log: report SSL ciphers and version in logs using logformat %sslc/%sslv
These two new log-format tags report the SSL protocol version (%sslv) and the
SSL ciphers (%sslc) used for the connection with the client. For instance, to
append these information just after the client's IP/port address information
on an HTTP log line, use the following configuration :
Willy Tarreau [Fri, 12 Oct 2012 16:01:49 +0000 (18:01 +0200)]
MEDIUM: log: add a new LW_XPRT flag to pin the transport layer
This flag will have to be set on log tags which require transport layer
information. They will prevent the conn_xprt_close() call from releasing
the transport layer too early.
Willy Tarreau [Fri, 12 Oct 2012 15:50:05 +0000 (17:50 +0200)]
MEDIUM: connection: add a flag to hold the transport layer
When we start logging SSL information, we need the SSL struct to be
present even past the conn_xprt_close() call. In order to achieve this,
we should use refcounting on the connection and the transport layer. At
the moment it's not worth using plain refcounting as only the logs require
this, so instead of real refcounting we just use a flag which will be set
by the log subsystem when SSL data need to be logged.
What happens then is that the xprt->close() call is ignored and the
transport layer is closed again during session_free(), after the log
line is emitted.
Willy Tarreau [Fri, 12 Oct 2012 15:42:13 +0000 (17:42 +0200)]
BUG/MEDIUM: session: enable the conn_session_update() callback
This callback was introduced by commit 9683e9a0 but never enabled because
the CO_FL_WAKE_DATA flag was not set. The result is that this function is
never called when an SSL handshake fails, so the connection is only closed
on timeout.
Willy Tarreau [Fri, 12 Oct 2012 15:36:40 +0000 (17:36 +0200)]
BUG/MINOR: session: fix some leftover from debug code
Commit 82569f91 moved the health and monitor-net checks to session.c
but a debug test introduced 0& to disable MSG_DONTWAIT in the recv()
call and this debug code remained there. Since the socket is marked
non-blocking, there should be no effect but it's dangerous to keep
such a thing here.
Willy Tarreau [Fri, 12 Oct 2012 15:00:05 +0000 (17:00 +0200)]
MEDIUM: connection: always unset the transport layer upon close
When calling conn_xprt_close(), we always clear the transport pointer
so that all transport layers leave the connection in the same state after
a close. This will also make it safer and cheaper to call conn_xprt_close()
multiple times if needed.
Willy Tarreau [Fri, 12 Oct 2012 12:56:11 +0000 (14:56 +0200)]
MEDIUM: log: suffix the frontend's name with '~' when using SSL
Until now it was not possible to know from the logs whether the incoming
connection was made over SSL or not. In order to address this in the existing
log formats, a new log format %ft was introduced, to log the frontend's name
suffixed with its transport layer. The only transport layer in use right now
is '~' for SSL, so that existing log formats for non-SSL traffic are not
affected at all, and SSL log formats have the frontend's name suffixed with
'~'.
The TCP, HTTP and CLF log format now use %ft instead of %f. This does not
affect existing log formats which still make use of %f however.
Emeric Brun [Thu, 11 Oct 2012 14:11:36 +0000 (16:11 +0200)]
MINOR: ssl: add statements 'verify', 'ca-file' and 'crl-file' on servers.
It now becomes possible to verify the server's certificate using the "verify"
directive. This one only supports "none" and "required", as it does not make
much sense to also support "optional" here.
Willy Tarreau [Wed, 10 Oct 2012 21:04:25 +0000 (23:04 +0200)]
MEDIUM: ssl: move "server" keyword SSL options parsing to ssl_sock.c
All SSL-specific "server" keywords are now processed in ssl_sock.c. At
the moment, there is no more "not implemented" hint when SSL is disabled,
but keywords could be added in server.c if needed.
Willy Tarreau [Wed, 10 Oct 2012 06:56:47 +0000 (08:56 +0200)]
MINOR: standard: make indent_msg() support empty messages
indent_msg() is called with dynamically generated messages, so these
may be empty (NULL) when an empty list is being dumped. Support this
and return a NULL too.
Willy Tarreau [Wed, 10 Oct 2012 06:27:36 +0000 (08:27 +0200)]
MINOR: server: add minimal infrastructure to parse keywords
Just like with the "bind" lines, we'll switch the "server" line
parsing to keyword registration. The code is essentially the same
as for bind keywords, with minor changes such as support for the
default-server keywords and support for variable argument count.
Willy Tarreau [Wed, 10 Oct 2012 14:49:28 +0000 (16:49 +0200)]
MINOR: halog: add a parameter to limit output line count
Sometimes it's useful to limit the output to a number of lines, for
example when output is already sorted (eg: 10 slowest URLs, ...). Now
we can use -m for this.
Willy Tarreau [Mon, 8 Oct 2012 18:11:03 +0000 (20:11 +0200)]
MEDIUM: listener: add support for linux's accept4() syscall
On Linux, accept4() does the same as accept() except that it allows
the caller to specify some flags to set on the resulting socket. We
use this to set the O_NONBLOCK flag and thus to save one fcntl()
call in each connection. The effect is a small performance gain of
around 1%.
The option is automatically enabled when target linux2628 is set, or
when the USE_ACCEPT4 Makefile variable is set. If the libc is too old
to provide the equivalent function, this is automatically detected and
our own function is used instead. In any case it is possible to force
the use of our implementation with USE_MY_ACCEPT4.
Willy Tarreau [Fri, 5 Oct 2012 20:41:26 +0000 (22:41 +0200)]
BUG/MAJOR: ensure that hdr_idx is always reserved when L7 fetches are used
Baptiste Assmann reported a bug causing a crash on recent versions when
sticking rules were set on layer 7 in a TCP proxy. The bug is easier to
reproduce with the "defer-accept" option on the "bind" line in order to
have some contents to parse when the connection is accepted. The issue
is that the acl_prefetch_http() function called from HTTP fetches relies
on hdr_idx to be preinitialized, which is not the case if there is no L7
ACL.
The solution consists in adding a new SMP_CAP_L7 flag to fetches to indicate
that they are expected to work on L7 data, so that the proxy knows that the
hdr_idx has to be initialized. This is already how ACL and HTTP mode are
handled.
Emeric Brun [Fri, 5 Oct 2012 13:47:31 +0000 (15:47 +0200)]
MINOR: ssl: add defines LISTEN_DEFAULT_CIPHERS and CONNECT_DEFAULT_CIPHERS.
These ones are used to set the default ciphers suite on "bind" lines and
"server" lines respectively, instead of using OpenSSL's defaults. These
are probably mainly useful for distro packagers.
Willy Tarreau [Fri, 5 Oct 2012 19:29:37 +0000 (21:29 +0200)]
BUG: connection: fix regression from commit 9e272bf9
Commit 9e272bf9 broke connection setup in TCP mode, the comment was
misleading and obviously wrong, as after a connection is established,
we *do* have none of the CONNECT* flags. However we can never have
them all at the same time, so let's use this to trigger a detection.
Willy Tarreau [Thu, 4 Oct 2012 22:04:16 +0000 (00:04 +0200)]
MEDIUM: checks: enable the PROXY protocol with health checks
When health checks are configured on a server which has the send-proxy
directive and no "port" nor "addr" settings, the health check connections
will automatically use the PROXY protocol. If "port" or "addr" are set,
the "check-send-proxy" directive may be used to force the protocol.
MAJOR: checks: completely use the connection transport layer
With this change, we now use the connection's transport layer to receive
and send data during health checks. It even becomes possible to send data
in multiple times, which was not possible before.
The transport layer used is the same as the one used for the traffic, unless
a specific address and/or port is specified for the checks using "port" or
"addr", in which case the transport layer defaults to raw_sock. An option
will be provided to force SSL checks on different IP/ports later.
Connection errors and timeouts are still reported.
Some situations where strerror() was able to report a precise error after
a failed connect() in the past might not be reported with as much precision
anymore, but the error message was already meaningless. During the tests,
no situation was found where a message became less precise.
MEDIUM: check: add the ctrl and transport layers in the server check structure
Since it's possible for the checks to use a different protocol or transport layer
than the prod traffic, we need to have them referenced in the server. The
SSL checks are not enabled yet, but the transport layers are completely used.
MEDIUM: checks: use real buffers to store requests and responses
Till now the request was made in the trash and sent to the network at
once, and the response was read into a preallocated char[]. Now we
allocate a full buffer for both the request and the response, and make
use of it.
Some of the operations will probably be replaced later with buffer macros
but the point was to ensure we could migrate to use the data layers soon.
One nice improvement caused by this change is that requests are now formed
at the beginning of the check and may safely be sent in multiple chunks if
needed.
REORG: server: move the check-specific parts into a check subsection
The health checks in the servers are becoming a real mess, move them
into their own subsection. We'll soon need to have a struct buffer to
replace the char * as well as check-specific protocol and transport
layers.
MAJOR: checks: make use of the connection layer to send checks
This is a first step, we now use the connection layer without the data
layers (send/recv are still used by hand). The connection is established
using tcp_connect_server() and raw_sock is assumed and forced for now.
fdtab is not manipulated anymore and polling is managed via the connection
layer.
It becomes quite clear that the server needs a second ->ctrl and ->xprt
dedicated to the checks.
Willy Tarreau [Thu, 4 Oct 2012 21:55:57 +0000 (23:55 +0200)]
MEDIUM: connection: add a new local send-proxy transport callback
This callback sends a PROXY protocol line on the outgoing connection,
with the local and remote endpoint information. This is used for local
connections (eg: health checks) where the other end needs to have a
valid address and no connection is relayed.
Willy Tarreau [Thu, 4 Oct 2012 22:10:55 +0000 (00:10 +0200)]
REORG: connection: move the PROXY protocol management to connection.c
It was previously in frontend.c but there is no reason for this anymore
considering that all the information involved is in the connection itself
only. Theorically this should be in the socket layer but we don't have
this yet.
Willy Tarreau [Thu, 4 Oct 2012 20:21:15 +0000 (22:21 +0200)]
MEDIUM: connection: automatically disable polling on error
We absolutely want to disable FD polling after an error is detected,
otherwise the data layer has to do it and it's far from being obvious
at these layers.
The way we did it was a bit tricky in conn_update_*_polling and
conn_*_polling_changes. However it has almost no impact on performance
and code size both for the fast and slow path.
We'll now be able to remove some flag updates in the stream interface.
Willy Tarreau [Thu, 4 Oct 2012 18:20:46 +0000 (20:20 +0200)]
MEDIUM: connection: it's not the data layer's role to validate the connection
Till now we used to perform the L4_CONN check in the data layer
(eg: stream interface) but that does not make sense, because some transport
layers will imply that the connection is opened (eg: SSL), and also because
the complexity to check for this is higher in the data layer than in the
transport layer. This is so much true that some read0 cases did not validate
the connection.
So as of now, the transport layer is responsible for clearing L4_CONN when
it detects an activity, and the data layer may safely rely on this flag. This
only impacts a minor change in raw_sock and stream_interface for now.
Willy Tarreau [Wed, 3 Oct 2012 19:17:23 +0000 (21:17 +0200)]
MEDIUM: session: register a data->wake callback to process errors
The connection layer will soon call ->wake() only when errors happen, and
not ->init(). So make the session layer use this callback to detect errors
and abort connections.
Willy Tarreau [Wed, 3 Oct 2012 19:12:16 +0000 (21:12 +0200)]
MEDIUM: connection: make it possible for data->wake to return an error
Just like ->init(), ->wake() may now be used to return an error and
abort the connection. Currently this is not used but will be with
embryonic sessions.
Willy Tarreau [Wed, 3 Oct 2012 19:04:48 +0000 (21:04 +0200)]
MEDIUM: connection: only call the data->wake callback on activity
We now check the connection flags for changes in order not to call the
data->wake callback when there is no activity. Activity means a change
on any of the CO_FL_*_SH, CO_FL_ERROR, CO_FL_CONNECTED, CO_FL_WAIT_CONN*
flags, as well as a call to data->recv or data->send.
Willy Tarreau [Wed, 3 Oct 2012 18:00:18 +0000 (20:00 +0200)]
MEDIUM: connection: reorganize connection flags
The connection flags have progressively been added one after the other
and were not very well organized. Some of them are often used together
and a number of operations are performed on the DATA/SOCK ENA/POL flags.
Thus, they have been reorganized so that flags that work together are
close to each other (allows immediate operands on ARM) and that polling
changes can be detected with fewer operations using a simple shift and
xor. The handshakes are now the last ones so that it will be easier to
add new ones after without risking a collision. All activity-related
flags are also grouped together.
Willy Tarreau [Tue, 2 Oct 2012 23:39:48 +0000 (01:39 +0200)]
MEDIUM: connection: use a generic data-layer init() callback
The generic data-layer init callback is now used after the transport
layer is complete and before calling the data layer recv/send callbacks.
This allows the session to switch from the embryonic session data layer
to the complete stream interface data layer, by making conn_session_complete()
the data layer's init callback.
It sill looks awkwards that the init() callback must be used opon error,
but except by adding yet another one, it does not seem to be mergeable
into another function (eg: it should probably not be merged with ->wake
to avoid unneeded calls during the handshake, though semantically that
would make sense).
Willy Tarreau [Tue, 2 Oct 2012 19:21:20 +0000 (21:21 +0200)]
MEDIUM: session: use a specific data_cb for embryonic sessions
We don't want to have the recv or send callbacks in embryonic
sessions, and we want the stream interface to be referenced as
the connection owner only once the session is instanciated. So
let's first have the embryonic session be the owner, then replaced
later by the stream interface once the transport layer is ready.
Willy Tarreau [Tue, 2 Oct 2012 18:07:22 +0000 (20:07 +0200)]
MINOR: connection: provide a generic data layer wakeup callback
Instead of calling conn_notify_si() from the connection handler, we
now call data->wake(), which will allow us to use a different callback
with health checks.
Note that we still rely on a flag in order to decide whether or not
to call this function. The reason is that with embryonic sessions,
the callback is already initialized to si_conn_cb without the flag,
and we can't call the SI notify function in the leave path before
the stream interface is initialized.
This issue should be addressed by involving a different data_cb for
embryonic sessions and for stream interfaces, that would be changed
during session_complete() for the final data_cb.
Willy Tarreau [Tue, 2 Oct 2012 22:41:04 +0000 (00:41 +0200)]
REORG: connection: rename app_cb "data"
Now conn->data will designate the data layer which is the client for
the transport layer. In practice it's the stream interface and will
soon also be the health checks.
Willy Tarreau [Tue, 2 Oct 2012 22:19:48 +0000 (00:19 +0200)]
REORG: connection: rename the data layer the "transport layer"
While working on the changes required to make the health checks use the
new connections, it started to become obvious that some naming was not
logical at all in the connections. Specifically, it is not logical to
call the "data layer" the layer which is in charge for all the handshake
and which does not yet provide a data layer once established until a
session has allocated all the required buffers.
In fact, it's more a transport layer, which makes much more sense. The
transport layer offers a medium on which data can transit, and it offers
the functions to move these data when the upper layer requests this. And
it is the upper layer which iterates over the transport layer's functions
to move data which should be called the data layer.
The use case where it's obvious is with embryonic sessions : an incoming
SSL connection is accepted. Only the connection is allocated, not the
buffers nor stream interface, etc... The connection handles the SSL
handshake by itself. Once this handshake is complete, we can't use the
data functions because the buffers and stream interface are not there
yet. Hence we have to first call a specific function to complete the
session initialization, after which we'll be able to use the data
functions. This clearly proves that SSL here is only a transport layer
and that the stream interface constitutes the data layer.
A similar change will be performed to rename app_cb => data, but the
two could not be in the same commit for obvious reasons.
When a connection setup is pending and we receive an error without a
POLL_IN flag, we're certain there will be nothing to read from it and
we can safely report an error without attempting a recv() call. This
will be significantly better for health checks which will avoid a useless
recv() on all failed checks.
Depending on the pollers used, a connection error may be notified
with POLLOUT|POLLERR|POLLHUP. POLLHUP by itself is enough for the
connection handler to call the read actor, which would only consider
this flag as a good indication of a hangup, without considering the
POLLERR flag.
In order to address this, we directly jump to the read0 label if
POLLERR was not set.
This will be important with health checks as we don't want to believe
a connection was properly established when it's not the case !
Willy Tarreau [Thu, 4 Oct 2012 17:19:36 +0000 (19:19 +0200)]
MINOR: signal: really ignore signals configured with no handler
Until now, signals configured with no handler were still enabled and
ignored upon signal reception. Until now it was not an issue but with
SSL causing many EPIPE all the time, it becomes obvious that signal
processing comes with a cost. So set the handler to SIG_IGN when the
function is NULL.
Willy Tarreau [Thu, 4 Oct 2012 19:54:41 +0000 (21:54 +0200)]
BUG/MINOR: epoll: correctly disable FD polling in fd_rem()
When calling fd_rem(), the polling was not correctly disabled because the
->prev state was set to zero instead of the previous value. fd_rem() is
very rarely used, only just before closing a socket.
The effect is that upon an error reported at the connection level, if the
task assigned to the connection was too slow to be woken up because of too
many other tasks in the run queue, the FD was still not disabled and caused
the connection handler to be called again with the same event until the task
was finally executed to close the fd.
This issue only affects the epoll poller, not the sepoll variant nor any of
the other ones.
It was already present in 1.4 and even 1.3 with the same almost unnoticeable
effects. The bug can in fact only be discovered during development where it
emphasizes other bugs.
Willy Tarreau [Thu, 4 Oct 2012 06:47:34 +0000 (08:47 +0200)]
MEDIUM: proxy: add the global frontend to the list of normal proxies
Since recent changes on the global frontend, it was not possible anymore
to soft-reload a process which had a stats socket because the socket would
not be disabled upon reload. The only solution to this endless madness is
to have the global frontend part of normal proxies.
Since we don't want to get an ID that shifts all other proxies and causes
trouble in deployed environments, we assign it ID #0 which other proxies
can't grab, and we don't report it in the stats pages.
Willy Tarreau [Thu, 4 Oct 2012 06:56:31 +0000 (08:56 +0200)]
BUG/MEDIUM: listener: don't pause protocols that do not support it
Pausing a UNIX_STREAM socket results in a major pain because the socket
does not correctly resume, it wakes poll() but return EAGAIN on accept(),
resulting in a busy loop. So let's only pause protocols that support it.
This issues has existed since UNIX sockets were introduced on bind lines.
Willy Tarreau [Thu, 4 Oct 2012 06:01:43 +0000 (08:01 +0200)]
BUG/MINOR: config: use a copy of the file name in proxy configurations
Each proxy contains a reference to the original config file and line
number where it was declared. The pointer used is just a reference to
the one passed to the function instead of being duplicated. The effect
is that it is not valid anymore at the end of the parsing and that all
proxies will be enumerated as coming from the same file on some late
configuration errors. This may happen for exmaple when reporting SSL
certificate issues.
By copying using strdup(), we avoid this issue.
1.4 has the same issue, though no report of the proxy file name is done
out of the config section. Anyway a backport is recommended to ease
post-mortem analysis.
Willy Tarreau [Wed, 3 Oct 2012 22:14:33 +0000 (00:14 +0200)]
BUG/MEDIUM: proxy: must not try to stop disabled proxies upon reload
Hervé Commowick reported an issue : haproxy dies in a segfault during a
soft restart if it tries to pause a disabled proxy. This is because disabled
proxies have no management task so we must not wake the task up. This could
easily remain unnoticed since the old process was expected to go away, so
having it go away faster was not really troubling. However, with sync peers,
it is obvious that there is no peer sync during this reload.
This issue has been introduced in 1.5-dev7 with the removal of the
maintain_proxies() function. No backport is needed.
BUG/MINOR: ssl: report the L4 connection as established when possible
If we get an SSL error during the handshake, we at least try to see
if a syscall reported an error or not. In case of an error, it generally
means that the connection failed. If there is no error, then the connection
established successfully.
The difference is important for health checks which report the precise cause
to the logs and to the stats.
Disables the stateless session resumption (RFC 5077 TLS Ticket
extension) and force to use stateful session resumption. Stateless
session resumption is more expensive in CPU usage.
Emeric Brun [Tue, 2 Oct 2012 11:45:20 +0000 (13:45 +0200)]
MINOR: ssl: add statement 'no-tls-tickets' on bind to disable stateless session resumption
Disables the stateless session resumption (RFC 5077 TLS Ticket extension)
and force to use stateful session resumption.
Stateless session resumption is more expensive in CPU usage.
Emeric Brun [Mon, 1 Oct 2012 15:10:30 +0000 (17:10 +0200)]
MEDIUM: config: authorize frontend and listen without bind.
This allows to easily add/remove "bind" entries to a frontend without
being forced to remove it when the last entry is temporarily removed.
While "disabled" may sometimes work in a frontend, it becomes trickier
on "listen" sections which can also hold servers and be referenced by
other frontends.
Note that a "listen" section with no "bind" is equivalent to a "backend"
section.
Configs without any listeners are still reported as invalid and refuse
to load.
MINOR: ssl : add statements 'notlsv11' and 'notlsv12' and rename 'notlsv1' to 'notlsv10'.
This is because "notlsv1" used to disable TLSv1.0 only and had no effect
on v1.1/v1.2. so better have an option for each version. This applies both
to "bind" and "server" statements.
MINOR: ssl: disable shared memory and locks on session cache if nbproc == 1
We don't needa to lock the memory when there is a single process. This can
make a difference on small systems where locking is much more expensive than
just a test.
Allow to ignore some verify errors and to let them pass the handshake.
Add option 'crt-ignore-err <list>'
Ignore verify errors at depth == 0 (client certificate)
<list> is string 'all' or a comma separated list of verify error IDs
(see http://www.openssl.org/docs/apps/verify.html)
Add option 'ca-ignore-err <list>'
Same as 'crt-ignore-err' for all depths > 0 (CA chain certs)
Ex ignore all errors on CA and expired or not-yet-valid errors
on client certificate:
MEDIUM: ssl: add client certificate authentication support
Add keyword 'verify' on bind:
'verify none': authentication disabled (default)
'verify optional': accept connection without certificate
and process a verify if the client sent a certificate
'verify required': reject connection without certificate
and process a verify if the client send a certificate
Add keyword 'cafile' on bind:
'cafile <path>' path to a client CA file used to verify.
'crlfile <path>' path to a client CRL file used to verify.