fix include dependency. The header file sample.h don't need to known
the content of the struct arg, so I remove the include, and replace
it by a simple pointer declaration.
This prevent an include dependecy issue with the next patch.
MINOR: tcp/http/conf: extends the keyword registration options
This patch permits to register a new keyword with the keyword "tcp-request content"
'tcp-request connection", tcp-response content", http-request" and "http-response"
which is identified only by matching the start of the keyword.
for example, we register the keyword "set-var" with the option "match_pfx"
and the configuration keyword "set-var(var_name)" matchs this entry.
This type is used to accept any type of sample as input, and prevent
any automatic "cast". It runs like the type "ADDR" which accept the
type "IPV4" and "IPV6".
Implementation of a DNS client in HAProxy to perform name resolution to
IP addresses.
It relies on the freshly created UDP client to perform the DNS
resolution. For now, all UDP socket calls are performed in the
DNS layer, but this might change later when the protocols are
extended to be more suited to datagram mode.
A new section called 'resolvers' is introduced thanks to this patch. It
is used to describe DNS servers IP address and also many parameters.
MEDIUM: server: add support for changing a server's address
Ability to change a server IP address during HAProxy run time.
For now this is provided via function update_server_addr() which
currently is not called.
A log is emitted on each change. For now we do it inconditionally,
but later we'll want to do it only on certain circumstances, which
explains why the logging block is enclosed in if(1).
Joris Dedieu [Fri, 12 Jun 2015 23:05:11 +0000 (01:05 +0200)]
BUG/MEDIUM: compat: fix segfault on FreeBSD
Since commit 65d805fd witch removes standard.h from compat.h some
values were not properly set on FreeBSD. This caused a segfault
at startup when smp_resolve_args is called.
As FreeBSD have IP_BINDANY, CONFIG_HAP_TRANSPARENT is define. This
cause struct conn_src to be extended with some fields. The size of
this structure was incorrect. Including netinet/in.h fix this issue.
While diving in code preprocessing, I found that limits.h was require
to properly set MAX_HOSTNAME_LEN, ULONG_MAX, USHRT_MAX and others
system limits on FreeBSD.
These functions could be used to create a certificate by hand, set it in the
cache used to store generated certificates and retrieve it. Here is an example
(pseudo code):
MEDIUM: ssl: Add options to forge SSL certificates
With this patch, it is possible to configure HAProxy to forge the SSL
certificate sent to a client using the SNI servername. We do it in the SNI
callback.
To enable this feature, you must pass following BIND options:
* ca-sign-file <FILE> : This is the PEM file containing the CA certitifacte and
the CA private key to create and sign server's certificates.
* (optionally) ca-sign-pass <PASS>: This is the CA private key passphrase, if
any.
* generate-certificates: Enable the dynamic generation of certificates for a
listener.
Because generating certificates is expensive, there is a LRU cache to store
them. Its size can be customized by setting the global parameter
'tune.ssl.ssl-ctx-cache-size'.
It lookup a key in a LRU cache for use with specified domain and revision. It
differs from lru64_get as it does not create missing keys. The function returns
NULL if an error or a cache miss occurs.
MINOR: lru: Add the possibility to free data when an item is removed
Now, When a item is committed in an LRU tree, you can define a function to free
data owned by this item. This function will be called when the item is removed
from the LRU tree or when the tree is destroyed..
Dragan Dosen [Thu, 11 Jun 2015 19:36:33 +0000 (21:36 +0200)]
BUG/MEDIUM: init: segfault if global._51d_property_names is not initialized
When using the "51d" converter without specifying the list of 51Degrees
properties to detect (see parameter "51degrees-property-name-list"), the
"global._51d_property_names" could be left uninitialized which will lead to
segfault during init.
Godbach [Tue, 9 Jun 2015 11:41:52 +0000 (19:41 +0800)]
CLEANUP: deinit: remove codes for cleaning p->block_rules
Since all rules listed in p->block_rules have been moved to the beginning of
the http-request rules in check_config_validity(), there is no need to clean
p->block_rules in deinit().
Willy Tarreau [Tue, 2 Jun 2015 12:10:28 +0000 (14:10 +0200)]
BUILD: make DeviceAtlas easier to build by defaulting to DEVICEATLAS_SRC
Since both DEVICEATLAS_INC and DEVICEATLAS_LIB are set to the same path
when building from sources, simply allow DEVICEATLAS_SRC to be set alone
to simplify the build procedure.
Willy Tarreau [Mon, 1 Jun 2015 09:12:35 +0000 (11:12 +0200)]
BUILD: make 51D easier to build by defaulting to 51DEGREES_SRC
Till now 3 paths were needed, 51DEGREES_SRC, 51DEGREES_INC, and
51DEGREES_LIB. Let's make the last two default to 51DEGREES_SRC since
it's the same location, and fix the doc to reflect this (all three were
documented but inconsistently).
Thomas Holmes [Wed, 13 May 2015 10:27:35 +0000 (11:27 +0100)]
MEDIUM: sample: add fiftyone_degrees converter.
It takes up to 5 string arguments that are to be 51Degrees property names.
It will then create a chunk with values detected based on the request header
supplied (this should be the User-Agent).
Thomas Holmes [Tue, 12 May 2015 14:06:21 +0000 (15:06 +0100)]
BUILD: add 51degrees options to makefile.
To build with 51Degrees set USE_51DEGREES=1. 51DEGREES_INC, 51DEGREES_LIB,
and 51DEGREES_SRC will need to be set to the 51Degrees pattern header and
C file.
Willy Tarreau [Mon, 1 Jun 2015 13:25:46 +0000 (15:25 +0200)]
BUG/MINOR: da: fix log-level comparison to emove annoying warning
When haproxy is run on the foreground with DeviceAtlas enabled, one
line of warning is seen for every test because the comparison is always
true even when loglevel is zero :
David Carlier [Mon, 1 Jun 2015 12:21:47 +0000 (14:21 +0200)]
DOC: README: explain how to build with DeviceAtlas
This diff is related to the additional documentation in
order to build the DeviceAtlas module and in addition with
an example of a basic configuration.
David Carlier [Mon, 1 Jun 2015 12:00:07 +0000 (14:00 +0200)]
BUILD: Makefile: add options to build with DeviceAtlas
This diff updates the Makefile to compile conditionally via
some new sets of flags, USE_DEVICEATLAS to enable the module
and the couple DEVICEATLAS_INC/DEVICEATLAS_LIB which needs to
point to the API root folder in order to compile the API and
the module.
David Carlier [Mon, 1 Jun 2015 11:57:22 +0000 (13:57 +0200)]
MEDIUM: init: DeviceAtlas initialization
This diff initialises few DeviceAtlas struct fields member with
their inherent default values.
Furthermore, the specific DeviceAtlas configuration keywords are
registered and the module is initialised and all necessary
resources are freed during the deinit phase.
David Carlier [Mon, 1 Jun 2015 11:53:01 +0000 (13:53 +0200)]
MEDIUM: global: add the DeviceAtlas required elements to struct global
This diff is the raw C struct definition of all DeviceAtlas module
data needed added to the main global struct haproxy configuration.
The three first members are needed for both init and deinit phases
as some dynamic memory allocations are done.
The useragentid serves to hold during the whole lifecycle of the
module the User-Agent HTTP Header identifier from the DeviceAtlas
data during the init process.
David Carlier [Mon, 1 Jun 2015 11:50:06 +0000 (13:50 +0200)]
MEDIUM: config: add DeviceAtlas global keywords
This diff is for the DeviceAtlas convertor.
This patch adds the following converters :
deviceatlas-json-file
deviceatlas-log-level
deviceatlas-property-separator
First, the configuration keywords handling (only the log
level configuration part does not end the haproxy process
if it is wrongly set, it fallbacks to the default level).
Furthermore, init, deinit phases and the API lookup phase,
the da_haproxy function which is fed by the input provided
and set all necessary properties chosen via the configuration
to the output, separated by the separator.
Remi Gacogne [Fri, 29 May 2015 14:26:17 +0000 (16:26 +0200)]
MEDIUM: ssl: replace standards DH groups with custom ones
It is likely that powerful adversaries have been pre-computing the
standardized DH groups, because being widely used have made them
valuable targets. While users are advised to generate their own
DH parameters, replace the ones we ship by values been randomly
generated for this product only.
Remi Gacogne [Fri, 29 May 2015 13:53:22 +0000 (15:53 +0200)]
MEDIUM: ssl: add the possibility to use a global DH parameters file
This patch adds the ssl-dh-param-file global setting. It sets the
default DH parameters that will be used during the SSL/TLS handshake when
ephemeral Diffie-Hellman (DHE) key exchange is used, for all "bind" lines
which do not explicitely define theirs.
Thierry FOURNIER [Fri, 29 May 2015 15:31:12 +0000 (17:31 +0200)]
MINOR: lua: Lua tcp action are not final action
Actually, the registered lua actions with "tcp-request lua" and
"tcp-response lua" are final actions. This patch change the action
type type and permit to continue the evaluation of tcp-* processing
after the evaluation of the lua actions.
Thierry FOURNIER [Fri, 29 May 2015 15:29:14 +0000 (17:29 +0200)]
MINOR: tcp: add custom actions that can continue tcp-(request|response) processing
Actually, the tcp-request and tcp-response custom ation are always final
actions. This patch create a new type of action that can permit to
continue the evaluation of tcp-request and tcp-response processing.
Emeric Brun [Tue, 12 May 2015 16:49:09 +0000 (18:49 +0200)]
MAJOR: peers: peers protocol version 2.0
This patch does'nt add any new feature: the functional behavior
is the same than version 1.0.
Technical differences:
In this version all updates on different stick tables are
multiplexed on the same tcp session. There is only one established
tcp session per peer whereas in first version there was one established
tcp session per peer and per stick table.
Messages format was reviewed to be more evolutive and to support
further types of data exchange such as SSL sessions or other sticktable's
data types (currently only the sticktable's server id is supported).
Willy Tarreau [Thu, 28 May 2015 23:09:15 +0000 (01:09 +0200)]
BUG/MEDIUM: http: fix body processing for the stats applet
Commit 9fbe18e ("MEDIUM: http: add a new option http-buffer-request")
introduced a regression due to a misplaced check causing the admin
mode of the HTTP stats not to work anymore.
This patch tried to ensure that when we need a request body for the
stats applet, and we have already waited for this body, we don't wait
for it again, but the condition was applied too early causing a
disabling of the entire processing the body, and based on the wrong
HTTP state (MSG_BODY) resulting in the test never matching.
MEDIUM: cfgparse: max arguments check in the global section
Most of the keywords in the global section does not check the maximum
number of arguments. This leds sometines to unused and wrong arguments
in the configuration file. This patch add a maximum argument test in
many keywords of this section.
Thierry FOURNIER [Thu, 28 May 2015 14:02:01 +0000 (16:02 +0200)]
MEDIUM: stats: escape some strings in the CSV dump
Some strings which must be dumped in the CSV output can contain one of
the following chars : <,>, <">, or CR/LF. This patch escapes these
strings if the case is encountered.
Thierry FOURNIER [Thu, 28 May 2015 14:00:28 +0000 (16:00 +0200)]
MINOR: common: escape CSV strings
This function checks a string for using it in a CSV output format. If
the string contains one of the following four char <">, <,>, CR or LF,
the string is encapsulated between <"> and the <"> are escaped by a <"">
sequence.
The rounding by <"> is optionnal. It can be canceled, forced or the
function choose automatically the right way.
Willy Tarreau [Thu, 28 May 2015 15:23:54 +0000 (17:23 +0200)]
MEDIUM: http: no need to close the request on redirect if data was parsed
There are two reasons for not keeping the client connection alive upon a
redirect :
- save the client from uploading all data
- avoid keeping a connection alive if the redirect goes to another domain
The first case should consider an exception when all the data from the
client have been read already. This specifically happens on response
redirects after a POST to a server. This is an easy situation to detect.
It could later be improved to cover the cases where option
http-buffer-request is used.
Sometimes it's problematic not to have "http-response redirect" rules,
for example to perform a browser-based redirect based on certain server
conditions (eg: match of a header).
This patch adds "http-response redirect location <fmt>" which gives
enough flexibility for most imaginable operations. The connection to
the server is closed when this is performed so that we don't risk to
forward any pending data from the server.
Any pending response data are trimmed so that we don't risk to
forward anything pending to the client. It's harmless to also do that
for requests so we don't need to consider the direction.
Willy Tarreau [Thu, 28 May 2015 13:26:58 +0000 (15:26 +0200)]
MINOR: http: prepare support for parsing redirect actions on responses
In order to support http-response redirect, the parsing needs to be
adapted a little bit to only support the "location" type, and to
adjust the log-format parser so that it knows the direction of the
sample fetch calls.
Willy Tarreau [Fri, 22 May 2015 14:27:37 +0000 (16:27 +0200)]
CLEANUP: http: explicitly reference request in http_apply_redirect_rules()
This function was made to perform a redirect on requests only, it was
using a message or txn->req in an inconsistent way and did not consider
the possibility that it could be used for the other direction. Let's
clean it up to have both a request and a response messages.
Thierry FOURNIER [Tue, 26 May 2015 16:06:31 +0000 (18:06 +0200)]
MEDIUM: capture: adds http-response capture
This patch adds a http response capture keyword with the same behavior
as the previous patch called "MEDIUM: capture: Allow capture with slot
identifier".
Thierry FOURNIER [Tue, 26 May 2015 15:58:29 +0000 (17:58 +0200)]
MEDIUM: capture: Allow capture with slot identifier
This patch modifies the current http-request capture function
and adds a new keyword "id" that permits to identify a capture slot.
If the identified doesn't exists, the action fails silently.
Note that this patch removs an unused list initilisation, which seems
to be inherited from a copy/paste. It's harmless and does not need to
be backported.
Thierry FOURNIER [Thu, 28 May 2015 11:22:03 +0000 (13:22 +0200)]
MINOR: capture: add two "capture" converters
This patch adds "capture-req" and "capture-res". These two converters
capture their entry in the allocated slot given in argument and pass
the input on the output.
Thierry FOURNIER [Tue, 26 May 2015 15:44:32 +0000 (17:44 +0200)]
MINOR: proxy: custom capture declaration
This patch adds a new keyword called "declare". This keyword
allow to declare some capture slots in requests and response.
It is useful for sharing capture between frontend and backends.
Willy Tarreau [Wed, 27 May 2015 14:46:26 +0000 (16:46 +0200)]
MEDIUM: proxy: add a new proxy_find_best_match() function
This function tries to spot a proxy by its name, ID and type, and
in case some elements don't match, it tries to determine which ones
could be ignored and reports which ones were ignored so that the
caller can decide whether or not it wants to pick this proxy. This
will be used for maintaining the status across reloads where the
config might have changed a bit.
Willy Tarreau [Tue, 26 May 2015 13:25:32 +0000 (15:25 +0200)]
MINOR: proxy: add a new function proxy_find_by_id()
It does the same as the other one except that it only focuses on the
numeric ID and the capabilities. It's used by proxy_find_by_name()
for numeric names.
Willy Tarreau [Tue, 26 May 2015 10:18:29 +0000 (12:18 +0200)]
MAJOR: config: remove the deprecated reqsetbe / reqisetbe actions
These ones were already obsoleted in 1.4, marked for removal in 1.5,
and not documented anymore. They used to emit warnings, and do still
require quite some code to stay in place. Let's remove them now.
Willy Tarreau [Tue, 26 May 2015 10:04:09 +0000 (12:04 +0200)]
MEDIUM: config: clarify the conflicting modes detection for backend rules
We don't use findproxy_mode() anymore so we can check the conflicting
modes and report the anomalies accordingly with line numbers and more
explicit details.
Willy Tarreau [Tue, 26 May 2015 09:35:41 +0000 (11:35 +0200)]
MINOR: proxy: simply ignore duplicates in proxy name lookups
Now that we can't have duplicate proxies with similar capabilities, we
can remove some painful check. The first one is the check that made the
lookup function return NULL when a duplicate is found, as it prevented
it from being used in the config parser to detect duplicates.
Willy Tarreau [Tue, 26 May 2015 09:24:42 +0000 (11:24 +0200)]
CLEANUP: proxy: make the proxy lookup functions more user-friendly
First, findproxy() was renamed proxy_find_by_name() so that its explicit
that a name is required for the lookup. Second, we give this function
the ability to search for tables if needed. Third we now provide inline
wrappers to pass the appropriate PR_CAP_* flags and to explicitly look
up a frontend, backend or table.
Willy Tarreau [Tue, 26 May 2015 08:49:46 +0000 (10:49 +0200)]
MEDIUM: config: reject conflicts in table names
A nasty situation happens when two tables have the same name. Since it
is possible to declare a table in a frontend and another one in a backend,
this situation may happen and result in a random behaviour each time a
table is designated in a "stick" or "track" rule. Let's make sure this
is properly detected and stopped. Such a config will now report :
[ALERT] 145/104933 (31571) : parsing [prx.cfg:36] : stick-table name 't' conflicts with table declared in frontend 't' at prx.cfg:30.
[ALERT] 145/104933 (31571) : Error(s) found in configuration file : prx.cfg
[ALERT] 145/104933 (31571) : Fatal errors found in configuration.
Willy Tarreau [Tue, 26 May 2015 08:35:50 +0000 (10:35 +0200)]
MEDIUM: config: reject invalid config with name duplicates
Since 1.4 we used to emit a warning when two frontends or two backends
had the same name. In 1.5 we added the same warning for two peers sections.
In 1.6 we added the same warning for two mailers sections. It's about time
to reject such invalid configurations, the impact they have on the code
complexity is huge and it is becoming a real obstacle to some improvements
such as restoring servers check status across reloads.
Now these errors are reported as fatal errors and will need to be fixed.
Anyway, till now there was no guarantee that what was written was working
as expected since the behaviour is not defined (eg: use_backend with a
name used by two backends leads to undefined behaviour).
Example of output :
[ALERT] 145/104759 (31564) : Parsing [prx.cfg:12]: mailers section 'm' has the same name as another mailers section declared at prx.cfg:10.
[ALERT] 145/104759 (31564) : Parsing [prx.cfg:16]: peers section 'p' has the same name as another peers section declared at prx.cfg:14.
[ALERT] 145/104759 (31564) : Parsing [prx.cfg:21]: frontend 'f' has the same name as another frontend declared at prx.cfg:18.
[ALERT] 145/104759 (31564) : Parsing [prx.cfg:27]: backend 'b' has the same name as another backend declared at prx.cfg:24.
[ALERT] 145/104759 (31564) : Error(s) found in configuration file : prx.cfg
[ALERT] 145/104759 (31564) : Fatal errors found in configuration.
Joseph Lynch [Tue, 12 May 2015 06:25:34 +0000 (23:25 -0700)]
MEDIUM: backend: Allow redispatch on retry intervals
For backend load balancing it sometimes makes sense to redispatch rather
than retrying against the same server. For example, when machines or routers
fail you may not want to waste time retrying against a dead server and
would instead prefer to immediately redispatch against other servers.
This patch allows backend sections to specify that they want to
redispatch on a particular interval. If the interval N is positive the
redispatch occurs on every Nth retry, and if the interval N is negative then
the redispatch occurs on the Nth retry prior to the last retry (-1 is the
default and maintains backwards compatibility). In low latency environments
tuning this setting can save a few hundred milliseconds when backends fail.
Thierry FOURNIER [Tue, 19 May 2015 12:46:23 +0000 (14:46 +0200)]
MINOR: http: split the url_param in two parts
This patch is the part of the body_param fetch. The goal is to have
generic url-encoded parser which can used for parsing the query string
and the body.
Willy Tarreau [Wed, 20 May 2015 08:39:04 +0000 (10:39 +0200)]
BUG/MEDIUM: peers: apply a random reconnection timeout
Commit 9ff95bb ("BUG/MEDIUM: peers: correctly configure the client timeout")
uncovered an old bug in the peers : upon disconnect, we reconnect immediately.
This sometimes results in both ends to do the same thing in parallel causing
a loop of connect/accept/close/close that can last several seconds. The risk
of occurrence of the trouble increases with latency, and is emphasized by the
fact that idle connections are now frequently recycled (after 5s of idle).
In order to avoid this we must apply a random delay before reconnecting.
Fortunately the mechanism already supports a reconnect delay, so here we
compute the random timeout when killing a session. The delay is 50ms plus
a random between 0 and 2 seconds. Ideally an exponential back-off would
be preferred but it's preferable to keep the fix simple.
This bug was reported by Marco Corte.
This fix must be backported to 1.5 since the fix above was backported into
1.5.12.
Willy Tarreau [Thu, 7 May 2015 14:06:18 +0000 (16:06 +0200)]
MEDIUM: http: make url_param iterate over multiple occurrences
There are some situations hwere it's desirable to scan multiple occurrences
of a same parameter name in the query string. This change ensures this can
work, even with an empty name which will then iterate over all parameters.
MEDIUM: Add support for updating TLS ticket keys via socket
Until now, HAproxy needed to be restarted to change the TLS ticket
keys. With this patch, the TLS keys can be updated on a per-file
basis using the admin socket. Two new socket commands have been
introduced: "show tls-keys" and "set ssl tls-keys".
MINOR: Add TLS ticket keys reference and use it in the listener struct
Within the listener struct we need to use a reference to the TLS
ticket keys which binds the actual keys with the filename. This will
make it possible to update the keys through the socket