[MINOR] http: take http request timeout from the backend
Since we can now switch from TCP to HTTP, we need to be able to apply
the HTTP request timeout after switching. That means we need to take
it from the backend and not from the frontend. Since the backend points
to the frontend before switching, that changes nothing for the normal
case.
[MINOR] ensure we can jump from swiching rules to http without data
In case of switching from TCP to HTTP, we want the HTTP request timeout
to be properly initialized. For this, we have to jump to the analyser
without breaking out of the loop nor waiting for incoming data. The way
it is done right now is not particularly clean but it works.
A cleaner method might involve pushing function pointers into a circular
list.
[MEDIUM] allow a TCP frontend to switch to an HTTP backend
This patch allows a TCP frontend to switch to an HTTP backend.
During the switch, missing structures are automatically allocated.
The HTTP parser is enabled so that the backend first waits for a
full HTTP request.
Now that we can perform TCP-based content switching, it makes sense
to be able to detect HTTP traffic and act accordingly. We already
have an HTTP decoder, we just have to call it in order to detect HTTP
protocol. Note that since the decoder will automatically fill in the
interesting fields of the HTTP transaction, it would make sense to
use this parsing to extend HTTP matching to TCP.
[MINOR] http: rely on proxy->acl_requires to allocate hdr_idx
Right now only HTTP proxies may use HTTP headers in ACLs, but
when this evolves, we'll need to be able to allocate the hdr_idx
on demand. The solution consists in allocating it only when it is
certain that at least one ACL requires HTTP parsing, regardless
of the mode the proxy is in. This is what is achieved by this
patch.
[MINOR] report in the proxies the requirements for ACLs
This patch propagates the ACL conditions' "requires" bitfield
to the proxies. This makes it possible to know exactly what a
proxy might have to support for any request, which helps knowing
whether we have to allocate some space for certain types of
structures or not (eg: the hdr_idx struct).
The concept might be extended to a lot more types of information,
such as detecting whether we need to allocate some space for some
request ACLs which need a result in the response, etc...
[MAJOR] http: complete splitting of the remaining stages
The HTTP processing has been splitted into 7 steps, one of which
is not anymore HTTP-specific (content-switching). That way, it
becomes possible to use "use_backend" rules in TCP mode. A new
"use_server" directive should follow soon.
[MEDIUM] session: tell analysers what bit they were called for
Some stream analysers might become generic enough to be called
for several bits. So we cannot have the analyser bit hard coded
into the analyser itself. Let's make the caller inform the callee.
[MEDIUM] http: split request waiter from request processor
We want to split several steps in HTTP processing so that
we can call individual analysers depending on what processing
we want to perform. The first step consists in splitting the
part that waits for a request from the rest.
[BUG] http: redirect rules were processed too early
redirect rules are documented as being processed last before
use_backend but were mistakenly processed before block rules.
Fortunately very few people use a mix of block and redirect
rules, so this bug has never been reported yet.
Willy Tarreau [Sun, 28 Jun 2009 21:10:19 +0000 (23:10 +0200)]
[MEDIUM] splice: set the capability on each stream_interface
The splice code did not consider compatibility between both ends
of the connection. Now we set different capabilities on each
stream interface, depending on what the protocol can splice to/from.
Right now, only TCP is supported. Thanks to this, we're now able to
automatically detect when splice() is not implemented and automatically
disable it on one end instead of reporting errors to the upper layer.
Willy Tarreau [Sun, 28 Jun 2009 17:37:53 +0000 (19:37 +0200)]
[MEDIUM] session: rework buffer analysis to permit permanent analysers
It will soon be necessary to support permanent analysers (eg: HTTP in
keep-alive mode). We first have to slightly rework the call to the
request analysers so that we don't force ->analysers to be 0 before
forwarding data.
Willy Tarreau [Sun, 28 Jun 2009 09:09:07 +0000 (11:09 +0200)]
[MEDIUM] stream_sock: don't close prematurely when nolinger is set
When the nolinger option is used, we must not close too fast because
some data might be left unsent. Instead we must proceed with a normal
shutdown first, then a close. Also, we want to avoid merging FIN with
the last segment if nolinger is set, because if that one gets lost,
there is no chance for it to be retransmitted.
Willy Tarreau [Mon, 22 Jun 2009 13:48:36 +0000 (15:48 +0200)]
[MEDIUM] config: split parser and checker in two functions
This is a first step towards support of multiple configuration files.
Now readcfgfile() only reads a file in memory and performs very minimal
parsing. The checks are performed afterwards.
Willy Tarreau [Sun, 21 Jun 2009 20:43:05 +0000 (22:43 +0200)]
[MAJOR] session: simplify buffer error handling
Buffer errors (timeouts and I/O errors) were handled at two places,
just after the analysers and after again.
Now that the timeout detection has moved, it has become easier to
handle those errors.
This has also made it possible for the request and response analysers
to be processed together as a down-up event, and all the up-down I/O
updates to be processed afterwards, which is exactly what we're looking
for. Interestingly this has reduced the number of iterations of
(stream_int, req_resp) from (5,6,5) to (5,5,4).
Several tests have been run without any issue found.
Willy Tarreau [Sun, 21 Jun 2009 20:03:51 +0000 (22:03 +0200)]
[MAJOR] session: only check for timeouts when they have just occurred.
It's useless to check for buffer timeouts every time we call
process_session() because we already control when we set the flag. So
let's check them at the precise moment where the flag is set.
Willy Tarreau [Sun, 21 Jun 2009 19:45:58 +0000 (21:45 +0200)]
[MAJOR] session: don't clear buffer status flags anymore
We want to be able to keep information about errors and timeouts
as long as possible in the buffer. Let's not clear these flags
anymore and keep them static. This does not seem to cause any
trouble, though a finer review might be wise.
Willy Tarreau [Sun, 14 Jun 2009 16:48:19 +0000 (18:48 +0200)]
[MEDIUM] add support for TCP MSS adjustment for listeners
Sometimes it can be useful to limit the advertised TCP MSS on
incoming connections, for instance when requests come through
a VPN or when the system is running with jumbo frames enabled.
Passing the "mss <value>" arguments to a "bind" line will set
the value. This works under Linux >= 2.6.28, and maybe a few
earlier ones, though due to an old kernel bug most of earlier
versions will probably ignore it. It is also possible that some
other OSes will support this.
Yitzhak Sapir [Sun, 14 Jun 2009 16:27:54 +0000 (18:27 +0200)]
[BUILD] add support for build under Cygwin
After considering various possibilities, we compiled haproxy under cygwin.
Attached is an updated full diff that also has the TARGET=cygwin documented.
The whole thing compiles and installs with this diff only.
In cygwin 1.7 (now in beta), there is apparently support for ipv6. Cygwin
1.5 (later versions, anyway) already includes some support in the form of a
define USE_IPV6. When defined, it declares the sockaddr_in6 struct and
possibly other things. The above definition AF_INET6=23 is taken from
their /usr/include/socket.h file (where it is #if 0'd out).
We are running into a socket limit. It appears that Cygwin (running on
Windows 2003 Server) will only allow us to set ulimit -n (maximum open
files) to 3200, which means we're a little short of 1600 connections.
The limit of 3200 is an internal Cygwin limit. Perhaps they can raise it in
the future. Using the nbproc option, I was able to bring up 10 servers. It
seems to me that they were able to handle over 2000 connections (even though
each had maxconn 1500 set, and the hard Cygwin fd limit).
Willy Tarreau [Sun, 14 Jun 2009 13:48:17 +0000 (15:48 +0200)]
[MEDIUM] implement tcp-smart-connect option at the backend
This new option enables combining of request buffer data with
the initial ACK of an outgoing TCP connection. Doing so saves
one packet per connection which is quite noticeable on workloads
mostly consisting in small objects. The option is not enabled by
default.
Willy Tarreau [Sun, 14 Jun 2009 13:24:37 +0000 (15:24 +0200)]
[MEDIUM] stream_sock: implement tcp-cork for use during shutdowns on Linux
Setting TCP_CORK on a socket before sending the last segment enables
automatic merging of this segment with the FIN from the shutdown()
call. Playing with TCP_CORK is not easy though as we have to track
the status of the TCP_NODELAY flag since both are mutually exclusive.
Doing so saves one more packet per session and offers about 5% more
performance.
There is no reason not to do it, so there is no associated option.
Willy Tarreau [Sun, 14 Jun 2009 10:07:01 +0000 (12:07 +0200)]
[MEDIUM] implement option tcp-smart-accept at the frontend
This option disables TCP quick ack upon accept. It is also
automatically enabled in HTTP mode, unless the option is
explicitly disabled with "no option tcp-smart-accept".
This saves one packet per connection which can bring reasonable
amounts of bandwidth for servers processing small requests.
Willy Tarreau [Sun, 14 Jun 2009 09:39:52 +0000 (11:39 +0200)]
[MINOR] config: support resetting options do default values
A new keyword prefix "default" has been introduced in order to
reset some options to their default values. This can be needed
for instance when an option is forced disabled or enabled in a
defaults section and when later sections want to use automatic
settings regardless of what was specified there. Right now it
is only supported by options, just like the "no" prefix.
Willy Tarreau [Sun, 14 Jun 2009 09:10:45 +0000 (11:10 +0200)]
[MINOR] config: track "no option"/"option" changes
Sometimes we would want to implement implicit default options,
but for this we need to be able to disable them, which requires
to keep track of "no option" settings. With this change, an option
explicitly disabled in a defaults section will still be seen as
explicitly disabled. There should be no regression as nothing makes
use of this yet.
Willy Tarreau [Wed, 10 Jun 2009 09:09:37 +0000 (11:09 +0200)]
[MEDIUM] add support for binding to source port ranges during connect
Some users are already hitting the 64k source port limit when
connecting to servers. The system usually maintains a list of
unused source ports, regardless of the source IP they're bound
to. So in order to go beyond the 64k concurrent connections, we
have to manage the source ip:port lists ourselves.
The solution consists in assigning a source port range to each
server and use a free port in that range when connecting to that
server, either for a proxied connection or for a health check.
The port must then be put back into the server's range when the
connection is closed.
This mechanism is used only when a port range is specified on
a server. It makes it possible to reach 64k connections per
server, possibly all from the same IP address. Right now it
should be more than enough even for huge deployments.
Willy Tarreau [Tue, 9 Jun 2009 12:36:00 +0000 (14:36 +0200)]
[BUG] ensure that we correctly re-start old process in case of error
When a new process fails to grab some ports, it sends a signal to
the old process in order to release them. Then it tries to bind
again. If it still fails (eg: one of the ports is bound to a
completely different process), it must send the continue signal
to the old process so that this one re-binds to the ports. This
is correctly done, but the newly bound ports are not released
first, which sometimes causes the old process to remain running
with no port bound. The fix simply consists in unbinding all
ports before sending the signal to the old process.
Willy Tarreau [Mon, 18 May 2009 14:29:51 +0000 (16:29 +0200)]
[MINOR] startup: don't imply -q with -D
It is recommended to have -D in init scripts, but -D also implies
quiet mode, which hides warning messages, and both options are now
completely unrelated. Remove the implication to get warnings with
-D.
Willy Tarreau [Sun, 10 May 2009 18:27:47 +0000 (20:27 +0200)]
[RELEASE] Released version 1.3.18
Released version 1.3.18 with the following main changes :
- [MEDIUM] add support for "balance hdr(name)"
- [CLEANUP] give a little bit more information in error message
- [MINOR] add X-Original-To: header
- [BUG] x-original-to: fix missing initialization to default value
- [BUILD] spec file: fix broken pipe during rpmbuild and add man file
- [MINOR] improve reporting of misplaced acl/reqxxx rules
- [MEDIUM] http: add options to ignore invalid header names
- [MEDIUM] http: capture invalid requests/responses even if accepted
- [BUILD] add format(printf) to printf-like functions
- [MINOR] fix several printf formats and missing arguments
- [BUG] stats: total and lbtot are unsigned
- [MINOR] fix a few remaining printf-like formats on 64-bit platforms
- [CLEANUP] remove unused make option from haproxy.spec
- [BUILD] make it possible to pass alternative arch at build time
- [MINOR] switch all stat counters to 64-bit
- [MEDIUM] ensure we don't recursively call pool_gc2()
- [CRITICAL] uninitialized response field can sometimes cause crashes
- [BUG] fix wrong pointer arithmetics in HTTP message captures
- [MINOR] rhel init script : support the reload operation
- [MINOR] add basic signal handling functions
- [BUILD] add signal.o to all makefiles
- [MEDIUM] call signal_process_queue from run_poll_loop
- [MEDIUM] pollers: don't wait if a signal is pending
- [MEDIUM] convert all signals to asynchronous signals
- [BUG] O(1) pollers should check their FD before closing it
- [MINOR] don't close stdio fds twice
- [MINOR] add options dontlog-normal and log-separate-errors
- [DOC] minor fixes and rearrangements
- [BUG] fix parser crash on unconditional tcp content rules
- [DOC] rearrange the configuration manual and add a summary
- [MINOR] standard: provide a new 'my_strndup' function
- [MINOR] implement per-logger log level limitation
- [MINOR] compute the max of sessions/s on fe/be/srv
- [MINOR] stats: report max sessions/s and limit in CSV export
- [MINOR] stats: report max sessions/s and limit in HTML stats
- [MINOR] stats/html: use the arial font before helvetica
Willy Tarreau [Sun, 10 May 2009 18:08:10 +0000 (20:08 +0200)]
[MINOR] stats/html: use the arial font before helvetica
The stats HTML output were barely readable on some browsers such as
firefox on Linux, due to the selected helvetica font which is too
small. Specifying "arial" first fixes the issue without changing the
table size. Also, the default size of 0.8em choosen to get 10px out
of 12px is wrong because it gets 9px when rounded down.
Willy Tarreau [Sun, 10 May 2009 16:52:49 +0000 (18:52 +0200)]
[MINOR] compute the max of sessions/s on fe/be/srv
Some users want to keep the max sessions/s seen on servers, frontends
and backends for capacity planning. It's easy to grab it while the
session count is updated, so let's keep it.
Willy Tarreau [Sun, 10 May 2009 15:20:05 +0000 (17:20 +0200)]
[MINOR] implement per-logger log level limitation
Some people are using haproxy in a shared environment where the
system logger by default sends alert and emerg messages to all
consoles, which happens when all servers go down on a backend for
instance. These people can not always change the system configuration
and would like to limit the outgoing messages level in order not to
disturb the local users.
The addition of an optional 4th field on the "log" line permits
exactly this. The minimal log level ensures that all outgoing logs
will have at least this level. So the logs are not filtered out,
just set to this level.
Benoit [Wed, 25 Mar 2009 12:02:10 +0000 (13:02 +0100)]
[MEDIUM] add support for "balance hdr(name)"
There is a patch made by me that allow for balancing on any http header
field.
[WT:
made minor changes:
- turned 'balance header name' into 'balance hdr(name)' to match more
closely the ACL syntax for easier future convergence
- renamed the proxy structure fields header_* => hh_*
- made it possible to use the domain name reduction to any header, not
only "host" since it makes sense to do it with other ones.
Otherwise patch looks good.
/WT]
Willy Tarreau [Sun, 10 May 2009 11:12:33 +0000 (13:12 +0200)]
[DOC] rearrange the configuration manual and add a summary
Several people have asked for a summary in order to ease finding
of sections in the configuration manual. It was the opportunity to
tidy it up a bit and rearrange some sections.
Willy Tarreau [Sun, 10 May 2009 09:57:02 +0000 (11:57 +0200)]
[MINOR] add options dontlog-normal and log-separate-errors
Some big traffic sites have trouble dealing with logs and tend to
disable them. Here are two new options to help cope with massive
logs.
- dontlog-normal only disables logging for 100% successful
connections, other ones will still be logged
- log-separate-errors will cause non-100% successful connections
to be logged at level "err" instead of level "info" so that a
properly configured syslog daemon can send them to a different
file for longer conservation.
Willy Tarreau [Sun, 10 May 2009 08:18:54 +0000 (10:18 +0200)]
[BUG] O(1) pollers should check their FD before closing it
epoll, sepoll and kqueue pollers should check that their fd is not
closed before attempting to close it, otherwise we can end up with
multiple closes of fd #0 upon exit, which is harmless but dirty.
Willy Tarreau [Sun, 10 May 2009 07:59:50 +0000 (09:59 +0200)]
[MEDIUM] convert all signals to asynchronous signals
The small list of signals currently handled by haproxy were processed
as soon as they were received. This has caused trouble with calls to
pool_gc2() occuring in the middle of libc's memory management functions
seldom causing deadlocks preventing the old process from leaving.
Now these signals use the new async signal framework and are called
asynchronously, when there is no risk of recursion. This ensures more
reliable operation, especially for sensible processing such as memory
management.
Willy Tarreau [Sun, 10 May 2009 07:57:21 +0000 (09:57 +0200)]
[MEDIUM] pollers: don't wait if a signal is pending
If an asynchronous signal is received outside of the poller, we don't
want the poller to wait for a timeout to occur before processing it,
so we set its timeout to zero, just like we do with pending tasks in
the run queue.
Willy Tarreau [Sun, 10 May 2009 06:53:33 +0000 (08:53 +0200)]
[MINOR] add basic signal handling functions
These functions will be used to deliver asynchronous signals in order
to make the signal handling functions more robust. The goal is to keep
the same interface to signal handlers.
I have attached a patch which will add on every http request a new
header 'X-Original-To'. If you have HAProxy running in transparent mode
with a big number of SQUID servers behind it, it is very nice to have
the original destination ip as a common header to make decisions based
on it.
The whole thing is configurable with a new option 'originalto'. I have
updated the sourcecode as well as the documentation. The 'haproxy-en.txt'
and 'haproxy-fr.txt' files are untouched, due to lack of my french
language knowledge. ;)
Also the patch adds this header for IPv4 only. I haven't any IPv6 test
environment running here and don't know if getsockopt() with SO_ORIGINAL_DST
will work on IPv6. If someone knows it and wants to test it I can modify
the diff. Feel free to ask me questions or things which should be changed. :)
Willy Tarreau [Fri, 1 May 2009 09:33:17 +0000 (11:33 +0200)]
[BUG] fix wrong pointer arithmetics in HTTP message captures
The pointer arithmetics was wrong in http_capture_bad_message().
This has no impact right now because the error only msg->som was
affected and right now it's always 0. But this was a bug waiting
for keepalive support to strike.
[CRITICAL] uninitialized response field can sometimes cause crashes
The response message in the transaction structure was not properly
initialised at session initialisation. In theory it cannot cause any
trouble since the affected field os expected to always remain NULL.
However, in some circumstances, such as building on 64-bit platforms
with certain options, the struct session can be exactly 1024 bytes,
the same size of the requri field, so the pools are merged and the
uninitialised field may contain non-null data, causing crashes if
an invalid response is encountered and archived.
The fix simply consists in correctly initialising the missing fields.
This bug cannot affect architectures where the session pool is not
shared (32-bit architectures), but this is only by pure luck.
[MEDIUM] ensure we don't recursively call pool_gc2()
A race condition exists in the hot reconfiguration code. It is
theorically possible that the second signal is sent during a free()
in the first list, which can cause crashes or freezes (the later
have been observed). Just set up a counter to ensure we do not
recurse.
The byte counters have long been 64-bit to avoid overflows. But with
several sites nowadays, we see session counters wrap around every 10-days
or so. So it was the moment to switch counters to 64-bit, including
error and warning counters which can theorically rise as fast as session
counters even if in practice there is very low risk.
The performance impact should not be noticeable since those counters are
only updated once per session. The stats output have been carefully checked
for proper types on both 32- and 64-bit platforms.
[BUILD] make it possible to pass alternative arch at build time
When trying to build a 32-bit binary on a 64-bit platform, we generally
need to pass "-m32" to gcc, which is not convenient with current makefile.
Note that this option requires gcc >= 3.
In order to ease parameter passing, a new ARCH= makefile option has been
added. If it receives a target architecture, according "-m32"/"-m64" and
"-march=xxxx" will be passed to gcc. Only the generic makefile has been
changed to support this option right now as the need only appeared on Linux.
The spec file now makes use of this option so that rpmbuild can automatically
build with the proper architecture.
[MEDIUM] http: capture invalid requests/responses even if accepted
It's useful to be able to accept an invalid header name in a request
or response but still be able to monitor further such errors. Now,
when an invalid request/response is received and accepted due to
an "accept-invalid-http-{request|response}" option, the invalid
request will be captured for later analysis with "show errors" on
the stats socket.
[MEDIUM] http: add options to ignore invalid header names
Sometimes it is required to let invalid requests pass because
applications sometimes take time to be fixed and other servers
do not care. Thus we provide two new options :
option accept-invalid-http-request (for the frontend)
option accept-invalid-http-response (for the backend)
When those options are set, invalid requests or responses do
not cause a 403/502 error to be generated.
Willy Tarreau [Sun, 29 Mar 2009 13:26:57 +0000 (15:26 +0200)]
[RELEASE] Released version 1.3.17
Released version 1.3.17 with the following main changes :
- Update specfile to build for v2.6 kernel.
- [BUG] reset the stream_interface connect timeout upon connect or error
- [BUG] reject unix accepts when connection limit is reached
- [MINOR] show sess: report number of calls to each task
- [BUG] don't call epoll_ctl() on closed sockets
- [BUG] stream_sock: disable I/O on fds reporting an error
- [MINOR] sepoll: don't count two events on the same FD.
- [MINOR] show sess: report a lot more information about sessions
- [BUG] stream_sock: check for shut{r,w} before refreshing some timeouts
- [BUG] don't set an expiration date directly from now_ms
- [MINOR] implement ulltoh() to write HTML-formatted numbers
- [MINOR] stats/html: group digits by 3 to clarify numbers
- [BUILD] remove haproxy-small.spec
- [BUILD] makefile: remove unused references to linux24eold and EPOLL_CTL_WORKAROUND
[PATCH] Update specfile to build for v2.6 kernel.
- Fix date in changelog.
- Stop using deprecated "REGEX=pcre", and start using "USE_PCRE=1" instead.
- Disable RPM-processing of perl dependencies, since haproxy
shouldn't depend on perl, and it's only the examples/check script
that's using perl.
Willy Tarreau [Sun, 29 Mar 2009 11:41:58 +0000 (13:41 +0200)]
[MINOR] implement ulltoh() to write HTML-formatted numbers
This function sets CSS letter spacing after each 3rd digit. The page must
create a class "rls" (right letter spacing) with style "letter-spacing: 0.3em"
in order to use it.
Willy Tarreau [Sun, 29 Mar 2009 08:18:41 +0000 (10:18 +0200)]
[BUG] stream_sock: check for shut{r,w} before refreshing some timeouts
Under some circumstances, it appears possible to refresh a timeout
just after a side has been shut. For instance, if poll() plans to
call both read and write, and the read side calls chk_snd() which
in turn causes a shutw to occur, then stream_sock_write could update
its write timeout. The same problem happens the other way.
The timeout checks will then not catch these cases because they
ignore timeouts in case of shut{r,w}.
This is very likely to be the major cause of the 100% CPU usages
reported by Bart Bobrowski.
The fix consists in always ensuring that a side is not shut before
updating its timeout.
Willy Tarreau [Sat, 28 Mar 2009 23:18:14 +0000 (00:18 +0100)]
[MINOR] show sess: report a lot more information about sessions
For complex troubleshooting, it's sometimes useful to be able to
completely dump all the states and flags related to a session.
Now "show sess" will report the stream interfaces and buffers
status for each session.
Willy Tarreau [Sat, 28 Mar 2009 20:10:48 +0000 (21:10 +0100)]
[MINOR] sepoll: don't count two events on the same FD.
sepoll counts the number of speculative events it has processed in
order to remain fair with epoll_wait(). If a same FD is processed
both for read and for write, it is counted twice. Fix this.
Willy Tarreau [Sat, 28 Mar 2009 19:54:53 +0000 (20:54 +0100)]
[BUG] stream_sock: disable I/O on fds reporting an error
Upon read or write error, we cannot immediately close the FD because
we want to first report the error to the upper layer which will do it
itself. However, we want to prevent any further I/O from being performed
on the FD. This is especially important in case of speculative I/O where
nothing else could stop the FD from still being polled until the upper
layer takes care of the condition.
Willy Tarreau [Sat, 28 Mar 2009 18:43:06 +0000 (19:43 +0100)]
[BUG] don't call epoll_ctl() on closed sockets
Some I/O callbacks are able to close their socket themselves. We
want to check this before calling epoll_ctl(EPOLL_CTL_DEL), otherwise
we get a -1 EBADF. Right now is looks like this could not cause any
trouble but the case is racy enough to fix it.
Willy Tarreau [Sat, 28 Mar 2009 10:02:18 +0000 (11:02 +0100)]
[BUG] reject unix accepts when connection limit is reached
unix sockets are not attached to a real frontend, so there is
no way to disable/enable the listener depending on the global
session count. For this reason, if the global maxconn is reached
and a unix socket comes in, it will just be ignored and remain
in the poll list, which will call again indefinitely.
So we need to accept then drop incoming unix connections when
the table is full.
This should not happen with clean configurations since the global
maxconn should provide enough room for unix sockets.
Willy Tarreau [Sun, 22 Mar 2009 22:46:12 +0000 (23:46 +0100)]
[RELEASE] Released version 1.3.16
Released version 1.3.16 with the following main changes :
- [BUILD] Fixed Makefile for linking pcre
- [CONTRIB] selinux policy for haproxy
- [MINOR] show errors: encode backslash as well as non-ascii characters
- [MINOR] cfgparse: some cleanups in the consistency checks
- [MINOR] cfgparse: set backends to "balance roundrobin" by default
- [MINOR] tcp-inspect: permit the use of no-delay inspection
- [MEDIUM] reverse internal proxy declaration order to match configuration
- [CLEANUP] config: catch and report some possibly wrong rule ordering
- [BUG] connect timeout is in the stream interface, not the buffer
- [BUG] session: errors were not reported in termination flags in TCP mode
- [MINOR] tcp_request: let the caller take care of errors and timeouts
- [CLEANUP] http: remove some commented out obsolete code in process_response
- [MINOR] update ebtree to version 4.1
- [MEDIUM] scheduler: get rid of the 4 trees thanks and use ebtree v4.1
- [BUG] sched: don't leave 3 lasts tasks unprocessed when niced tasks are present
- [BUG] scheduler: fix improper handling of duplicates __task_queue()
- [MINOR] sched: permit a task to stay up between calls
- [MINOR] task: keep a task count and clean up task creators
- [MINOR] stats: report number of tasks (active and running)
- [BUG] server check intervals must not be null
- [OPTIM] stream_sock: don't retry to read after a large read
- [OPTIM] buffer: new BF_READ_DONTWAIT flag reduces EAGAIN rates
- [MEDIUM] session: don't resync FSMs on non-interesting changes
- [BUG] check for global.maxconn before doing accept()
- [OPTIM] sepoll: do not re-check whole list upon accepts
Willy Tarreau [Sun, 22 Mar 2009 18:25:46 +0000 (19:25 +0100)]
[OPTIM] sepoll: do not re-check whole list upon accepts
There is already an optimisation in the speculative poller which
causes newly created FDs to be checked immediately after being
created. Unfortunately, this optimisation causes the whole spec
list to be re-checked while we're only interested in the new FDs.
Doing this minor change causes performance gains of up to 6% on
medium-sized objects with a few hundreds concurrent connections.
Willy Tarreau [Sat, 21 Mar 2009 21:43:12 +0000 (22:43 +0100)]
[BUG] check for global.maxconn before doing accept()
If the accept() is done before checking for global.maxconn, we can
accept too many connections and encounter a lack of file descriptors
when trying to connect to the server. This is the cause of the
"cannot get a server socket" message encountered in debug mode
during injections with low timeouts.
Willy Tarreau [Sat, 21 Mar 2009 21:09:29 +0000 (22:09 +0100)]
[MEDIUM] session: don't resync FSMs on non-interesting changes
While processing the session, we used to resync the FSMs when buffer
flags changed. But since BF_KERN_SPLICING and BF_READ_DONTWAIT were
introduced, sometimes we could resync after they were set, which is
not what we want. This was because there were some old checks left
which did not mask changes with BF_MASK_STATIC before checking.
Willy Tarreau [Sat, 21 Mar 2009 20:10:04 +0000 (21:10 +0100)]
[OPTIM] buffer: new BF_READ_DONTWAIT flag reduces EAGAIN rates
When the reader does not expect to read lots of data, it can
set BF_READ_DONTWAIT on the request buffer. When it is set,
the stream_sock_read callback will not try to perform multiple
reads, it will return after only one, and clear the flag.
That way, we can immediately return when waiting for an HTTP
request without trying to read again.
On pure request/responses schemes such as monitor-uri or
redirects, this has completely eliminated the EAGAIN occurrences
and the epoll_ctl() calls, resulting in a performance increase of
about 10%. Similar effects should be observed once we support
HTTP keep-alive since we'll immediately disable reads once we
get a full request.
Willy Tarreau [Sat, 21 Mar 2009 19:43:57 +0000 (20:43 +0100)]
[OPTIM] stream_sock: don't retry to read after a large read
If we get very large data at once, it's almost certain that it's
worthless trying to read again, because we got everything we could
get.
Doing this has made all -EAGAIN disappear from splice reads. The
threshold has been put in the global tunable structures so that if
we one day want to make it accessible from user config, it will be
easy to do so.
Willy Tarreau [Sat, 21 Mar 2009 17:13:21 +0000 (18:13 +0100)]
[MINOR] task: keep a task count and clean up task creators
It's sometimes useful at least for statistics to keep a task count.
It's easy to do by forcing the rare task creators to always use the
same functions to create/destroy a task.
Willy Tarreau [Sat, 21 Mar 2009 12:26:05 +0000 (13:26 +0100)]
[MINOR] sched: permit a task to stay up between calls
If a task wants to stay in the run queue, it is possible. It just
needs to wake itself up. We just want to ensure that a reniced
task will be processed at the right instant.
Willy Tarreau [Sat, 21 Mar 2009 11:51:40 +0000 (12:51 +0100)]
[BUG] scheduler: fix improper handling of duplicates __task_queue()
The top of a duplicate tree is not where bit == -1 but at the most
negative bit. This was causing tasks to be queued in reverse order
within duplicates. While this is not dramatic, it's incorrect and
might lead to longer than expected duplicate depths under some
circumstances.
Willy Tarreau [Sat, 21 Mar 2009 10:53:09 +0000 (11:53 +0100)]
[BUG] sched: don't leave 3 lasts tasks unprocessed when niced tasks are present
When there are niced tasks, we would only process #tasks/4 per
turn, without taking care of running #tasks when #tasks was below
4, leaving those tasks waiting for a few other tasks to push them.
Willy Tarreau [Sat, 21 Mar 2009 09:01:42 +0000 (10:01 +0100)]
[MEDIUM] scheduler: get rid of the 4 trees thanks and use ebtree v4.1
Since we're now able to search from a precise expiration date in
the timer tree using ebtree 4.1, we don't need to maintain 4 trees
anymore. Not only does this simplify the code a lot, but it also
ensures that we can always look 24 days back and ahead, which
doubles the ability of the previous scheduler. Indeed, while based
on absolute values, the timer tree is now relative to <now> as we
can always search from <now>-31 bits.
The run queue uses the exact same principle now, and is now simpler
and a bit faster to process. With these changes alone, an overall
0.5% performance gain was observed.
Tests were performed on the few wrapping cases and everything works
as expected.