[RELEASE] Released version 1.3.8.2 with the following changes :
- rewriting either the status line or request line could crash the
process due to a pointer which ought to be reset before parsing.
- rewriting the status line in the response did not work, it caused
a 502 Bad Gateway due to an erroneous state during parsing
[BUG] initialize msg->sol before parsing first line
Before calling http_parse_{sts,req}line(), it is necessary
to make msg->sol point to the beginning of the line. This
was not done, resulting in the proxy sometimes crashing when
URI rewriting or result rewriting was used.
[RELEASE] Released version 1.3.8.1 with the following changes :
- fix reqadd when no option httpclose is used.
- removed now unused fiprm and beprm from proxies
- split logs into two versions : TCP and HTTP
- added some docs about http headers storage and acls
- added a VIM script for syntax color highlighting (Bruno Michel)
Willy Tarreau [Sat, 31 Mar 2007 23:30:43 +0000 (01:30 +0200)]
[MEDIUM] splitted logs into two versions : TCP and HTTP
logs are handled better with dedicated functions. The HTTP implementation
moved to proto_http.c. It has been cleaned up a bit. Now a frontend with
option httplog and no log will not call the function anymore.
Willy Tarreau [Sat, 31 Mar 2007 22:01:37 +0000 (00:01 +0200)]
[MEDIUM] removed now unused fiprm and beprm from proxies
The fiprm and beprm were added to ease the transition between
a single listener mode to frontends+backends. They are no longer
needed and make the code a bit more complicated. Remove them.
Willy Tarreau [Sun, 25 Mar 2007 22:24:56 +0000 (00:24 +0200)]
[RELEASE] Released version 1.3.8 with the following changes :
- fixed several bugs which might have caused a crash with bad configs
- several optimizations in header processing
- many progresses towards transaction-based processing
- option forwardfor may be used in frontends
- completed HTTP response processing
- some code refactoring between request and response processing
- new HTTP header manipulation functions
- optimizations on the recv() patch to reduce CPU usage under very
high data rates.
- more user-friendly help about the 'usesrc' keyword (CTTPROXY)
- username/groupname support from Marcus Rueckert
- added the "except" keyword to the "forwardfor" option (Bryan German)
- support for health-checks on other addresses (Fabrice Dulaunoy)
- makefile for MacOS 10.4 / Darwin (Dan Zinngrabe)
- do not insert "Connection: close" in HTTP/1.0 messages
Willy Tarreau [Sun, 25 Mar 2007 19:03:01 +0000 (21:03 +0200)]
[CLEANUP] slightly reorganized the struct server
Struct server has gathered lots of informations over the time, but
it's better for clarity and performance to group those information
by usage, the most common ones at the top and the least ones at the
bottom.
Willy Tarreau [Sun, 25 Mar 2007 14:55:56 +0000 (16:55 +0200)]
[BUILD] makefile for MacOS 10.4 / Darwin
Contribution from Dan Zinngrabe :
Here is a Makefile based on that for BSD that builds HAProxy 1.3.7 on
MacOS 10.4 and Darwin. I haven't tested it extensively yet, but it
does seem to work so far.
Willy Tarreau [Sun, 25 Mar 2007 14:45:16 +0000 (16:45 +0200)]
[MEDIUM] add support for health-checks on other addresses
Patch from Fabrice Dulaunoy. Explanation below, and script
merged in examples/.
This patch allow to put a different address in the check part for each
server (and not only a specific port)
I need this feature because I've a complex settings where, when a specific
farm goes down, I need to switch a set of other farm either if these other
farm behave perfectly well.
For that purpose, I've made a small PERL daemon with some REGEX or PORT
test which allow me to test a bunch of thing.
Willy Tarreau [Sun, 25 Mar 2007 14:00:04 +0000 (16:00 +0200)]
[MEDIUM] add the "except" keyword to the "forwardfor" option
Patch from Bryan Germann for 1.2.17.
In some circumstances, it is useful not to add the X-Forwarded-For
header, for instance when the client is another reverse-proxy or
stunnel running on the same machine and which already adds it. This
patch adds the "except" keyword to the "forwardfor" option, allowing
to specify an address or network which will not be added to this
header.
Willy Tarreau [Sun, 25 Mar 2007 13:39:23 +0000 (15:39 +0200)]
[MEDIUM] add user/groupname support
Patch from Marcus Rueckert for 1.2.17 :
"I added the attached patch to haproxy. I don't have a static uid/gid for
haproxy so i need to specify the username/groupname to run it as non
root user."
Willy Tarreau [Sat, 24 Mar 2007 16:24:39 +0000 (17:24 +0100)]
[MEDIUM] check for cttproxy support when required
Previously, use of the "usesrc" keyword could silently fail if
either the module was not loaded, or the user did not have enough
permissions. Now the errors are better diagnosed and more appropriate
advices are given.
Willy Tarreau [Sat, 24 Mar 2007 11:47:24 +0000 (12:47 +0100)]
[MINOR] more friendly reports of wrong uses of the usesrc keyword
It was difficult to find how to enter the "usesrc" keyword. Now the
configuration checker is a bit more friendly and tries to identify
most mistakes and gives some hints back.
Willy Tarreau [Fri, 23 Mar 2007 22:02:09 +0000 (23:02 +0100)]
[MINOR] read optimizations based on the MSS
Generally, if a recv() returns less bytes than the MSS, it means that
there is nothing left in the system's buffers, and that it's not worth
trying to read again because we are very likely to get nothing. A
default read low limit has been set to 1460 bytes below which we stop
reading.
This has brought a little speed boost on small objects while maintaining
the same speed on large objects.
Willy Tarreau [Fri, 23 Mar 2007 21:39:59 +0000 (22:39 +0100)]
[MEDIUM] re-implemented the multiple read polling
Multiple read polling was temporarily disabled, which had the side
effect of burning huge amounts of CPU on large objects. It has now
been re-implemented with a limit of 8 calls per wake-up, which seems
to provide best results at least on Linux.
Willy Tarreau [Mon, 19 Mar 2007 15:20:06 +0000 (16:20 +0100)]
[BUG] fix pointer initializations for TCP connections.
Very recent changes consisting in moving some pointers to the
transaction instead of the session have lead to a bug because
those pointers were only initialized if the protocol was HTTP,
but they were freed based on their value. In some cases, it
was possible to cause double frees.
Willy Tarreau [Sun, 18 Mar 2007 22:50:16 +0000 (23:50 +0100)]
[MINOR] added new function http_header_match2()
HTTP header matching is now made easier with http_header_match2().
Various locations have been adapted to use it. A small bug was also
fixed causing empty headers to be matched till next one.
Willy Tarreau [Sun, 18 Mar 2007 21:36:26 +0000 (22:36 +0100)]
[MINOR] HTTP: factorize all the header insertions
Two new functions http_header_add_tail() and http_header_add_tail2()
make it easier to append headers, and also reduce the number of
sprintf() calls and perform stricter checks.
Willy Tarreau [Sun, 18 Mar 2007 17:34:41 +0000 (18:34 +0100)]
[MINOR] move some flags from session.h to proto_http.h
Some session flags were clearly related to HTTP transactions.
A new 'flags' field has been added to http_txn, and the
associated flags moved to proto_http.h.
Willy Tarreau [Sun, 18 Mar 2007 15:22:39 +0000 (16:22 +0100)]
[MAJOR] completed the HTTP response processing.
Now the response is correctly processed in the backend first
then in the frontend. It has followed intensive tests to
catch regressions, and everything seems OK now, but the code
is young anyway.
Willy Tarreau [Sun, 4 Mar 2007 17:17:17 +0000 (18:17 +0100)]
[MINOR] code factoring : capture_headers() serves requests and responses
Both request and response captures will have to parse headers following
the same methods. It's better to factorize the code, hence the new
capture_headers() function.
Willy Tarreau [Sat, 3 Mar 2007 12:54:32 +0000 (13:54 +0100)]
[CLEANUP] renamed several HTTP structures
Some parts of HTTP processing were incorrectly called "request" while
they are messages or transactions. The following structure members
have changed :
Willy Tarreau [Sun, 11 Feb 2007 23:59:08 +0000 (00:59 +0100)]
[MINOR] slightly optimize time calculation for rbtree
The new rbtree-based scheduler makes heavy use of tv_cmp2(), and
this function becomes a huge CPU eater. Refine it a little bit in
order to slightly reduce CPU usage.
Willy Tarreau [Thu, 1 Feb 2007 22:15:45 +0000 (23:15 +0100)]
[BUG] segfault on some erroneous configurations
If captures were configured in a TCP-only listener, and
the logs were enabled, the proxy could segfault when
trying to scan the capture buffer which was NULL. Such
an erroneous configuration will not be possible anymore
soon, but let's avoid the problem for now by detecting
the NULL condition.
Willy Tarreau [Fri, 26 Jan 2007 22:49:01 +0000 (23:49 +0100)]
[RELEASE] Released version 1.3.7 with the following changes :
- fix critical bug introduced with 1.3.6 : an empty request header
may lead to a crash due to missing pointer assignment
- hdr_idx might be left uninitialized in debug mode
- fixed build on FreeBSD due to missing fd_set declaration
Willy Tarreau [Fri, 26 Jan 2007 22:39:38 +0000 (23:39 +0100)]
[CRITICAL] an empty header may lead to a crash
A missing pointer assignment in case of an empty header
will result in this header's length being 65535, causing
a SEGV when accessing the next header. It should not be
possible to exploit this problem to run arbitrary code
because the crash occurs while reading the data.
Willy Tarreau [Thu, 25 Jan 2007 11:03:42 +0000 (12:03 +0100)]
[BUG] hdr_idx might be left uninitialized in some cases
When a request is invalid during RQ_BEFORE AND the debug mode is active,
the hdr_idx might be used uninitialized. Let's initialize it right after
the accept() for now.
Willy Tarreau [Wed, 24 Jan 2007 17:20:50 +0000 (18:20 +0100)]
[BUILD] fix build on FreeBSD (missing fd_set declaration)
Sorin Pop reported a patch to fix build on FreeBSD.
The file common/standard.h used an fd_set in a declaration
but did not include enough headers for it to be known.
Willy Tarreau [Mon, 22 Jan 2007 07:55:47 +0000 (08:55 +0100)]
[MAJOR] invalid header offset broke cookies and authentication
Since the request is no longer part of the headers, cookies and
authentication did not work anymore. Obvious fix is to add the
request offset to the start pointer.
Willy Tarreau [Sun, 21 Jan 2007 23:56:46 +0000 (00:56 +0100)]
[RELEASE] Released 1.3.6 with the following changes :
- stats now support the HEAD method too
- extracted http request from the session
- huge rework of the HTTP parser which is now a 28-state FSM.
- linux-style likely/unlikely macros for optimization hints
- do not create a server socket when there's no server
Willy Tarreau [Sun, 21 Jan 2007 18:16:41 +0000 (19:16 +0100)]
[MAJOR] huge rework of the HTTP request FSM
The HTTP parser has been rewritten for better compliance to RFC2616.
The same parser is now usable for both requests and responses, and
it now supports HTTP/0.9 as well as multi-line headers. It has also
been improved for speed ; a typicial HTTP request is parsed in about
2 microseconds on a 1 GHz processor.
The monitor-uri check has been moved so that the requests are not
logged. The httpclose option now tries to change as little as
possible in the request, and does not affect the first header if
it is already set to 'close'. HTTP/0.9 requests are converted to
HTTP/1.0 before being forwarded.
Headers and request transformations are now distinct. The headers
list is updated after each insertion/removal/transformation. The
request is re-parsed and checked after each transformation. It is
not possible anymore to remove a request, and requests which lead
to invalid request lines are now rejected.
Willy Tarreau [Sat, 20 Jan 2007 10:07:46 +0000 (11:07 +0100)]
[MINOR] do not create a socket if there is no server
Since the distinction of backends and frontends, it has become
possible that some requests reach a frontend which has no
backend parameters. We must not create a socket on the backend
side just to destroy it later in such a case. The real problem
comes from the dispatch mode not being explictly stated.
Willy Tarreau [Sun, 7 Jan 2007 14:46:13 +0000 (15:46 +0100)]
[MEDIUM] separate the http request from the session (step 1)
A struct http_req has been created to collect every information
related to an HTTP request being processed. Right now, it is
still in the struct session but the frontier is clear now.
Willy Tarreau [Sun, 7 Jan 2007 12:47:30 +0000 (13:47 +0100)]
[MEDIUM] Stats: add support for the HEAD method
There are browsers which sometimes send HEAD requests to the stats
page, but it was not handled so it returned a 503 server error or
was simply sent to the default backend servers.
Now with a HEAD request, the stats return the headers and finish
there. Normally, other methods should be blocked so that the stats
page really catches the whole URI. Other methods would need to cause
a 405 Method not allowed to be returned.
Willy Tarreau [Sun, 7 Jan 2007 01:47:01 +0000 (02:47 +0100)]
[RELEASE] Released 1.3.5 with the following major changes :
- added complete support and doc for TCP Splicing
- replaced the wait-queue linked list with an rbtree.
- stats: swap color sets for active and backup servers
- try to guess server check port when unset
- a few bugfixes and cleanups
Willy Tarreau [Sun, 7 Jan 2007 01:40:09 +0000 (02:40 +0100)]
[MINOR] try to guess server check port when unset
When a server has no port specified and there is a check
enabled on it, the check is disabled because the port is
unknown. However, people expect the "listen" line to set
the check port just like it sets the server's port. Now,
if a port is specified in the listen or in the first bind
and nowhere else, it will be used for the checks as well.
Willy Tarreau [Sat, 6 Jan 2007 23:38:00 +0000 (00:38 +0100)]
[MAJOR] replace the wait-queue linked list with an rbtree.
This patch from Sin Yu makes use of an rbtree for the wait queue,
which will solve the slowdown problem encountered when timeouts
are heterogenous in the configuration. The next step will be to
turn maintain_proxies() into a per-proxy task so that we won't
have to scan them all after each poll() loop.
Willy Tarreau [Sun, 7 Jan 2007 01:03:04 +0000 (02:03 +0100)]
[MAJOR] complete support and doc for tcp-splicing
The tcp-splicing code has been merged, and a doc has been written.
A configuration example has been derived from the previous content
switching sample.
Willy Tarreau [Sat, 6 Jan 2007 20:09:17 +0000 (21:09 +0100)]
[MINOR] the options table now sets the prerequisite checks
Some options will need some checks (or initializations) to be performed
before starting everything. The cfg_opts table has been extended to
allow storing of options-dependant checks.
Willy Tarreau [Mon, 1 Jan 2007 23:44:53 +0000 (00:44 +0100)]
[RELEASE] released 1.3.4
Released 1.3.4 with the following major changes :
- support for cttproxy on the server side to present the client
address to the server.
- added support for SO_REUSEPORT on Linux (needs kernel patch)
- new RFC2616-compliant HTTP request parser with header indexing
- split proxies in frontends, rulesets and backends
- implemented the 'req[i]setbe' to select a backend depending
on the contents
- added the 'default_backend' keyword to select a default BE.
- new stats page featuring FEs and BEs + bytes in both dirs
- improved log format to indicate the backend and the time in ms.
- lots of cleanups
Willy Tarreau [Mon, 1 Jan 2007 22:32:30 +0000 (23:32 +0100)]
[CRITICAL] fixed memory leak in session_free()
Since the introduction of hdr_idx, session_free() had not
been updated to free the header ! It implied a consumption
of about 400 bytes per new session.
Willy Tarreau [Mon, 1 Jan 2007 20:38:07 +0000 (21:38 +0100)]
[MAJOR] udpated the stats page to clearly distinguish FEs and BEs
The stats page could not tell the difference between a FE and a BE.
It has been revamped to indicate all relevant information. The font
is also slightly smaller in order for all the info to fit into small
screens. The data output path has been greatly simplified to use
string chunks.
Willy Tarreau [Sat, 30 Dec 2006 23:24:10 +0000 (00:24 +0100)]
[MEDIUM] use an array to store most common options
Most common options are now stored in an array which eases
the parsing and which also permits reporting of ignored
options depending on the proxy's capabilities (back/front).
Willy Tarreau [Sat, 30 Dec 2006 22:43:54 +0000 (23:43 +0100)]
[MINOR] option httpclose is now checked both in FE and BE
The "httpclose" option affects both frontend and backend, so it
was logical to check for its presence at both places. A request
which traverses either a frontend or a backend with this option
set will have a "Connection: close" header appended.
Willy Tarreau [Sat, 30 Dec 2006 10:54:15 +0000 (11:54 +0100)]
[MEDIUM] updated log format to report frontend and backend
The log format has been slightly updated to separately report the name
of the frontend and the name of the backend. The accept date has been
enhanced to report the millisecond. The number of remaining connections
has also been updated and their order reversed, to include the number
of connections on the frontend. The new log format is now :
- $1: IP:port
- $2: accept date in this format : [dd/mm/YYYY:HH:MM:SS.ttt]
- $3: frontend name
- $4: backend name '/' server name
- $5: req time '/' queue time '/' conn time '/' header time '/' total time
- $6: HTTP status code
- $7: number of bytes returned
- $8: captures (request)
- $9: captures (response)
- $10: completion flags
- $11: remaining conns on process '/' frontend '/' backend '/' server
- $12: srv queue size '/' backend queue size
- $13..: '"' full request '"'
Willy Tarreau [Fri, 29 Dec 2006 13:19:17 +0000 (14:19 +0100)]
[MAJOR] distinguish between frontend, backend, ruleset and listen
The notion of capabilities has been added to the proxy so that we
know whether a proxy supports frontend, backend, or rulesets. Given
this, some parameters are optionnal, some are ignored with a warning
and others are forbidden. It is now possible to write valid two level
configs without binding to dummy address/ports.
Willy Tarreau [Thu, 28 Dec 2006 23:10:33 +0000 (00:10 +0100)]
[MEDIUM] split fe->maxconn into fe->maxconn and be->fullconn
The maxconn argument is used only for the listeners, and the
fullconn is used only for the backends. If unset, it inherits
maxconn's value which itself can inherit the default or the
global value (we might need to change this).
Willy Tarreau [Wed, 27 Dec 2006 16:18:38 +0000 (17:18 +0100)]
[MEDIUM] session logging is now defined by the frontend
To solve the logging maze, it has been decided that the frontend
and nothing else will define how a session will be logged. It might
change in the future but at least this choice allows all sort of
fantasies.
Willy Tarreau [Sun, 24 Dec 2006 16:47:20 +0000 (17:47 +0100)]
[MEDIUM] errorloc now checked first from backend then from frontend
It is now possible to define an errorloc in the backend as well as
in the frontend. The backend's will be used first, and if undefined,
then the frontend's will be used instead. If none is used, then the
original error messages will be used.
Willy Tarreau [Sat, 23 Dec 2006 19:51:41 +0000 (20:51 +0100)]
[MINOR] store HTTP error messages into a chunk array
HTTP error messages were all specific cases handled by an IF.
Now they are all in an array so that it will be easier to add
new ones. Also, the return functions now use chunks as inputs
so that it should be easier to provide alternative return
messages if needed.
Willy Tarreau [Sat, 23 Dec 2006 10:12:04 +0000 (11:12 +0100)]
[BUILD] makefile now detects and uses git to set the version
If git is found during the build process, then it will be used
to set the version, the commit number and the commit date. This
way, it will not be needed anymore to update the code to change
the version. The version is the last tag, and the commit number
is the number of commits since the last tag.