Thierry FOURNIER [Tue, 10 Dec 2013 17:54:58 +0000 (18:54 +0100)]
MEDIUM: dumpstat: make the CLI parser understand the backslash as an escape char
We'll need to pass patterns on the CLI for lookups. Till now there was no
need for a backslash, so it's still time to support them just like in the
config file.
MINOR: pattern: add support for compiling patterns for lookups
With this patch, patterns can be compiled for two modes :
- match
- lookup
The match mode is used for example in ACLs or maps. The lookup mode
is used to lookup a key for pattern maintenance. For example, looking
up a network is different from looking up one address belonging to
this network.
A special case is made for regex. In lookup mode they return the input
regex string and do not compile the regex.
MEDIUM: pattern: Extract the index process from the pat_parse_*() functions
Now, the pat_parse_*() functions parses the incoming data. The input
"pattern" struct can be preallocated. If the parser needs to add some
buffers, it allocates memory.
The function pattern_register() runs the call to the parser, process
the key indexation and associate the "sample_storage" used by maps.
This patch remove the compatibility check from the input type and the
match method. Now, it checks if a casts from the input type to output
type exists and the pattern_exec_match() function apply casts before
each pattern matching.
MINOR: pattern: Each pattern sets the expected input type
This is used later for increasing the compability with incoming
sample types. When multiple compatible types are supported, one
is arbitrarily used (eg: UINT).
Thierry FOURNIER [Wed, 11 Dec 2013 11:38:57 +0000 (12:38 +0100)]
BUG/MEDIUM: sample: conversion from str to ipv6 may read data past end
Applying inet_pton() to input contents is not reliable because the
function requires a zero-terminated string. While inet_pton() will
stop when contents do not match an IPv6 address anymore, it could
theorically read past the end of a buffer if the data to be converted
was at the end of a buffer (this cannot happen right now thanks to
the reserve at the end of the buffer). At least the conversion does
not work.
Fix this by using buf2ip6() instead, which copies the string into a
padded aread.
This bug came with recent commit b805f71 (MEDIUM: sample: let the
cast functions set their output type), no backport is needed.
Thierry FOURNIER [Wed, 11 Dec 2013 10:20:24 +0000 (11:20 +0100)]
BUG/MEDIUM: map: Bad map file parser
There is a mix-up between input type of the data and input type of the
map file. This mix-up causes that all pattern matching function based
on "string" (reg, beg, end, ...) don't run.
This bug came with commit d5f624d (MEDIUM: sample: add the "map" converter),
no backport is needed.
Willy Tarreau [Wed, 11 Dec 2013 20:40:11 +0000 (21:40 +0100)]
BUG/MEDIUM: checks: agent doesn't get the response if server does not closes
The agent refrains from reading the server's response until the server
closes, but if the server waits for the client to close, the response
is never read. Let's try to fetch a whole line before deciding to wait
more.
Godbach [Wed, 11 Dec 2013 13:11:41 +0000 (21:11 +0800)]
BUG/MINOR: check_config_validity: check the returned value of stktable_init()
The function stktable_init() will return 0 if create_pool() returns NULL. Since
the returned value of this function is ignored, HAProxy will crash if the pool
of stick table is NULL and stksess_new() is called to allocate a new stick
session. It is a better choice to check the returned value and make HAProxy exit
with alert message if any error is caught.
Willy Tarreau [Tue, 10 Dec 2013 17:58:23 +0000 (18:58 +0100)]
BUG/MEDIUM: channel: bo_getline() must wait for \n until buffer is full
We must not report incomplete data if the buffer is not full, otherwise
we can abort some processing on the stats socket when dealing with massive
amounts of commands.
Willy Tarreau [Tue, 10 Dec 2013 10:19:04 +0000 (11:19 +0100)]
BUILD: use format tags in VERDATE and SUBVERS files
The first line now contains a git format tag asking git-archive to
place the last commit's commit date and the last commit's abbreviated
ID respectively. The makefile will use these information in preference
when they're available and git is not available.
Now it's only necessary to add the two following lines in
.git/info/attributes to have the files automatically filled by git-archive :
Willy Tarreau [Tue, 10 Dec 2013 10:16:09 +0000 (11:16 +0100)]
BUILD: prepare the makefile to skip format lines in SUBVERS and VERDATE
We're going to put format lines in these files for use by git archive,
so let's ensure that the current default format still works. For this
we'll use two lines and only take the first one without a format tag.
Willy Tarreau [Tue, 10 Dec 2013 08:31:06 +0000 (09:31 +0100)]
BUILD: simplify the date and version retrieval in the makefile
The makefile currently uses some complex and non-always portable
methods to retrieve the date and version (eg: linux's date command).
For the date, we can use git log -1 --pretty=format:%ci instead of
date+sed. For the version, it's easier and safer to count single log
lines.
Note that the VERSION variable was wrong since it could contain the
version+subversion instead of just the version. This is now fixed by
adding --abbrev=0 in describe.
Lukas Tribus [Tue, 10 Dec 2013 07:32:56 +0000 (08:32 +0100)]
BUILD/MINOR: systemd: fix compiler warning about unused result
There is a compiler warning after commit 1b6e75fa84 ("MEDIUM: haproxy-
systemd-wrapper: Use haproxy in same directory"):
src/haproxy-systemd-wrapper.c: In function â\80\98locate_haproxyâ\80\99:
src/haproxy-systemd-wrapper.c:28:10: warning: ignoring return value of â\80\98readlinkâ\80\99, declared with attribute warn_unused_result [-Wunused-result]
Fix the compiler warning by checking the return value of readlink().
Willy Tarreau [Mon, 14 May 2012 10:11:47 +0000 (12:11 +0200)]
MINOR: stream_interface: add reporting of ressouce allocation errors
SSL and keep-alive will need to be able to fail on allocation errors,
and the stream interface did not allow to report such a cause. The flag
will then be "RC" as already documented.
Willy Tarreau [Sat, 7 Dec 2013 00:01:39 +0000 (01:01 +0100)]
DIET/MINOR: task: reduce struct task size by 8 bytes
Just by reordering the struct task, we could shrink it by 8 bytes from
120 to 112 bytes. A careful reordering allowed each part to be located
closer to the hot parts it's used with, resulting in another performance
increase of about 0.5%.
Willy Tarreau [Fri, 6 Dec 2013 23:01:53 +0000 (00:01 +0100)]
MINOR: http: switch the http state to an enum
This reduces its size which is not reused by anything else. However it
will significantly improve the debugger's output since we'll now get
real state values.
The default case had to be enabled in the parsers because gcc tries
to optimize the switch/case and noticed some values were missing from
the enums and emitted a warning.
Willy Tarreau [Fri, 6 Dec 2013 22:43:17 +0000 (23:43 +0100)]
DIET/MINOR: http: reduce the size of struct http_txn by 8 bytes
Here again we had some oversized and misaligned entries. The method
and the status don't need 4 bytes each, and there was a hole after
the status that does not exist anymore. That's 8 additional bytes
saved from http_txn and as much for the session.
Also some fields were slightly moved to present better memory access
patterns resulting in a steady 0.5% performance increase.
Willy Tarreau [Fri, 6 Dec 2013 22:31:58 +0000 (23:31 +0100)]
DIET/MINOR: stream-int: rearrange a few fields in struct stream_interface to save 8 bytes
The current and previous states are now packed enums instead of ints. This will
also help in gdb. The flags have been turned to 16-bit instead of 32 since only
10 are used. This resulted in saving 8 bytes per streamm interface, or 16 per
session.
Willy Tarreau [Fri, 6 Dec 2013 20:29:13 +0000 (21:29 +0100)]
DIET/MINOR: proxy: rearrange a few fields in struct proxy to save 16 bytes
Turn the proxy state to a packed enum (1 char), same for the proxy mode,
and store the capabitilies as a char. These 3 ints can now fill the hole
after obj_type and save 8 bytes in the proxy struct. Moving the maxconn
value just after, which is frequently accessed and was in a block of 3
ints saved another 8 bytes.
Willy Tarreau [Fri, 6 Dec 2013 20:09:57 +0000 (21:09 +0100)]
DIET/MINOR: connection: rearrange a few fields to save 8 bytes in the struct
By moving the error code to 8 bits the send_proxy_ofs to 16 bits, and
moving them just after the obj_type, we can save 8 bytes in the struct
connection, down from 328 to 320.
Willy Tarreau [Fri, 6 Dec 2013 20:08:11 +0000 (21:08 +0100)]
DIET/MINOR: obj: pack the obj_type enum to 8 bits
Taking 32-bit in each struct just to store an obj_type is a waste
considering the very small amount of possible values. Let's force
it to be as small as possible (1 char) and we'll be able to move
some structs around to save some space.
Willy Tarreau [Sun, 1 Dec 2013 19:45:00 +0000 (20:45 +0100)]
MINOR: stats: provide some appctx information in "show sess all"
When dumping a session, it can be useful to know what applet it is
connected to instead of having just the appctx pointer. We also
report st0/st1/st2 to help debugging.
Willy Tarreau [Sun, 1 Dec 2013 11:47:35 +0000 (12:47 +0100)]
CLEANUP: peers: use less confusing state/status code names
Currently, all states, all status codes and a few constants used in
the peers are all prefixed with "PEER_SESSION_". It's confusing because
there is no way to know which one is a state, a status code or anything
else. Thus, let's rename them this way :
PEER_SESS_ST_* : states
PEER_SESS_SC_* : status codes
Additionally the states have been numbered from zero and contigously.
This will allow us not to have to deal with the stream interface
initialization anymore and to ease debugging using enums.
Willy Tarreau [Sun, 1 Dec 2013 11:25:52 +0000 (12:25 +0100)]
MEDIUM: session: automatically register the applet designated by the target
Some applet users don't need to initialize their applet, they just want
to route the traffic there just as if it were a server. Since applets
are now connected to from session.c, let's simply ensure that when
connecting, the applet in si->end matches the target, and allocate
one there if it's not already done. In case of error, we force the
status code to resource and connection so that it's clear that it
happens because of a memory shortage.
Willy Tarreau [Sun, 1 Dec 2013 10:31:38 +0000 (11:31 +0100)]
MAJOR: stream-interface: dynamically allocate the applet context
From now on, a call to stream_int_register_handler() causes a call
to si_alloc_appctx() and returns an initialized appctx for the
current stream interface. If one was previously allocated, it is
released. If the stream interface was attached to a connection, it
is released as well.
The appctx are allocated from the same pools as the connections, because
they're substantially smaller in size, and we can't have both a connection
and an appctx on an interface at any moment.
In case of memory shortage, the call may return NULL, which is already
handled by all consumers of stream_int_register_handler().
The field appctx was removed from the stream interface since we only
rely on the endpoint now. On 32-bit, the stream_interface size went down
from 108 to 44 bytes. On 64-bit, it went down from 144 to 64 bytes. This
represents a memory saving of 160 bytes per session.
It seems that a later improvement could be to move the call to
stream_int_register_handler() to session.c for most cases.
Willy Tarreau [Sun, 1 Dec 2013 08:35:41 +0000 (09:35 +0100)]
MEDIUM: stream-int: return the allocated appctx in stream_int_register_handler()
The task returned by stream_int_register_handler() is never used, however we
always need to access the appctx afterwards. So make it return the appctx
instead. We already plan for it to fail, which is the reason for the addition
of a few tests and the possibility for the HTTP analyser to return a status
code 500.
Willy Tarreau [Sun, 1 Dec 2013 08:15:12 +0000 (09:15 +0100)]
MEDIUM: stream-int: replace occurrences of si->appctx with si_appctx()
We're about to remove si->appctx, so first let's replace all occurrences
of its usage with a dynamic extract from si->end. A lot of code was changed
by search-n-replace, but the behaviour was intentionally not altered.
The code surrounding calls to stream_int_register_handler() was slightly
changed since we can only use si->end *after* the registration.
Willy Tarreau [Thu, 24 Oct 2013 20:01:26 +0000 (22:01 +0200)]
MEDIUM: connection: merge the send_proxy and local_send_proxy calls
We used to have two very similar functions for sending a PROXY protocol
line header. The reason is that the default one relies on the stream
interface to retrieve the other end's address, while the "local" one
performs a local address lookup and sends that instead (used by health
checks).
Now that the send_proxy_ofs is stored in the connection and not the
stream interface, we can make the local_send_proxy rely on it and
support partial sends. This also simplifies the code by removing the
local_send_proxy function, making health checks use send_proxy_ofs,
resulting in the removal of the CO_FL_LOCAL_SPROXY flag, and the
associated test in the connection handler. The other flag,
CO_FL_SI_SEND_PROXY was renamed without the "SI" part so that it
is clear that it is not dedicated anymore to a usage with a stream
interface.
Willy Tarreau [Thu, 24 Oct 2013 19:10:08 +0000 (21:10 +0200)]
MEDIUM: connection: move the send_proxy offset to the connection
Till now the send_proxy_ofs field remained in the stream interface,
but since the dynamic allocation of the connection, it makes a lot
of sense to move that into the connection instead of the stream
interface, since it will not be statically allocated for each
session.
Also, it turns out that moving it to the connection fils an alignment
hole on 64 bit architectures so it does not consume more memory, and
removing it from the stream interface was an opportunity to correctly
reorder fields and reduce the stream interface's size from 160 to 144
bytes (-10%). This is 32 bytes saved per session.
Willy Tarreau [Fri, 11 Oct 2013 17:34:20 +0000 (19:34 +0200)]
MAJOR: stream interface: dynamically allocate the outgoing connection
The outgoing connection is now allocated dynamically upon the first attempt
to touch the connection's source or destination address. If this allocation
fails, we fail on SN_ERR_RESOURCE.
As we didn't use si->conn anymore, it was removed. The endpoints are released
upon session_free(), on the error path, and upon a new transaction. That way
we are able to carry the existing server's address across retries.
The stream interfaces are not initialized anymore before session_complete(),
so we could even think about allocating them dynamically as well, though
that would not provide much savings.
The session initialization now makes use of conn_new()/conn_free(). This
slightly simplifies the code and makes it more logical. The connection
initialization code is now shorter by about 120 bytes because it's done
at once, allowing the compiler to remove all redundant initializations.
The si_attach_applet() function now takes care of first detaching the
existing endpoint, and it is called from stream_int_register_handler(),
so we can safely remove the calls to si_release_endpoint() in the
application code around this call.
A call to si_detach() was made upon stream_int_unregister_handler() to
ensure we always free the allocated connection if one was allocated in
parallel to setting an applet (eg: detect HTTP proxy while proceeding
with stats maybe).
Willy Tarreau [Thu, 24 Oct 2013 13:50:53 +0000 (15:50 +0200)]
MEDIUM: stream-interface: introduce si_attach_conn to replace si_prepare_conn
si_prepare_conn() is not appropriate in our case as it both initializes and
attaches the connection to the stream interface. Due to the asymmetry between
accept() and connect(), it causes some fields such as the control and transport
layers to be reinitialized.
Now that we can separately initialize these fields using conn_prepare(), let's
break this function to only attach the connection to the stream interface.
Also, by analogy, si_prepare_none() was renamed si_detach(), and
si_prepare_applet() was renamed si_attach_applet().
Willy Tarreau [Thu, 24 Oct 2013 13:31:04 +0000 (15:31 +0200)]
MINOR: connection: replace conn_assign with conn_attach
We don't want to assign the control nor transport layers anymore
at the same time as the data layer, because it prevents one from
keeping existing settings when reattaching a connection to an
existing stream interface.
Let's have conn_attach() replace conn_assign() for this purpose.
Thus, conn_prepare() + conn_attach() do exactly the same as the
previous conn_assign().
Willy Tarreau [Thu, 24 Oct 2013 13:08:37 +0000 (15:08 +0200)]
MINOR: connection: reintroduce conn_prepare to set the protocol and transport
Now that we can assign conn->xprt regardless of the initialization state,
we can reintroduce conn_prepare() to set only the protocol, the transport
layer and initialize the transport layer's state.
Willy Tarreau [Thu, 24 Oct 2013 09:51:38 +0000 (11:51 +0200)]
MINOR: stream-interface: introduce si_reset() and si_set_state()
The first function is used to (re)initialize a stream interface and
the second to force it into a known state. These are intended for
cleaning up the stream interface initialization code in session.c
and peers.c and avoiding future issues with missing initializations.
Willy Tarreau [Mon, 21 Oct 2013 14:30:56 +0000 (16:30 +0200)]
MAJOR: connection: add two new flags to indicate readiness of control/transport
Currently the control and transport layers of a connection are supposed
to be initialized when their respective pointers are not NULL. This will
not work anymore when we plan to reuse connections, because there is an
asymmetry between the accept() side and the connect() side :
- on accept() side, the fd is set first, then the ctrl layer then the
transport layer ; upon error, they must be undone in the reverse order,
then the FD must be closed. The FD must not be deleted if the control
layer was not yet initialized ;
- on the connect() side, the fd is set last and there is no reliable way
to know if it has been initialized or not. In practice it's initialized
to -1 first but this is hackish and supposes that local FDs only will
be used forever. Also, there are even less solutions for keeping trace
of the transport layer's state.
Also it is possible to support delayed close() when something (eg: logs)
tracks some information requiring the transport and/or control layers,
making it even more difficult to clean them.
So the proposed solution is to add two flags to the connection :
- CO_FL_CTRL_READY is set when the control layer is initialized (fd_insert)
and cleared after it's released (fd_delete).
- CO_FL_XPRT_READY is set when the control layer is initialized (xprt->init)
and cleared after it's released (xprt->close).
The functions have been adapted to rely on this and not on the pointers
anymore. conn_xprt_close() was unused and dangerous : it did not close
the control layer (eg: the socket itself) but still marks the transport
layer as closed, preventing any future call to conn_full_close() from
finishing the job.
The problem comes from conn_full_close() in fact. It needs to close the
xprt and ctrl layers independantly. After that we're still having an issue :
we don't know based on ->ctrl alone whether the fd was registered or not.
For this we use the two new flags CO_FL_XPRT_READY and CO_FL_CTRL_READY. We
now rely on this and not on conn->xprt nor conn->ctrl anymore to decide what
remains to be done on the connection.
In order not to miss some flag assignments, we introduce conn_ctrl_init()
to initialize the control layer, register the fd using fd_insert() and set
the flag, and conn_ctrl_close() which unregisters the fd and removes the
flag, but only if the transport layer was closed.
Similarly, at the transport layer, conn_xprt_init() calls ->init and sets
the flag, while conn_xprt_close() checks the flag, calls ->close and clears
the flag, regardless xprt_ctx or xprt_st. This also ensures that the ->init
and the ->close functions are called only once each and in the correct order.
Note that conn_xprt_close() does nothing if the transport layer is still
tracked.
conn_full_close() now simply calls conn_xprt_close() then conn_full_close()
in turn, which do nothing if CO_FL_XPRT_TRACKED is set.
In order to handle the error path, we also provide conn_force_close() which
ignores CO_FL_XPRT_TRACKED and closes the transport and the control layers
in turns. All relevant instances of fd_delete() have been replaced with
conn_force_close(). Now we always know what state the connection is in and
we can expect to split its initialization.
Willy Tarreau [Mon, 14 Oct 2013 17:46:29 +0000 (19:46 +0200)]
MEDIUM: connection: replace conn_prepare with conn_assign
Everywhere conn_prepare() is used, the call to conn_init() has already
been done. We can now safely replace all instances of conn_prepare()
with conn_assign() which does not reset the transport layer, and remove
conn_prepare().
Willy Tarreau [Mon, 14 Oct 2013 15:10:08 +0000 (17:10 +0200)]
MINOR: connection: add conn_init() to (re)initialize a connection
This function will ease the initialization of new connections as well
as their reuse. It initializes the obj_type and a few fields so that
the connection is fresh again. It leaves the addresses and target
untouched so it is suitable for use across connection retries.
Willy Tarreau [Mon, 14 Oct 2013 19:32:07 +0000 (21:32 +0200)]
MEDIUM: session: attach incoming connection to target on embryonic sessions
In order to reduce the dependency over stream-interfaces, we now
attach the incoming connection to the embryonic session's target
instead of the stream-interface's connection. This means we won't
need to initialize stream interfaces anymore after we implement
dynamic connection allocation. The session's target is reset to
NULL after the session has been converted to a complete session.
Willy Tarreau [Tue, 1 Oct 2013 15:12:05 +0000 (17:12 +0200)]
MEDIUM: stream-int: do not allocate a connection in parallel to applets
When we know we're not going to use a connection on a stream interface
because we're using an applet instead, do not allocate a connection, or
release the preallocated one. We do that for peers and CLI only at the
moment, and not for HTTP stats which in the future might be adapted to
support keep-alive.
The connection pointer is simply set to NULL, which pool_free2() already
supports.
Willy Tarreau [Tue, 1 Oct 2013 08:45:07 +0000 (10:45 +0200)]
MAJOR: stream-int: stop using si->conn and use si->end instead
The connection will only remain there as a pre-allocated entity whose
goal is to be placed in ->end when establishing an outgoing connection.
All connection initialization can be made on this connection, but all
information retrieved should be applied to the end point only.
This change is huge because there were many users of si->conn. Now the
only users are those who initialize the new connection. The difficulty
appears in a few places such as backend.c, proto_http.c, peers.c where
si->conn is used to hold the connection's target address before assigning
the connection to the stream interface. This is why we have to keep
si->conn for now. A future improvement might consist in dynamically
allocating the connection when it is needed.
CLEANUP: stream-int: remove obsolete si_ctrl function
This function makes no sense anymore and will cause trouble to convert
the remains of connection/applet to end points. Let's replace it now
with its contents.
MEDIUM: stream-int: make ->end point to the connection or the appctx
The long-term goal is to have a context for applets as an alternative
to the connection and not as a complement. At the moment, the context
is still stored into the stream interface, and we only put a pointer
to the applet's context in si->end, initialize the context with object
type OBJ_TYPE_APPCTX, and this allows us not to allocate an entry when
deciding to switch to an applet.
A special care is taken to never dereference si->conn anymore when
dealing with an applet. That's why it's important that si->end is
always set to the proper type :
si->end == NULL => not connected to anything
*si->end == OBJ_TYPE_APPCTX => connected to an applet
*si->end == OBJ_TYPE_CONN => real connection (server, proxy, ...)
The session management code used to check the applet from the connection's
target. Now it uses the stream interface's end point and does not touch the
connection at all. Similarly, we stop checking the connection's addresses
and file descriptors when reporting the applet's status in the stats dump.
Willy Tarreau [Thu, 24 Oct 2013 18:10:45 +0000 (20:10 +0200)]
MAJOR: stream interface: remove the ->release function pointer
Since last commit, we now have a pointer to the applet in the
applet context. So we don't need the si->release function pointer
anymore, it can be extracted from applet->applet.release. At many
places, the ->release function was still tested for real connections
while it is only limited to applets, so most of them were simply
removed. For the remaining valid uses, a new inline function
si_applet_release() was added to simplify the check and the call.
Willy Tarreau [Thu, 24 Oct 2013 18:03:27 +0000 (20:03 +0200)]
MEDIUM: stream-interface: set the pointer to the applet into the applet context
In preparation for a later move of all the applet context outside of the
stream interface, we'll need to have access to the applet itself from the
context. Let's have a pointer to it inside the context.
MINOR: stream-int: split si_prepare_embedded into si_prepare_none and si_prepare_applet
si_prepare_embedded() was used both to attach an applet and to detach
anything from a stream interface. Split it into si_prepare_none() to
detach and si_prepare_applet() to attach an applet.
si->conn->target is now assigned from within these two functions instead
of their respective callers.
Willy Tarreau [Sun, 24 Nov 2013 08:38:33 +0000 (09:38 +0100)]
MINOR: stream-int: rename ->applet to ->appctx
Since this is the applet context, call it ->appctx to avoid the confusion
with the pointer to the applet. Many places were changed but it's only a
renaming.
Willy Tarreau [Thu, 24 Oct 2013 20:45:25 +0000 (22:45 +0200)]
MINOR: obj: introduce a new type appctx
The object type was added to "struct appctx". The purpose will be
to identify an appctx when the applet context is detached from the
stream interface. For now, it's still attached, so this patch only
adds the new type and does not replace its use.
Willy Tarreau [Sun, 24 Nov 2013 01:28:49 +0000 (02:28 +0100)]
MINOR: stream-interface: move the applet context to its own struct
In preparation of making the applet context dynamically allocatable,
we create a "struct appctx". Nothing else was changed, it's the same
struct as the one which was in the stream interface.
Willy Tarreau [Thu, 24 Oct 2013 18:44:19 +0000 (20:44 +0200)]
MEDIUM: stream interface: move the peers' ptr into the applet context
A long time ago when peers were introduced, there was no applet nor
applet context. Applet contexts were introduced but the peers still
did not make use of them and the "ptr" pointer remains present in
every stream interface in addition to the other contexts.
Simply move this pointer to its own location in the context.
Note that this pointer is still a void* because its type and contents
varies depending on the peers session state. Probably that this could
be cleaned up in the future given that all other contexts already store
much more than a single pointer.
MINOR: obj: provide a safe and an unsafe access to pointed objects
Most of the times, the caller of objt_<type>(ptr) will know that <ptr>
is valid and of the correct type (eg: in an "if" condition). Let's provide
an unsafe variant that does not perform the check again for these usages.
The new functions are called "__objt_<type>".
MINOR: stream-int: make the shutr/shutw functions void
This is to be more consistent with the other functions. The only
reason why these functions used to return a value was to let the
caller adjust polling by itself, but now their only callers were
the si_shutr()/si_shutw() inline functions. Now these functions
do not depend anymore on the connection.
These connection variant of these functions now call
conn_data_stop_recv()/conn_data_stop_send() before returning order
not to require a return code anymore. The applet version does not
need this at all.
MEDIUM: stream-int: split the shutr/shutw functions between applet and conn
These functions induce a lot of ifs everywhere because they consider two
different cases, one which is where the connection exists and has a file
descriptor, and the other one which is the default case where at most an
applet has to be notified.
Let's have them in si_ops and automatically decide which one to use.
The connection shutdown sequence has been slightly simplified, and we
now clear the flags at the end.
Also we remove SHUTR_NOW after a shutw with nolinger, as it's cleaner
not to keep it.
Willy Tarreau [Sat, 30 Nov 2013 08:21:49 +0000 (09:21 +0100)]
MAJOR: session: pass applet return traffic through the response analysers
Now that applets work like real connections, there is no reason for
them to evade the response analysers. The stats applet emits valid
HTTP responses, it can flow through the HTTP response analyser just
fine. This now allows http-response/rsprep/rspadd rules to be applied
on top of stats. Cookie insertion does nothing since applets are not
servers and thus do not have a cookie. We can imagine compression to be
applied later if the stats output is emitted in chunks and in HTTP/1.1.
A minor visible effect of this change is that there is no more "-1" in
the timers presented in the logs when viewing the stats, all timers are
real.
Willy Tarreau [Sat, 30 Nov 2013 08:06:53 +0000 (09:06 +0100)]
MAJOR: session: check for a connection to an applet in sess_prepare_conn_req()
Instead of having applets bypass the whole connection process, we now
follow the common path through sess_prepare_conn_req(). It is this
function which detects an applet an sets the output state so SI_ST_EST
instead of initiating a connection to a server. It is made possible
because we now have s->target pointing to the applet.
Willy Tarreau [Fri, 25 Oct 2013 16:41:10 +0000 (18:41 +0200)]
MEDIUM: session: detect applets from the session by using s->target
We used to rely on the stream interface's target to detect an applet
from within the session while trying to process the connection request,
but this is incorrect, as this target is the one currently connected
and not the next one to process. This will make a difference when we
later support keep-alive. The only "official" value indicating where
we want to connect is the session's target, which can be :
- &applet : connect to this applet
- NULL : connect using the normal LB algos
- anything else : direct connection to some entity
Since we're interested in detecting the specific case of applets, it's
OK to make use of s->target then.
Also, applets are being isolated from connections, and as such there
will not be any ->connect method available when an applet is running,
so we can get rid of this test as well.
Willy Tarreau [Fri, 22 Nov 2013 23:30:38 +0000 (00:30 +0100)]
MEDIUM: stats: move request argument processing to the final step
At the moment, stats require some preliminary storage just to store
some flags and codes that are parsed very early and used later. In
fact that doesn't make much sense and makes it very hard to allocate
the applet dynamically.
This patch changes this. Now stats_check_uri() only checks for the
validity of the request and the fact that it matches the stats uri.
It's handle_stats() which parses it. It makes more sense because
handle_stats() used to already perform some preliminary processing
such as verifying that POST contents are not missing, etc...
There is only one minor hiccup in doing so : the reqrep rules might
be processed in between. This has been addressed by moving
http_handle_stats() just after stats_check_uri() and setting s->target
at the same time. Now that s->target is totally operational, it's used
to mark the current request as being targetted at the stats, and this
information is used after the request processing to remove the HTTP
analysers and only let the applet handle the request.
Thus we guarantee that the storage for the applet is filled with the
relevant information and not overwritten when we switch to the applet.
Willy Tarreau [Fri, 22 Nov 2013 16:51:09 +0000 (17:51 +0100)]
MAJOR: stats: move the HTTP stats handling to its applet
There is a big trouble with the way POST is handled for the admin
stats page. The POST parameters are extracted from some http-request
rules, and if not round they return zero hoping for being called again
when more data passes. This results in the HTTP analyser being called
several times and all the rules prior to the stats being executed
multiple times as well. That includes rewrite rules.
So instead of doing this, we now move all the processing of the stats
into the stats applet.
That way we just set the stats applet in the HTTP analyser when a stats
request is detected, and the applet takes the time it needs to read the
arguments and respond. We could even imagine improving the applet to
support requests larger than a single buffer.
The code was almost only moved and minimally changed. Several new HTTP
states were added to the stats applet to emit headers, redirects and
to read POST. It was necessary to do this because the headers sent
depend on the parsing of the POST request. In the end it's beneficial
because we removed two stream_int_retnclose() calls.
Willy Tarreau [Fri, 22 Nov 2013 11:25:24 +0000 (12:25 +0100)]
MEDIUM: stats: prepare the HTTP stats I/O handler to support more states
In preparation for moving the POST processing to the applet, we first
add new states to the HTTP I/O handler. Till now st0 was only 0/1 for
start/end. We now replace it with an enum.
Willy Tarreau [Tue, 1 Oct 2013 09:41:55 +0000 (11:41 +0200)]
MINOR: connection: make it easier to emit proxy protocol for unknown addresses
Currently a connection is required on the remote side to emit a proxy
protocol header line. Let's support NULL addresses to emit an UNKNOWN
tag as well.
MINOR: http: prevent smp_fetch_url_{ip,port} from using si->conn
These two fetch methods predate the samples and used to store the
destination address into the server-facing connection's address field
because we had no other place at this time.
This will become problematic with the current connection changes, so
let's fix this.
MEDIUM: peers: don't rely on conn->xprt_ctx anymore
We make the peers code use applet->ptr instead of conn->xprt_ctx to
store the pointer to the current peer. That way it does not depend
on a connection anymore.
This field was used by dumpstats to retrieve a pointer to the current
session, which may already be found from ->owner. With this change,
the stats code doesn't need the connection at all anymore.
We're trying to move the applets out of the struct connection. So
let's remove the dependence on xprt_st and introduce si->applet.st2
to store the missing contextual data instead.
BUG/MEDIUM: pattern: Pattern node has type of "struct pat_idx_elt" in place of "struct eb_node"
The free() function must free the "struct pat_idx_elt".
This bug was introduced by commit ed66c29 (REORG: acl/pattern: extract
pattern matching from the acl file and create pattern.c), no backport
is needed.
Willy Tarreau [Mon, 9 Dec 2013 11:52:13 +0000 (12:52 +0100)]
BUG/MEDIUM: stick-tables: complete the latest fix about store-responses
The commit 37e340c (BUG/MEDIUM: stick: completely remove the unused flag
from the store entries) was incomplete. We also need to ensure that only
the first store-response for a table is applied and that it may coexist
with a possible store-request that was already done on this table.
This patch with the previous one should be backported to 1.4.
Willy Tarreau [Sat, 7 Dec 2013 00:09:04 +0000 (01:09 +0100)]
OPTIM: ebtree: pack the struct eb_node to avoid holes on 64-bit
struct eb_node is 36 bytes on a 64-bit machine. It's thus rounded
up to 40 bytes, and when forming a struct eb32_node, another 4 bytes
are added, rounded up to 48 bytes. We waste 8 bytes of space on 48
bytes because of alignments. It's basically the same with memory
blocks and immediate strings.
By packing the structure, eb32_node is down to 40 bytes. This saves
16 bytes per struct task and 20 bytes per struct stksess, used to
store each stick-table key.