Emeric Brun [Thu, 2 Nov 2017 16:32:43 +0000 (17:32 +0100)]
BUG/MINOR: freq: fix infinite loop on freq_ctr_period.
Using peers or stick table we could update an freq_ctr
using a tick value with the first bit set but this
bit is reserved for lock since multithreading support.
Willy Tarreau [Thu, 2 Nov 2017 16:16:07 +0000 (17:16 +0100)]
BUG/MAJOR: buffers: fix get_buffer_nc() for data at end of buffer
This function incorrectly dealt with the case where data doesn't
wrap but lies at the end of the buffer, resulting in Lukas' reported
data corruption with HTTP/2. No backport is needed, it was introduced
for HTTP/2 in 1.8-dev.
Willy Tarreau [Thu, 2 Nov 2017 14:45:00 +0000 (15:45 +0100)]
BUG/MEDIUM: checks/mux: always enable send-polling after connecting
Before introducing the mux layer, tcp_connect() would poll for sending
to detect the connection establishment. It happens that the health
checks have apparently never explicitly enabled this polling and have
been relying on this implicit one.
Now that there's the mux layer, the conn_stream needs to be enabled
for polling as well and since it's not done in the checks, it's never
done and the check's request doesn't leave the machine, as can be
noticed with http checks.
The solution simply consists in going back to the well-known case
where we enable polling after connecting using cs_want_send() if we
have anything but just a plain connect(). The regular data path is
not affected because the stream interface code automatically computes
the polling needs based on buffer contents.
Willy Tarreau [Thu, 2 Nov 2017 14:14:19 +0000 (15:14 +0100)]
BUG/MEDIUM: h2: don't try to parse incomplete H1 responses
This situation which must not happen does in fact happen when feeding
artificial responses using errorfiles, Lua or an applet. For now it
causes the H1 response parser to loop forever trying to get a more
complete response. Since it cannot progress, let's return an error.
Don't bother testing if len is nonzero, we know it is, as we're in the
"else" part of a if (!len), and testing it confuses clang into thinking
ret may be left uninitialized.
Willy Tarreau [Tue, 31 Oct 2017 22:18:29 +0000 (23:18 +0100)]
[RELEASE] Released version 1.8-rc1
Released version 1.8-rc1 with the following main changes :
- BUG/MEDIUM: server: Allocate tmptrash before using it.
- CONTRIB: trace: add the possibility to place trace calls in the code
- CONTRIB: trace: try to display the function's return value on exit
- CONTRIB: trace: report the base name only for file names
- BUILD: ssl: support OPENSSL_NO_ASYNC #define
- MINOR: ssl: build with recent BoringSSL library
- BUG/MINOR: ssl: OCSP_single_get0_status can return -1
- BUG/MINOR: cli: restore "set ssl tls-key" command
- CLEANUP: cli: remove undocumented "set ssl tls-keys" command
- IMPORT: sha1: import SHA1 functions
- MINOR: sample: add the sha1 converter
- MINOR: sample: add the hex2i converter
- MINOR: stream-int: stop checking for useless connection flags in chk_snd_conn
- MINOR: ssl: don't abort after sending 16kB
- MINOR: connection: move the cleanup of flag CO_FL_WAIT_ROOM
- MINOR: connection: add flag CO_FL_WILL_UPDATE to indicate when updates are granted
- MEDIUM: connection: make use of CO_FL_WILL_UPDATE in conn_sock_shutw()
- MINOR: raw_sock: make use of CO_FL_WILL_UPDATE
- MINOR: ssl_sock: make use of CO_FL_WILL_UPDATE
- BUG/MINOR: checks: Don't forget to release the connection on error case.
- MINOR: buffer: add the buffer input manipulation functions
- BUG/MEDIUM: prevent buffers being overwritten during build_logline() execution
- MEDIUM: cfgparse: post section callback
- MEDIUM: cfgparse: post parsing registration
- MINOR: lua: add uuid to the Class Proxy
- MINOR: hlua: Add regex class
- MINOR: http: Mark the 425 code as "Too Early".
- MEDIUM: ssl: convert CBS (BoringSSL api) usage to neutral code
- MINOR: ssl: support Openssl 1.1.1 early callback for switchctx
- MINOR: ssl: generated certificate is missing in switchctx early callback
- MEDIUM: ssl: Handle early data with OpenSSL 1.1.1
- BUILD: Makefile: disable -Wunused-label
- MINOR: ssl/proto_http: Add keywords to take care of early data.
- BUG/MINOR: lua: const attribute of a string is overridden
- MINOR: ssl: Don't abuse ssl_options.
- MINOR: update proxy-protocol-v2 #define
- MINOR: merge ssl_sock_get calls for log and ppv2
- MINOR: add ALPN information to send-proxy-v2
- MEDIUM: h1: ensure that 1xx, 204 and 304 don't have a payload body
- CLEANUP: shctx: get ride of the shsess_packet{_hdr} structures
- MEDIUM: lists: list_for_each_entry{_safe}_from functions
- REORG: shctx: move lock functions and struct
- MEDIUM: shctx: allow the use of multiple shctx
- REORG: shctx: move ssl functions to ssl_sock.c
- MEDIUM: shctx: separate ssl and shctx
- MINOR: shctx: rename lock functions
- MINOR: h1: store the status code in the H1 message
- BUG/MINOR: spoe: Don't compare engine name and SPOE scope when both are NULL
- BUG/MINOR: spoa: Update pointer on the end of the frame when a reply is encoded
- MINOR: action: Add trk_idx inline function
- MINOR: action: Use trk_idx instead of tcp/http_trk_idx
- MINOR: action: Add a function pointer in act_rule struct to check its validity
- MINOR: action: Add function to check rules using an action ACT_ACTION_TRK_*
- MINOR: action: Add a functions to check http capture rules
- MINOR: action: Factorize checks on rules calling check_ptr if defined
- MINOR: acl: Pass the ACLs as an explicit parameter of build_acl_cond
- MEDIUM: spoe: Add support of ACLS to enable or disable sending of SPOE messages
- MINOR: spoe: Check uniqness of SPOE engine names during config parsing
- MEDIUM: spoe: Parse new "spoe-group" section in SPOE config file
- MEDIUM: spoe/rules: Add "send-spoe-group" action for tcp/http rules
- MINOR: spoe: Move message encoding in its own function
- MINOR: spoe: Add a type to qualify the message list during encoding
- MINOR: spoe: Add a generic function to encode a list of SPOE message
- MEDIUM: spoe/rules: Process "send-spoe-group" action
- BUG/MINOR: dns: Fix CLI keyword declaration
- MAJOR: dns: Refactor the DNS code
- BUG/MINOR: mailers: Fix a memory leak when email alerts are released
- MEDIUM: mailers: Init alerts during conf parsing and refactor their processing
- MINOR: mailers: Use pools to allocate email alerts and its tcpcheck_rules
- MINOR: standard: Add memvprintf function
- MINOR: log: Save alerts and warnings emitted during HAProxy startup
- MINOR: cli: Add "show startup-logs" command
- MINOR: startup: Extend the scope the MODE_STARTING flag
- MINOR: threads: Prepare makefile to link with pthread
- MINOR: threads: Add THREAD_LOCAL macro
- MINOR: threads: Add atomic-ops and plock includes in import dir
- MEDIUM: threads: Add hathreads header file
- MINOR: threads: Add mechanism to register per-thread init/deinit functions
- MINOR: threads: Add nbthread parameter
- MEDIUM: threads: Adds a set of functions to handle sync-point
- MAJOR: threads: Start threads to experiment multithreading
- MINOR: threads: Define the sync-point inside run_poll_loop
- MEDIUM: threads/buffers: Define and register per-thread init/deinit functions
- MEDIUM: threads/chunks: Transform trash chunks in thread-local variables
- MEDIUM: threads/time: Many global variables from time.h are now thread-local
- MEDIUM: threads/logs: Make logs thread-safe
- MEDIUM: threads/pool: Make pool thread-safe by locking all access to a pool
- MAJOR: threads/fd: Make fd stuffs thread-safe
- MINOR: threads/fd: Add a mask of threads allowed to process on each fd in fdtab array
- MEDIUM: threads/fd: Initialize the process mask during the call to fd_insert
- MINOR: threads/fd: Process cached events of FDs depending on the process mask
- MINOR: threads/polling: pollers now handle FDs depending on the process mask
- WIP: SQUASH WITH SYNC POINT
- MAJOR: threads/task: handle multithread on task scheduler
- MEDIUM: threads/signal: Add a lock to make signals thread-safe
- MEDIUM: threads/listeners: Make listeners thread-safe
- MEDIUM: threads/proxy: Add a lock per proxy and atomically update proxy vars
- MEDIUM: threads/server: Make connection list (priv/idle/safe) thread-safe
- MEDIUM: threads/server: Add a lock per server and atomically update server vars
- MINOR: threads/server: Add a lock to deal with insert in updates_servers list
- MEDIUM: threads/lb: Make LB algorithms (lb_*.c) thread-safe
- MEDIUM: threads/stick-tables: handle multithreads on stick tables
- MINOR: threads/sample: Change temp_smp into a thread local variable
- MEDIUM: threads/http: Make http_capture_bad_message thread-safe
- MINOR: threads/regex: Change Regex trash buffer into a thread local variable
- MAJOR: threads/applet: Handle multithreading for applets
- MAJOR: threads/peers: Make peers thread safe
- MAJOR: threads/buffer: Make buffer wait queue thread safe
- MEDIUM: threads/stream: Make streams list thread safe
- MAJOR: threads/ssl: Make SSL part thread-safe
- MEDIUM: threads/queue: Make queues thread-safe
- MAJOR: threads/map: Make acls/maps thread safe
- MEDIUM: threads/freq_ctr: Make the frequency counters thread-safe
- MEDIUM: thread/vars: Make vars thread-safe
- MEDIUM: threads/filters: Add init/deinit callback per thread
- MINOR: threads/filters: Update trace filter to add _per_thread callbacks
- MEDIUM: threads/compression: Make HTTP compression thread-safe
- MEDIUM: threads/lua: Makes the jmpbuf and some other buffers local to the current thread.
- MEDIUM: threads/lua: Add locks around the Lua execution parts.
- MEDIUM: threads/lua: Ensure that the launched tasks runs on the same threads than me
- MEDIUM: threads/lua: Cannot acces to the socket if we try to access from another thread.
- MEDIUM: threads/xref: Convert xref function to a thread safe model
- MEDIUM: threads/tasks: Add lock around notifications
- MEDIUM: thread/spoe: Make the SPOE thread-safe
- MEDIUM: thread/dns: Make DNS thread-safe
- MINOR: threads: Add thread-map config parameter in the global section
- MINOR: threads/checks: Add a lock to protect the pid list used by external checks
- MINOR: threads/checks: Set the task process_mask when a check is executed
- MINOR: threads/mailers: Add a lock to protect queues of email alerts
- MEDIUM: threads/server: Use the server lock to protect health check and cli concurrency
- MINOR: threads: Don't start when device a detection module is used
- BUG/MEDIUM: threads: Run the poll loop on the main thread too
- BUG/MINOR: threads: Add missing THREAD_LOCAL on static here and there
- MAJOR: threads: Offically enable the threads support in HAProxy
- BUG/MAJOR: threads/freq_ctr: fix lock on freq counters.
- BUG/MAJOR: threads/time: Store the time deviation in an 64-bits integer
- BUILD: stick-tables: silence an uninitialized variable warning
- BUG/MINOR: dns: Fix SRV records with the new thread code.
- MINOR: ssl: Remove the global allow-0rtt option.
- CLEANUP: threads: replace the last few 1UL<<tid with tid_bit
- CLEANUP: threads: rename process_mask to thread_mask
- MINOR: h1: add a function to measure the trailers length
- MINOR: threads: add a portable barrier for threads and non-threads
- BUG/MAJOR: threads/freq_ctr: use a memory barrier to detect changes
- BUG/MEDIUM: threads: Initialize the sync-point
- MEDIUM: connection: start to introduce a mux layer between xprt and data
- MINOR: connection: implement alpn registration of muxes
- MINOR: mux: register the pass-through mux for any ALPN string
- MEDIUM: session: use the ALPN token and proxy mode to select the mux
- MINOR: connection: report the major HTTP version from the MUX for logging (fc_http_major)
- MINOR: connection: introduce conn_stream
- MINOR: mux: add more methods to mux_ops
- MINOR: connection: introduce the conn_stream manipulation functions
- MINOR: mux_pt: implement remaining mux_ops methods
- MAJOR: connection : Split struct connection into struct connection and struct conn_stream.
- MINOR: connection: make conn_stream users also check for per-stream error flag
- MINOR: conn_stream: new shutr/w status flags
- MINOR: conn_stream: modify cs_shut{r,w} API to pass the desired mode
- MEDIUM: connection: make conn_sock_shutw() aware of lingering
- MINOR: connection: add cs_close() to close a conn_stream
- MEDIUM: mux_pt: make cs_shutr() / cs_shutw() properly close the connection
- MEDIUM: connection: replace conn_full_close() with cs_close()
- MEDIUM: connection: make mux->detach() release the connection
- MEDIUM: stream: do not forcefully close the client connection anymore
- MEDIUM: checks: exclusively use cs_destroy() to release a connection
- MEDIUM: connection: add a destroy callback
- MINOR: session: release the listener with the session, not the stream
- MEDIUM: session: make use of the connection's destroy callback
- CONTRIB: hpack: implement a reverse huffman table generator for hpack
- MINOR: hpack: implement the HPACK Huffman table decoder
- MINOR: hpack: implement the header tables management
- MINOR: hpack: implement the decoder
- MEDIUM: hpack: implement basic hpack encoding
- MINOR: h2: centralize all HTTP/2 protocol elements and constants
- MINOR: h2: create a very minimalistic h2 mux
- MINOR: h2: expose tune.h2.header-table-size to configure the table size
- MINOR: h2: expose tune.h2.initial-window-size to configure the window size
- MINOR: h2: expose tune.h2.max-concurrent-streams to limit the number of streams
- MINOR: h2: create the h2c struct and allocate its pool
- MINOR: h2: create the h2s struct and the associated pool
- MINOR: h2: handle two extra stream states for errors
- MINOR: h2: add a frame header descriptor for incoming frames
- MEDIUM: h2: allocate and release the h2c context on connection init/end
- MEDIUM: h2: implement basic recv/send/wake functions
- MEDIUM: h2: dynamically allocate the demux buffer on Rx
- MEDIUM: h2: implement the mux buffer allocator
- MINOR: h2: add the connection and stream flags listing the causes for blocking
- MINOR: h2: add function h2s_id() to report a stream's ID
- MINOR: h2: small function to know when the mux is busy
- MINOR: h2: new function h2c_error to mark an error on the connection
- MINOR: h2: new function h2s_error() to mark an error on a stream
- MINOR: h2: add h2_set_frame_size() to update the size in a binary frame
- MINOR: h2: new function h2_peek_frame_hdr() to retrieve a new frame header
- MINOR: h2: add a few functions to retrieve contents from a wrapping buffer
- MINOR: h2: add stream lookup function based on the stream ID
- MINOR: h2: create dummy idle and closed streams
- MINOR: h2: add the function to create a new stream
- MINOR: h2: update the {MUX,DEM}_{M,D}ALLOC flags on buffer availability
- MEDIUM: h2: start to consider the H2_CF_{MUX,DEM}_* flags for polling
- MINOR: h2: also terminate the connection on shutr
- MEDIUM: h2: properly consider all conditions for end of connection
- MEDIUM: h2: wake the connection up for send on pending streams
- MEDIUM: h2: start to implement the frames processing loop
- MINOR: h2: add a function to send a GOAWAY error frame
- MINOR: h2: match the H2 connection preface on init
- MEDIUM: h2: enable connection polling for send when a cs wants to emit
- MEDIUM: h2: enable reading again on the connection if it was blocked on stream buffer full
- MEDIUM: h2: process streams pending for sending
- MINOR: h2: send a real SETTINGS frame based on the configuration
- MEDIUM: h2: detect the presence of the first settings frame
- MINOR: h2: create a stream parser for the demuxer
- MINOR: h2: implement PING frames
- MEDIUM: h2: decode SETTINGS frames and extract relevant settings
- MINOR: h2: lookup the stream during demuxing
- MEDIUM: h2: honor WINDOW_UPDATE frames
- MINOR: h2: implement h2_send_rst_stream() to send RST_STREAM frames
- MINOR: h2: handle CONTINUATION frames
- MEDIUM: h2: partial implementation of h2_detach()
- MEDIUM: h2: unblock a connection when its current stream detaches
- MEDIUM: h2: basic processing of HEADERS frame
- MEDIUM: h2: don't use trash to decode headers!
- MEDIUM: h2: implement the response HEADERS frame to encode the H1 response
- MEDIUM: h2: send the H1 response body as DATA frames
- MEDIUM: h2: skip the response trailers if any
- MEDIUM: h2: properly continue to parse header block when facing a 1xx response
- MEDIUM: h2: send WINDOW_UPDATE frames for connection
- MEDIUM: h2: handle request body in DATA frames
- MINOR: h2: handle RST_STREAM frames
- MEDIUM: h2: send DATA+ES or RST_STREAM on shutw/shutr
- MINOR: h2: use a common function to signal some and all streams.
- MEDIUM: h2: handle GOAWAY frames
- MINOR: h2: centralize the check for the idle streams
- MINOR: h2: centralize the check for the half-closed(remote) streams
- MEDIUM: h2: silently ignore frames higher than last_id after GOAWAY
- MINOR: h2: properly reject PUSH_PROMISE frames coming from the client
- MEDIUM: h2: perform a graceful shutdown on "Connection: close"
- MEDIUM: h2: send a GOAWAY frame when dealing with an empty response
- MEDIUM: h2: apply a timeout to h2 connections
- BUG/MEDIUM: h2: fix incorrect timeout handling on the connection
- MEDIUM: shctx: forbid shctx to read more than expected
- MEDIUM: cache: configuration parsing and initialization
- MEDIUM: cache: store objects in cache
- MEDIUM: cache: deliver objects from cache
Store object in the cache. The cache use an shctx for storage.
It uses an http-response action to store the headers and a filter to
store the body. The http-response action is used in order to allow
modifications by other actions before caching.
MEDIUM: shctx: forbid shctx to read more than expected
Forbid shctx to read more than expected, it allows you to use a greater
value as a len with shctx_row_data_get(), the size of the destination
buffer for example.
Willy Tarreau [Tue, 31 Oct 2017 18:21:06 +0000 (19:21 +0100)]
BUG/MEDIUM: h2: fix incorrect timeout handling on the connection
Previous commit ea3928 (MEDIUM: h2: apply a timeout to h2 connections)
was wrong for two reasons. The first one is that if the client timeout
is not set, it's used as zero, preventing connections from establishing.
The second reason is that if the timeout triggers with active streams
(normally it should not since the task is supposed to be disabled), the
task is removed (h2c->task=NULL), and the last quitting stream might
try to dereference it.
Instead of doing this, we simply not register the task if there's no
timeout (it's useless) and we always control its presence in the streams.
Willy Tarreau [Tue, 31 Oct 2017 09:02:25 +0000 (10:02 +0100)]
MEDIUM: h2: apply a timeout to h2 connections
Till now there was no way to deal with a dead H2 connection. Now each
connection creates a task that wakes up to kill the connection. Its
timeout is constantly refreshed when there's some activity. In case
the timeout triggers, the best effort attempts are made at sending a
clean GOAWAY message before closing and signaling the streams.
The timeout is automatically disabled when there's an active stream on
the connection, and restarted when the last stream finishes. This way
it should not affect long sessions.
Willy Tarreau [Tue, 31 Oct 2017 06:41:55 +0000 (07:41 +0100)]
MEDIUM: h2: send a GOAWAY frame when dealing with an empty response
Given that we're processing data produced by haproxy, we know that the
situations where haproxy doesn't return anything are :
- request timeout with option http-ignore-probes : there's no reason to
hit this since we're creating the stream with the request into it ;
- tcp-request content reject : this definitely means we want to kill the
connection and abort keep-alive and any further processing ;
- using /dev/null as the error file to hide an error
In practice it appears that using the abort on empty response as a hint to
trigger a connection close is very appropriate to continue to give the
control over the connection management. This patch thus tries to send a
GOAWAY frame with the max_id presented as the last stream ID, then sends
an RST_STREAM for the current stream. For the client, this means that the
connection must be shut down immediately after processing the last pending
streams and that the current stream is aborted. This way it's still possible
to force connections to be closed using tcp-request rules.
Willy Tarreau [Mon, 30 Oct 2017 20:54:49 +0000 (21:54 +0100)]
MEDIUM: h2: perform a graceful shutdown on "Connection: close"
After some long brainstorming sessions, it appears that "Connection: close"
seems to be the best signal from the L7 layer to indicate the need to close
the connection. Indeed, in H1 it is only present in very rare cases (eg:
certain unrecoverable errors, some of which could remove it now by the way).
It will also be added when the L7 layer wants to force the connection to
terminate. By default when running in keep-alive mode it is not present.
It's worth mentionning that in H1 with persistent connections, we have sort
of a concurrency-1 mux and this header field is used the same way.
Thus here this patch detects "Connection: close" in response headers and
if seen, sends a GOAWAY frame with the highest possible ID so that the
client knows that it can quit whenever it wants to. If more aggressive
closures are needed in the future, we may decide to advertise the max_id
to abort after the current requests and better honor "http-request deny".
Willy Tarreau [Mon, 30 Oct 2017 17:38:00 +0000 (18:38 +0100)]
MEDIUM: h2: silently ignore frames higher than last_id after GOAWAY
For a graceful shutdown, the specs requries to discard frames with a
stream ID higher than the advertised last_id. (RFC7540#6.8). Well,
finally for now the code is disabled (see last page of #6.8). Some
frames need to be processed anyway to maintain the compression state
and the flow control window state, but we don't have any trivial way
to do this and ignore them at the same time. For the headers it's
the worst case where we can't parse headers frames without coming
from the streams, and we don't want to create such streams as we'd
have to abort them, and aborting would cause errors to flow back.
Possibly that a longterm solution might involve using some dummy
streams and dummy buffers for this and calling the parsers directly.
Willy Tarreau [Mon, 30 Oct 2017 11:03:50 +0000 (12:03 +0100)]
MINOR: h2: centralize the check for the half-closed(remote) streams
RFC7540#5.1 is pretty clear : "any frame other than WINDOW_UPDATE,
PRIORITY, or RST_STREAM in this state MUST be treated as a connection
error of type STREAM_CLOSED". Instead of dealing with this for each
and every frame type, let's do it once for all in the main demux loop.
Willy Tarreau [Mon, 30 Oct 2017 10:46:49 +0000 (11:46 +0100)]
MINOR: h2: centralize the check for the idle streams
RFC7540#5.1 is pretty clear : "any frame other than HEADERS or PRIORITY
in this state MUST be treated as a connection error". Instead of dealing
with this for each and every frame type, let's do it once for all in the
main demux loop.
Willy Tarreau [Sun, 29 Oct 2017 23:28:29 +0000 (00:28 +0100)]
MEDIUM: h2: handle GOAWAY frames
The ID is respected, and only IDs greater than the advertised last_id
are woken up, with a CS_FL_ERROR flag to signal that the stream is
aborted. This is necessary for a browser to abort a download or to
reject a bad response that affects the connection's state.
Willy Tarreau [Sun, 29 Oct 2017 23:26:54 +0000 (00:26 +0100)]
MINOR: h2: use a common function to signal some and all streams.
Let's replace h2_wake_all_streams() with h2_wake_some_streams(), to
support signaling only streams by their ID (for GOAWAY frames) and
to pass the flags to add on the conn_stream.
Willy Tarreau [Sun, 29 Oct 2017 21:00:09 +0000 (22:00 +0100)]
MEDIUM: h2: send DATA+ES or RST_STREAM on shutw/shutr
When a stream sends a shutw, we send an empty DATA frame with the ES
flag set, except if no HEADERS were sent, in which case we rather send
RST_STREAM. On shutr(1) to abort a request, an RST_STREAM frame is sent
if the stream is OPEN and the stream is closed. Care is taken to switch
the stream's state accordingly and to avoid sending an ES bit again or
another RST once already done.
Willy Tarreau [Thu, 26 Oct 2017 17:40:35 +0000 (19:40 +0200)]
MEDIUM: h2: handle request body in DATA frames
Data frames are received and transmitted. The per-connection and
per-stream amount of data to ACK is automatically updated. Each
DATA frame is ACKed because usually the downstream link is large
and the upstream one is small, so it seems better to waste a few
bytes every few kilobytes to maintain a low ACK latency and help
the sender keep the link busy. The connection's ACK however is
sent at the end of the demux loop and at the beginning of the mux
loop so that a single aggregated one is emitted (connection
windows tend to be much larger than stream windows).
A future improvement would consist in sending a single ACK for
multiple subsequent DATA frames of the same stream (possibly
interleaved with window updates frames), but this is much trickier
as it also requires to remember the ID of the stream for which
DATA frames have to be sent.
Ideally in the near future we should chunk-encode the body sent
to HTTP/1 when there's no content length and when the request is
not a CONNECT. It's just uncertain whether it's the best option
or not for now.
Willy Tarreau [Thu, 26 Oct 2017 14:55:59 +0000 (16:55 +0200)]
MEDIUM: h2: send WINDOW_UPDATE frames for connection
When it is detected that the number of received bytes is > 0 on the
connection at the end of the demux call or before starting to process
pending output data, an attempt is made at sending a WINDOW UPDATE on
the connection. In case of failure, it's attempted later.
Willy Tarreau [Tue, 31 Oct 2017 16:15:59 +0000 (17:15 +0100)]
MEDIUM: h2: skip the response trailers if any
For now we don't build a HEADERS frame with them, but at least we remove
them from the response so that the L7 chunk parser inside isn't blocked
on these (often two) remaining bytes that don't want to leave the buffer.
It also ensures that trailers delivered progressively will correctly be
skipped.
Willy Tarreau [Thu, 19 Oct 2017 09:16:37 +0000 (11:16 +0200)]
MEDIUM: h2: send the H1 response body as DATA frames
The H1 response data are processed (either following content-length or
chunks) and emitted as H2 DATA frames. In the case of content-length,
the maximum size permitted by the mux buffer, the max frame size, the
connection's window and the stream's window it used to determine the
frame size. For chunked encoding, the same limitation applies, but in
addition, each chunk leads to a distinct frame. This could be improved
in the future to aggregate chunks into larger frames.
Streams blocked on the connection's flow control subscribe to the
connection's fctl_list to be woken up when the window opens again.
Streams blocked on their own flow control don't subscribe to anything,
they just sit waiting for window update frames to reopen the window.
The connection-close mode (without content-length) partially works thanks
to the fact that the SHUTW event leads to a close of the stream. In
practice an empty DATA frame should be sent in this case though.
Willy Tarreau [Tue, 17 Oct 2017 17:58:20 +0000 (19:58 +0200)]
MEDIUM: h2: implement the response HEADERS frame to encode the H1 response
This calls the h1 response parser and feeds the output through the hpack
encoder to produce stateless HPACK bytecode into an output chunk. For now
it's a bit naive but reasonably efficient.
The HPACK encoder relies on hpack_encode_header() so that the most common
response header fields are encoded based on the static header table. The
forbidden header field names (connection, proxy-connection, upgrade,
transfer-encoding, keep-alive) are dropped before calling the hpack
encoder.
A new flag (H2_CF_HEADERS_SENT) is set once such a frame is emitted. It
will be used to know if we can send an empty DATA+ES frame to use as a
shutdown() signal or if we have to use RST_STREAM.
The trash is already used by the hpack layer and for Huffman decoding,
it's unsafe to use here as a buffer and results in corrupted data. Use
a safely allocated trash instead.
Willy Tarreau [Fri, 13 Oct 2017 17:23:14 +0000 (19:23 +0200)]
MEDIUM: h2: basic processing of HEADERS frame
This takes care of creating a new h2s and a new conn_stream when a
HEADERS frame arrives. The recv() callback from the data layer is then
called to extract the frame into the stream's buffer. It is verified
that the stream ID is strictly greater than the known max stream ID.
And the last_id is updated if the current request is properly converted.
The streams are created in open or half-closed(remote) states.
For now there are some limitations :
- frames without END_HEADERS are rejected (CONTINUATION not supported
yet, will require some more changes so that the stream processor
checks the H2 frame header by itself and steals the frames from the
connection)
- padding/stream_dep/priority are currently ignored
- limited error handling, could be improved
But at least the request is properly decoded, transcoded and processed.
Willy Tarreau [Mon, 30 Oct 2017 14:44:59 +0000 (15:44 +0100)]
MEDIUM: h2: unblock a connection when its current stream detaches
If a stream is killed for whatever reason and it happens to be the one
currently blocking the connection, we must unblock the connection and
enable polling again so that it can attempt to make progress. This may
happen for example on upload timeout, where the demux is blocked due to
a full stream buffer, and the stream dies on server timeout and quits.
Willy Tarreau [Mon, 16 Oct 2017 16:11:19 +0000 (18:11 +0200)]
MEDIUM: h2: partial implementation of h2_detach()
This does the very minimum required to release a stream and/or a connection
upon the stream's request. The only thing is that it doesn't kill the
connection unless it's already closed or in error or the stream ID reached
the one specified in GOAWAY frame. We're supposed to arm a timer to close
after some idle timeout but it's not done.
Willy Tarreau [Tue, 17 Oct 2017 06:19:21 +0000 (08:19 +0200)]
MINOR: h2: handle CONTINUATION frames
For now we have nowhere to store partial header frames so we can't
handle CONTINUATION frames and we must reject them. In this case we
respond with a stream error of type INTERNAL_ERROR.
Willy Tarreau [Tue, 17 Oct 2017 06:10:17 +0000 (08:10 +0200)]
MINOR: h2: implement h2_send_rst_stream() to send RST_STREAM frames
This one sends an RST_STREAM for a given stream, using the current
demux stream ID. It's also used to send RST_STREAM for streams which
have lost their CS part (ie were aborted).
Now they really increase the window size of connections and streams.
If a stream was not queued but requested to send, it means it was
flow-controlled so it's added again into the connection's send list.
MEDIUM: h2: decode SETTINGS frames and extract relevant settings
The INITIAL_WINDOW_SIZE and MAX_FRAME_SIZE settings are now extracted
from the settings frame, assigned to the connection, and attempted to
be propagated to all existing streams as per the specification. In
practice clients rarely update the settings after sending the first
stream, so the propagation will rarely be used. The ACK is properly
sent after the frame is completely parsed.
Willy Tarreau [Tue, 10 Oct 2017 13:56:59 +0000 (15:56 +0200)]
MINOR: h2: create a stream parser for the demuxer
The function h2_process_demux() now tries to parse the incoming bytes
to process as many streams as possible. For now it does nothing but
dropping all incoming frames.
Willy Tarreau [Tue, 10 Oct 2017 13:16:55 +0000 (15:16 +0200)]
MEDIUM: h2: detect the presence of the first settings frame
Instead of doing a special processing of the first SETTINGS frame, we
simply parse its header, check that it matches the expected frame type
and flags (ie no ACK), and switch to FRAME_P to parse it as any regular
frame. The regular frame parser will take care of decoding it.
MINOR: h2: send a real SETTINGS frame based on the configuration
An initial settings frame is emitted upon receipt of the connection
preface, which takes care of configured values. These settings are
only emitted when they differ from the protocol's default value :
- header_table_size (defaults to 4096)
- initial_window_size (defaults to 65535)
- max_concurrent_streams (defaults to unlimited)
- max_frame_size (defaults to 16384)
The max frame size is a copy of tune.bufsize. Clients will most often
reject values lower than 16384 and currently there's no trivial way to
check if H2 is going to be used at boot time.
Willy Tarreau [Tue, 17 Oct 2017 08:57:04 +0000 (10:57 +0200)]
MEDIUM: h2: process streams pending for sending
The send() callback calls h2_process_mux() which iterates over the list
of flow controlled streams first, then streams waiting for room in the
send_list. If a stream from the send_list ends up being flow controlled,
it is then moved to the fctl_list. This way we can maintain the most
accurate fairness by ensuring that flows are always processed in order
of arrival except when they're blocked by flow control, in which case
only the other ones may pass in front of them.
It's a bit tricky as we want to remove a stream from the active lists
if it doesn't block (ie it has no reason for staying there).
Willy Tarreau [Mon, 30 Oct 2017 14:38:23 +0000 (15:38 +0100)]
MEDIUM: h2: enable reading again on the connection if it was blocked on stream buffer full
If the polling update function is called with RD_ENA while H2_CF_DEM_SFULL
indicates the demux had to block on a stream buffer full condition, we can
remove the flag and re-enable polling for receiving because this is the
indication that a consumer stream has made some room in the buffer. Probably
that we should improve this to ensure that h2s->id == h2c->dsi and avoid
trying to receive multiple times in a row for the wrong stream.
Willy Tarreau [Tue, 17 Oct 2017 08:26:19 +0000 (10:26 +0200)]
MEDIUM: h2: enable connection polling for send when a cs wants to emit
A conn_stream indicates its intent to send by setting the WR_ENA flag
and calling mux->update_poll(). There's no synchronous write so the only
way to emit a response from a stream is to proceed this way. The sender
h2s is then queued into the h2c's send_list if it was not yet queued.
Once the connection is ready, it will enter its send() callback to visit
writers, calling their data->send_cb() callback to complete the operation
using mux->snd_buf().
Also we enable polling if the mux contains data and wasn't enabled. This
may happen just after a response has been transmitted using chk_snd().
It likely is incomplete for now and should probably be refined.
MINOR: h2: match the H2 connection preface on init
The H2 preface is properly detected to switch to the settings state.
It's important to note that for now we don't send out settings frame
so the operation is not complete yet.
Willy Tarreau [Tue, 16 May 2017 19:51:05 +0000 (21:51 +0200)]
MINOR: h2: add a function to send a GOAWAY error frame
For now it's only used to report immediate errors by announcing the
highest known stream-id on the mux's error path. The function may be
used both while processing a stream or directly in relation with the
connection. The wake() callback will automatically ask for send access
if an error is reported. The function should be usable for graceful
shutdowns as well by simply setting h2c->last_sid to the highest
acceptable stream-id (2^31-1) prior to calling the function.
A connection flag (H2_CF_GOAWAY_SENT) is set once the frame was
successfully sent. It will be usable to detect when it's safe to
close the connection.
Another flag (H2_CF_GOAWAY_FAILED) is set in case of unrecoverable
error while trying to send. It will also be used to know when it's safe
to close the connection.
Willy Tarreau [Mon, 9 Oct 2017 14:21:43 +0000 (16:21 +0200)]
MEDIUM: h2: start to implement the frames processing loop
The rcv_buf() callback now calls h2_process_demux() after an recv() call
leaving some data in the buffer, and the snd_buf() callback calls
h2_process_mux() to try to process pending data from streams.
Willy Tarreau [Tue, 17 Oct 2017 13:30:07 +0000 (15:30 +0200)]
MEDIUM: h2: wake the connection up for send on pending streams
If some streams were blocked on flow control and the connection's
window was recently opened, or if some streams are waiting while
no block flag remains, we immediately want to try to send again.
This can happen if a recv() for a stream wants to send after the
send() loop has already been processed.
Willy Tarreau [Tue, 31 Oct 2017 05:59:15 +0000 (06:59 +0100)]
MEDIUM: h2: properly consider all conditions for end of connection
During h2_wake(), there are various situations that can lead to the
connection being closed :
- low-level connection error
- read0 received
- fatal error (ERROR2)
- failed to emit a GOAWAY
- empty stream list with max_id >= last_sid
In such cases, all streams are notified and we have to wait for all
streams to leave while doing nothing, or if the last stream is gone,
we can simply terminate the connection.
It's important to do this test there again because an error might arise
while trying to send a pending GOAWAY after the last stream for example,
thus there's possibly no way to get notified of a closing stream.
Willy Tarreau [Mon, 9 Oct 2017 14:47:04 +0000 (16:47 +0200)]
MINOR: h2: also terminate the connection on shutr
It happens that an H2 mux is totally unusable once the client has shut,
so we must consider this situation equivalent to the connection error,
and let the possible streams drain their data if needed then stop.
Willy Tarreau [Mon, 9 Oct 2017 13:14:19 +0000 (15:14 +0200)]
MEDIUM: h2: start to consider the H2_CF_{MUX,DEM}_* flags for polling
Now we start to set the flags to indicate that the response buffer is
being awaited or that it is full, it makes it possible to centralize a
little bit the polling management into the wake() callback.
In case of error, we wake all the streams up so that they are aware of
the nature of the event and are able to detach if needed.
Willy Tarreau [Tue, 16 May 2017 13:20:39 +0000 (15:20 +0200)]
MINOR: h2: create dummy idle and closed streams
It will be more convenient to always manipulate existing streams than
null pointers. Here we create one idle stream and one closed stream.
The idea is that we can easily point any stream to one of these states
in order to merge maintenance operations.
Willy Tarreau [Tue, 10 Oct 2017 15:10:03 +0000 (17:10 +0200)]
MINOR: h2: add a few functions to retrieve contents from a wrapping buffer
Functions h2_get_buf_n{16,32,64}() and h2_get_buf_bytes() respectively
extract a network-ordered 16/32/64 bit value from a possibly wrapping
buffer, or any arbitrary size. They're convenient to retrieve a PING
payload or to parse SETTINGS frames. Since they copy one byte at a time,
they will be less efficient than a memcpy-based implementation on large
blocks.
MINOR: h2: new function h2_peek_frame_hdr() to retrieve a new frame header
This function extracts the next frame header but doesn't consume it.
This will allow to detect a stream-id change and to perform a yielding
window update without losing information. The result is stored into a
temporary frame descriptor. We could also store the next frame header
into the connection but parsing the header again is much cheaper than
wasting bytes in the connection for a rare use case.
A function (h2_skip_frame_hdr()) is also provided to skip the parsed
header (always 9 bytes) and another one (h2_get_frame_hdr()) to do both
at once.
MINOR: h2: add h2_set_frame_size() to update the size in a binary frame
This function is called after preparing a frame, in order to update the
frame's size in the frame header. It takes the frame payload length in
argument.
It simply writes a 24-bit frame size into a buffer, making use of the
net_helper functions which try to optimize per platform (this is a
frequently used operation).
Willy Tarreau [Tue, 17 Oct 2017 06:03:59 +0000 (08:03 +0200)]
MINOR: h2: new function h2s_error() to mark an error on a stream
This one will store the error into the stream's errcode if it's neither
idle nor closed (since these ones are read-only) and switch its state to
H2_SS_ERROR. If a conn_stream is attached, it will be flagged with
CS_FL_ERROR.
MINOR: h2: small function to know when the mux is busy
A mux is busy when any stream id >= 0 is currently being handled
and the current stream's id doesn't match. When no stream is
involved (ie: demuxer), stream 0 is considered. This will be
necessary to know when it's possible to send frames.
MINOR: h2: add the connection and stream flags listing the causes for blocking
A demux may be prevented from receiving for the following reasons :
- no receive buffer could be allocated
- the receive buffer is full
- a response is needed and the mux is currently being used by a stream
- a response is needed and some room could not be found in the mux
buffer (either full or waiting for allocation)
- the stream buffer is waiting for allocation
- the stream buffer is full
A mux may stop accepting data for the following reasons :
- the buffer could not be allocated
- the buffer is full
A stream may stop sending data to a mux for the following reaons :
- the mux is busy processing another stream
- the mux buffer lacks room (full or not allocated)
- the mux's flow control prevents from sending
- the stream's flow control prevents from sending
All these conditions were turned into flags for use by the respective
places.
The idea is that we may need a mux buffer for anything, ranging from
receiving to sending traffic. For now it's unclear where exactly the
calls will be placed so let's block both send and recv when a buffer
is missing, and re-enable both of them at the end. This will have to
be changed later.
MEDIUM: h2: dynamically allocate the demux buffer on Rx
This patch implements a very basic Rx buffer management. The mux needs
an rx buffer to decode the connection's stream. If this buffer it
available upon Rx events, we fill it with whatever input data are
available. Otherwise we try to allocate it and subscribe to the buffer
wait queue in case of failure. In such a situation, a function
"h2_dbuf_available()" will be called once a buffer may be allocated.
The buffer is released if it's still empty after recv().
MEDIUM: h2: allocate and release the h2c context on connection init/end
The connection's h2c context is now allocated and initialized on mux
initialization, and released on mux destruction. Note that for now the
release() code is never called.
Willy Tarreau [Mon, 16 Oct 2017 16:34:34 +0000 (18:34 +0200)]
MINOR: h2: handle two extra stream states for errors
We need to deal with stream error notifications (RST_STREAM) as well as
internal reporting. The problem is that we don't know in which order
this will be done so we can't unilaterally decide to deallocate the
stream. In order to help, we add two extra stream states, H2_SS_ERROR
and H2_SS_RESET. The former mentions that the stream has an error pending
and the latter indicates that the error was already sent and that the
stream is now closed. It's equivalent to H2_SS_CLOSED except that in this
state we'll avoid sending new RST_STREAM as per RFC7540#5.4.2.
With this it will be possible to only detach or deallocate the h2s once
the stream is closed.
Willy Tarreau [Wed, 11 Oct 2017 05:57:07 +0000 (07:57 +0200)]
MINOR: h2: create the h2s struct and the associated pool
This describes an HTTP/2 stream with its relation to the connection
and to the conn_stream on the other side.
For now we also allocate request and response state for HTTP/1 because
the internal HTTP representation is HTTP/1 at the moment. Later this
should evolve towards a version-agnostic representation and this H1
message state will disappear.
It's important to consider that the streams are necessarily polarized
depending on h2c : if the connection is incoming, streams initiated by
the connection receive requests and send responses. Otherwise it's the
other way around. Such information is known during the connection
instanciation by h2c_frt_init() and will normally be reflected in the
stream ID (odd=demux from client, even=demux from server). The initial
H2_CS_PREFACE state will also depend on the direction. The current h2c
state machine doesn't allow for outgoing connections as it uses a single
state for both (rx state only). It should be the demux state only.
MINOR: h2: create the h2c struct and allocate its pool
The h2c struct describes an H2 connection context and is assigned as the
mux's context. It has its own pool, allocated at boot time and released
after deinit().
For now it only supports literals and a bit of static header table
references for the 9 most common header field names (date, server,
content-type, content-length, last-modified, accept-ranges, etag,
cache-control, location).
A previous incarnation of this commit used to strip the forbidden H2
header names (connection, proxy-connection, upgrade, transfer-encoding,
keep-alive) but this is no longer the case as this filtering is irrelevant
to HPACK encoding and is specific to H2, so this will have to be done by
the caller.
It's quite not optimal but works fine enough to prepare some valid and
partially compressed responses during development.
Willy Tarreau [Tue, 30 May 2017 17:09:44 +0000 (19:09 +0200)]
MINOR: hpack: implement the decoder
The decoder is now fully functional. It makes use of the dynamic header
table. Dynamic header table size updates are currently ignored, as our
initially advertised value is the highest we support. Strictly speaking,
the impact is that a client referencing a header field after such an
update wouldn't observe an error instead of the connection being dropped
if it was implemented.
Decoded header fields are copied into a target buffer in HTTP/1 format
using HTTP/1.1 as the version. The Host header field is automatically
appended if a ":authority" header field is present.
All decoded header fields can be displayed if the file is compiled with
DEBUG_HPACK.
Willy Tarreau [Tue, 30 May 2017 16:46:58 +0000 (18:46 +0200)]
MINOR: hpack: implement the header tables management
This code deals with header insertion, retrieval and eviction, as well
as with dynamic header table defragmentation. It is functional for use
as a decoder and was heavily tested in this context. There's still some
room for optimization (eg: the defragmentation code currently does it
in place using a memcpy).
Also for now the dynamic header table is allocated using malloc() while
a pool needs to be created instead.
This code was mostly imported from https://github.com/wtarreau/http2-exp
with "hpack_" prepended in front of most names to avoid risks of conflicts.
Some small cleanups and renamings were applied during the import. This
version must be considered more recent.
Some HPACK error codes were placed here (HPACK_ERR_*), not exactly because
they're needed by the decoder but they'll be needed by all callers. Maybe
a different location should be found.
Willy Tarreau [Tue, 30 May 2017 15:22:18 +0000 (17:22 +0200)]
MINOR: hpack: implement the HPACK Huffman table decoder
The code was borrowed from the HPACK experimental implementations
available here :
https://github.com/wtarreau/http2-exp
It contains the Huffman table as specified in RFC7541 Appendix B, and a
set of reverse tables used to decode a Huffman byte stream, and produced
by contrib/h2/gen-rht. The encoder is not finalized, it doesn't emit the
byte stream but this is not needed for now.
Willy Tarreau [Fri, 19 May 2017 16:14:51 +0000 (18:14 +0200)]
CONTRIB: hpack: implement a reverse huffman table generator for hpack
This one was built by studying the HPACK Huffman table (RFC7541
appendix B). It creates 5 small tables (4*512 bytes, 1*64 bytes) to
map one byte at a time from the input stream based on the following
observations :
* rht_bit31_24[256] is indexed on bits 31..24 when < 0xfe
* rht_bit24_17[256] is indexed on bits 24..17 when 31..24 >= 0xfe
* rht_bit15_11_fe[32] is indexed on bits 15..11 when 24..17 == 0xfe
* rht_bit15_8[256] is indexed on bits 15..8 when 24..17 == 0xff
* rht_bit11_4[256] is indexed on bits 11..4 when 15..8 == 0xff
* when 11..4 == 0xff, 3..2 provide the following mapping :
* 00 => 0x0a, 01 => 0x0d, 10 => 0x16, 11 => EOS
Willy Tarreau [Sun, 8 Oct 2017 09:26:30 +0000 (11:26 +0200)]
MEDIUM: session: make use of the connection's destroy callback
Now we don't remove the session when a stream dies, instead we
detach the stream and let the mux decide to release the connection
and call session_free() instead.
Willy Tarreau [Wed, 18 Oct 2017 13:01:14 +0000 (15:01 +0200)]
MINOR: session: release the listener with the session, not the stream
Since multiple streams can share one session attached to one listener,
the listener_release() call must be done in session_free() and not in
stream_free(), otherwise we end up with a negative count in H2.
Willy Tarreau [Sun, 8 Oct 2017 09:16:46 +0000 (11:16 +0200)]
MEDIUM: connection: add a destroy callback
This callback will be used to release upper layers when a mux is in
use. Given that the mux can be asynchronously deleted, we need a way
to release the extra information such as the session.
This callback will be called directly by the mux upon releasing
everything and before the connection itself is released, so that
the callee can find its information inside the connection if needed.
The way it currently works is not perfect, and most likely this should
instead become a mux release callback, but for now we have no easy way
to add mux-specific stuff, and since there's one mux per connection,
it works fine this way.
Willy Tarreau [Sun, 8 Oct 2017 09:33:44 +0000 (11:33 +0200)]
MEDIUM: stream: do not forcefully close the client connection anymore
Now that the mux will take care of closing the client connection at the
right moment, we don't need to close the client connection anymore, and
we just need to close the conn_stream.