Graham Leggett [Fri, 3 Jul 2020 12:44:53 +0000 (12:44 +0000)]
core: Remove support for the Content-MD5 header, removed in RFC7231.
Functions ap_md5digest() and ap_md5contextTo64() removed, and
ContentDigest directive.
Joe Orton [Thu, 2 Jul 2020 13:31:30 +0000 (13:31 +0000)]
Add additional test suite runs for the prefork & event pool-debug builds:
- run with SSLSessionCache defined (after r1879430)
- run with "SSLProtocol TLSv1.2"
mod_proxy_http: handle async tunneling of Upgrade(d) protocols.
When supported by the MPM (i.e. "event"), provide async callbacks and let
them be scheduled by ap_mpm_register_poll_callback_timeout(), while the
handler returns SUSPENDED.
The new ProxyAsyncDelay directive (if positive) enables async handling,
while ProxyAsyncIdleTimeout determines the timeout applied on both ends
while tunneling.
server/mpm_fdqueue.h;
Rename "remove" field to "pfds" in timer_event_t.
server/mpm/event/event.c:
update_reqevents_from_sense():
New helper to update pfd->reqevents according to the given cs->sense
for CONN_STATE_WRITE_COMPLETION, reusable in process_socket() and
event_resume_suspended().
event_resume_suspended():
Process lingering close if given cs->state = CONN_STATE_LINGER.
Call notify_suspend() before entering CONN_STATE_WRITE_COMPLETION.
event_register_poll_callback_ex():
Don't poll pfds with reqevents == 0.
listener_thread():
Run event_cleanup_poll_callback to both remove the registered pfds
and leave pfds->pool in a consistent state.
Process users callabacks after all PT_USER batons have been collected
in the result pfds loop, otherwise we might race with the callbacks
within the loop if multiple events/sockets concern the same baton, and
crash if pfds->pool is cleared.
modules/proxy/mod_proxy.h:
Rename AP_PROXY_TRANSFER_SHOULD_YIELD to AP_PROXY_TRANSFER_YIELD_PENDING
and add AP_PROXY_TRANSFER_YIELD_MAX_READS.
modules/proxy/mod_proxy_http.c:
modules/proxy/mod_proxy_wstunnel.c:
Removing of reqtimeout filter is now handled by ap_proxy_tunnel_create().
modules/proxy/proxy_util.c:
ap_proxy_transfer_between_connections():
Reorganize loop to break out early.
When AP_PROXY_TRANSFER_YIELD_PENDING, if !ap_filter_should_yield() we
still need to run and check ap_filter_output_pending() since it may
release pending data.
When AP_PROXY_TRANSFER_YIELD_MAX_READS, stop the loop after too much
reads (PROXY_TRANSFER_MAX_READS = 10000) to release the thread and
give the caller a chance to schedule the other direction.
Don't return APR_INCOMPLETE when it comes from an incomplete body
detected by ap_http_filter().
ap_proxy_tunnel_create():
Start with POLLOUT on both directions so that any pending output data
is flushed first.
ap_proxy_tunnel_run():
Remove re-init/clear of the pollset for each call so that the function
is reentrant.
Handle POLLOUT before POLLIN so that we can read in the same pass once
all buffered output data are flushed, using ap_filter_input_pending()
to drain buffered input data.
This is preparatory patch for async websocket tunneling is mod_proxy_http.
Yann Ylavic [Tue, 30 Jun 2020 16:05:56 +0000 (16:05 +0000)]
Follow up to r1877955: don't reuse the connection for mixed C-L / T-E requests
Disable keepalive on the connection if we received both Content-Length and
chunked Transfer-Encoding in the request, to avoid confusion with front
intermediaries and potential further request/response splitting.
This is what we do already for mod_proxy backend connections in the same case.
While at it, replace draft httpbis links with final RFC7230's.
Joe Orton [Tue, 30 Jun 2020 14:24:43 +0000 (14:24 +0000)]
Travis changes:
a) add job to check for empty APLOGNO() macros
b) don't install CPAN stuff if SKIP_TESTING is set
c) don't tail the error_log on test suite failure since it's almost
always irrelevant shutdown stuff and obscures the actual failures
d) use coloured grep output when showing segfaults to make them obvious in the
travis error_log output.
e.g. https://travis-ci.org/github/apache/httpd/jobs/703517978#L2537
Graham Leggett [Mon, 29 Jun 2020 16:21:52 +0000 (16:21 +0000)]
mod_dav: Add method_precondition hook. WebDAV extensions define
conditions that must exist before a WebDAV method can be executed.
This hook allows a WebDAV extension to verify these preconditions.
Graham Leggett [Sat, 27 Jun 2020 23:41:00 +0000 (23:41 +0000)]
"[mod_dav_fs etag handling] should really honor the FileETag setting".
- It now does.
- Add "Digest" to FileETag directive, allowing a strong ETag to be
generated using a file digest.
- Add ap_make_etag_ex() and ap_set_etag_fd() to allow full control over
ETag generation.
- Add concept of "binary notes" to request_rec, allowing packed bit flags
to be added to a request.
- First binary note - AP_REQUEST_STRONG_ETAG - allows modules to force
the ETag to a strong ETag to comply with RFC requirements, such as those
mandated by various WebDAV extensions.
Follow up after Joe's feedback in STATUS:
- If APR_DATE_BAD is returned for Last-Modified, log it at INFO level
(as opposed to trace).
- Remove unnecessary guard for APLOGrtrace1(r).
Yann Ylavic [Fri, 26 Jun 2020 12:51:08 +0000 (12:51 +0000)]
Follow up to r1879080 and r1879137: servlet-normalize r->uri if matched.
If a ProxyPass mapping=servlet matches (in pre_trans hook), update r->uri with
the servlet normalization so that later <Location> or any dir context match
does not have to handle potential path parameters.
Yann Ylavic [Thu, 25 Jun 2020 11:08:33 +0000 (11:08 +0000)]
EVP_PKEY_up_ref(): fix bad reference count locking.
When enabling client authentication for proxy (SSLProxyMachineCertificateFile),
the client certificate callback function ssl_callback_proxy_cert uses another
reference count locking type then one that is used by the caller function when
trying to free the private key afterwards by using EVP_PKEY_free.
This can lead to a race-condition on pkey->references resulting in a double
free error.
On my system, the error occurs sporadically when threaded health checking
(mod_watchdog) forces two threads competing for the client's private key.
For example, see following two backtraces of a coredump where thread 1 and
thread 15 both run into CRYPTO_free(). Actually, the private key should never
be freed during run-time nor should two threads ever enter CRYPTO_free()
concurrently.
(gdb) t 1
[Switching to thread 1 (Thread 0xb2cfbb40 (LWP 16054))]
#0 0xf7f3f329 in __kernel_vsyscall ()
(gdb) bt
#0 0xf7f3f329 in __kernel_vsyscall ()
#1 0xf7cec9e7 in raise () from /lib32/libc.so.6
#2 0xf7cedfb9 in abort () from /lib32/libc.so.6
#3 0xf7d2a14d in ?? () from /lib32/libc.so.6
#4 0xf7d2fd27 in ?? () from /lib32/libc.so.6
#5 0xf7d3047d in ?? () from /lib32/libc.so.6
#6 0x08499c70 in CRYPTO_free (str=0x93376b0) at mem.c:434
#7 0x084cc063 in EVP_PKEY_free (x=0x93376b0) at p_lib.c:406
#8 0x08463917 in ssl3_send_client_certificate (s=0xad21f070) at s3_clnt.c:3475
#9 0x0845d62c in ssl3_connect (s=0xad21f070) at s3_clnt.c:426
#10 0x08484213 in SSL_connect (s=0xad21f070) at ssl_lib.c:1008
#11 0x0846f9c8 in ssl23_get_server_hello (s=0xad21f070) at s23_clnt.c:832
#12 0x0846ea45 in ssl23_connect (s=0xad21f070) at s23_clnt.c:231
#13 0x08484213 in SSL_connect (s=0xad21f070) at ssl_lib.c:1008
#14 0x08261e73 in ssl_io_filter_handshake (filter_ctx=0xb4d3f450) at ssl_engine_io.c:1245
#15 0x08263ba6 in ssl_io_filter_output (f=0xb4d3f480, bb=0xacc079a0) at ssl_engine_io.c:1760
#16 0x080ea2c9 in ap_pass_brigade (next=0xb4d3f480, bb=0xacc079a0) at util_filter.c:590
#17 0x08263b07 in ssl_io_filter_coalesce (f=0xb4d3f468, bb=0xacc079a0) at ssl_engine_io.c:1728
#18 0x080ea2c9 in ap_pass_brigade (next=0xb4d3f468, bb=0xacc079a0) at util_filter.c:590
#19 0x08251658 in hc_send (r=0xacc069b0, out=0x8c25ec8 "GET /hcheck HTTP/1.0\r\nHost: XXX\r\n\r\n", bb=0xacc079a0) at mod_proxy_hcheck.c:664
#20 0x08251eb3 in hc_check_http (baton=0xacc068d8) at mod_proxy_hcheck.c:806
#21 0x08252653 in hc_check (thread=0x8cc6b10, b=0xacc068d8) at mod_proxy_hcheck.c:870
#22 0x08383185 in thread_pool_func (t=0x8cc6b10, param=0x8c245e0) at misc/apr_thread_pool.c:266
#23 0x083baef6 in dummy_worker (opaque=0x8cc6b10) at threadproc/unix/thread.c:142
#24 0xf7ec615f in start_thread () from /lib32/libpthread.so.0
#25 0xf7da862e in clone () from /lib32/libc.so.6
(gdb) t 15
[Switching to thread 15 (Thread 0xb44feb40 (LWP 16049))]
#0 0xf7dd90a5 in _dl_addr () from /lib32/libc.so.6
(gdb) bt
#0 0xf7dd90a5 in _dl_addr () from /lib32/libc.so.6
#1 0xf7db610c in backtrace_symbols_fd () from /lib32/libc.so.6
#2 0xf7cd89ab in ?? () from /lib32/libc.so.6
#3 0xf7d2a148 in ?? () from /lib32/libc.so.6
#4 0xf7d2fd27 in ?? () from /lib32/libc.so.6
#5 0xf7d3047d in ?? () from /lib32/libc.so.6
#6 0x08499c70 in CRYPTO_free (str=0x93376b0) at mem.c:434
#7 0x084cc063 in EVP_PKEY_free (x=0x93376b0) at p_lib.c:406
#8 0x08463917 in ssl3_send_client_certificate (s=0xacf1baa0) at s3_clnt.c:3475
#9 0x0845d62c in ssl3_connect (s=0xacf1baa0) at s3_clnt.c:426
#10 0x08484213 in SSL_connect (s=0xacf1baa0) at ssl_lib.c:1008
#11 0x0846f9c8 in ssl23_get_server_hello (s=0xacf1baa0) at s23_clnt.c:832
#12 0x0846ea45 in ssl23_connect (s=0xacf1baa0) at s23_clnt.c:231
#13 0x08484213 in SSL_connect (s=0xacf1baa0) at ssl_lib.c:1008
#14 0x08261e73 in ssl_io_filter_handshake (filter_ctx=0xb4d37430) at ssl_engine_io.c:1245
#15 0x08263ba6 in ssl_io_filter_output (f=0xb4d37460, bb=0xad101588) at ssl_engine_io.c:1760
#16 0x080ea2c9 in ap_pass_brigade (next=0xb4d37460, bb=0xad101588) at util_filter.c:590
#17 0x08263b07 in ssl_io_filter_coalesce (f=0xb4d37448, bb=0xad101588) at ssl_engine_io.c:1728
#18 0x080ea2c9 in ap_pass_brigade (next=0xb4d37448, bb=0xad101588) at util_filter.c:590
#19 0x08251658 in hc_send (r=0xad100598, out=0x8c25898 "GET /hcheck HTTP/1.0\r\nHost: XXX\r\n\r\n", bb=0xad101588) at mod_proxy_hcheck.c:664
#20 0x08251eb3 in hc_check_http (baton=0xad1004c0) at mod_proxy_hcheck.c:806
#21 0x08252653 in hc_check (thread=0x8cc6ab0, b=0xad1004c0) at mod_proxy_hcheck.c:870
#22 0x08383185 in thread_pool_func (t=0x8cc6ab0, param=0x8c245e0) at misc/apr_thread_pool.c:266
#23 0x083baef6 in dummy_worker (opaque=0x8cc6ab0) at threadproc/unix/thread.c:142
#24 0xf7ec615f in start_thread () from /lib32/libpthread.so.0
#25 0xf7da862e in clone () from /lib32/libc.so.6
Yann Ylavic [Wed, 24 Jun 2020 10:16:06 +0000 (10:16 +0000)]
Follow up to r1879080: replace ProxyUseOriginalURI by mapping=encoded.
Instead of having a separate ProxyUseOriginalURI directive to control pre_ vs
normal translate stage, let's handle this at each ProxyPass level, with the
mapping= parameter.
At pre_translate stage mod_proxy will handle the "encoded" mapping only, and
at translate stage only the others (unless a worker was already elected at
the first stage).
Note that since mapping=servlet needs to happen encoded too, it's defined like:
#define PROXYPASS_MAP_ENCODED 0x08
#define PROXYPASS_MAP_SERVLET 0x18 /* + MAP_ENCODED */
so uch that proxy_trans does the right thing.
Follow up to r1879080: replace ProxyUseOriginalURI by mapping=encoded.
Instead of having a separate ProxyUseOriginalURI directive to control pre_ vs
normal translate stage, let's handle this at each ProxyPass level, with the
mapping= parameter.
At pre_translate stage mod_proxy will handle the "encoded" mapping only, and
at translate stage only the others (unless a worker was already elected at
the first stage).
Note that since mapping=servlet needs to happen encoded too, it's defined like:
#define PROXYPASS_MAP_ENCODED 0x08
#define PROXYPASS_MAP_SERVLET 0x18 /* + MAP_ENCODED */
so that proxy_trans does the right thing.
This allows for simpler and consistent mapping configuration, where the
translate stage depends only on the mapping= parameter.
To implement a fast path (do nothing) when no encoded mapping is configured
at pre_trans stage, or all mappings are encoded at translate stage, two bits
are added to proxy_server_conf (map_encoded_one:1, map_encoded_all:1) and
updated at load time. Thus MINOR is bumped too.
Yann Ylavic [Wed, 24 Jun 2020 07:47:58 +0000 (07:47 +0000)]
Follow up to r1879079, r1879080: change to DONE semantics for pre_trans hooks.
Don't decode r->uri when pre_trans returns DONE instead of OK, which allows to
preserve previous behaviour where decoding was avoided for "ProxyRequests on"
or post_read_request RewriteRule [P] only, but not ProxyPass'ed requests.
This also preserves decoded location walk in most/same cases.
Joe Orton [Tue, 23 Jun 2020 16:39:25 +0000 (16:39 +0000)]
* modules/generators/mod_cgid.c (cgid_handler): Bail immediately with
a 503 response on errors when talking to the daemon. Check the pid
returned is not zero.
Joe Orton [Tue, 23 Jun 2020 07:24:03 +0000 (07:24 +0000)]
Add workaround for IPv6 configuration on non-x86 hosts which
appears to fix the connection failures. Almost certainly a bug
here, not at all sure where.
Yann Ylavic [Mon, 22 Jun 2020 10:37:41 +0000 (10:37 +0000)]
Allow for proxy servlet mapping at pre_translate_name stage.
Provide alias_match_servlet(), the servlet counterpart of alias_match(),
which maps the request URI-path to the ProxyPass alias ignoring path
parameters, while still forwarding them (above the alias).
This is needed to proxy servlet URIs for application handled by Tomcat,
which can then make use of the path/segments parameters.
Yann Ylavic [Mon, 22 Jun 2020 10:36:55 +0000 (10:36 +0000)]
Allow for URI-path pre_translate_name before (and/or instead of) decoding.
Apply minimal normalization (AP_NORMALIZE_DECODE_UNRESERVED) first in
ap_process_request_internal() before running pre_translate_name hooks,
such that the hooks can work with undecoded r->uri.
Only if no hook takes "ownership" of the URI (returning OK), apply
percent decoding for the rest of request handling. Otherwise r->uri remains
encoded meaning that further location/directory/file/if/.. sections (walks)
should that into account.
Since normalization now happens before decoding, we might have to
re-normalize after decoding if "AllowEncodedSlahes on" transformed any
"%2F" sequence to "/", potentially creating new "/./" or "/../" sequences.
Note that for (lookup) subrequests, the path may be relative so we have
to allow for that.
Yann Ylavic [Mon, 22 Jun 2020 10:32:15 +0000 (10:32 +0000)]
Add pre_translate_name hook running before URI-path decoding.
This allows any module to work with un-decoded URI-path (besides
unreserved characters) in r->uri, and eventually to avoid decoding by
returning OK.
The first candidate is mod_proxy (following commit) when
ProxyMappingDecoded is disabled, such that the forwarded URI is
equivalent to the original one.
Yann Ylavic [Mon, 22 Jun 2020 10:29:27 +0000 (10:29 +0000)]
Add ap_normalize_path() to replace ap_getparents() (with options).
include/httpd.h: Declare ap_normalize_path() and flags.
AP_NORMALIZE_ALLOW_RELATIVE:
Don't require that the path be absolute as per RFC 7230.
This is needed for lookup subrequests.
AP_NORMALIZE_NOT_ABOVE_ROOT:
Check that directory traversal ("..") don't go above root, or
initial directory with relative paths.
AP_NORMALIZE_DECODE_UNRESERVED:
Decode unreserved characters (like '.') first since they have
the same semantics encoded and decoded.
AP_NORMALIZE_MERGE_SLASHES:
Merge multiple slahes into a single one.
AP_NORMALIZE_DROP_PARAMETERS:
Ignore path parameters (";foo=bar"). Not used by httpd but since
ap_normalize_path() is taken from mod_jk's jk_servlet_normalize()
it can allow them to use the upstream version now.
server/util.c: Implement ap_normalize_path().
modules/dav/main/util.c: Replace call to ap_getparents() using
ap_normalize_path() with AP_NORMALIZE_DECODE_UNRESERVED flag since
the path comes from an obsolute URL (thus potentially %-encoded).
modules/generators/mod_autoindex.c: Replace call to ap_getparents() using
ap_normalize_path() with AP_NORMALIZE_ALLOW_RELATIVE and
AP_NORMALIZE_NOT_ABOVE_ROOT flags to be consistent with original code.
include/ap_mmn.h: MINOR bump for ap_normalize_path().
Ruediger Pluem [Fri, 19 Jun 2020 11:45:16 +0000 (11:45 +0000)]
* Revert r1878939, r1878938, r1878936, the changes to
modules/http2/h2_request.c and CHANGES of r1878926 and r1878708 as a result of
https://lists.apache.org/thread.html/red499ac4750b88e5943c25abb86434c59dfff4d4f386ffc53742755d%40%3Cdev.httpd.apache.org%3E
and
https://lists.apache.org/thread.html/ra79eee019e2357703b0ea81153458a29817b58ce92e3605949eee1fe%40%3Cdev.httpd.apache.org%3E
Stefan Eissing [Wed, 17 Jun 2020 15:10:29 +0000 (15:10 +0000)]
*) mod_http2: workaround to facilitate use of common internal protocol/method/uri
checks. The module now handles master/secondary connections and has marked
methods according to use.
Joe Orton [Tue, 16 Jun 2020 14:31:26 +0000 (14:31 +0000)]
mod_ldap: Use the LDAP API directly to implement the rebind callback
for modern versions of OpenLDAP, avoiding the overhead of the apr-util
implementation.
* modules/ldap/util_ldap.c:
Define USE_APR_LDAP_REBIND if a modern version of OpenSSL is used.
(uldap_rebind_proc): New function.
(uldap_rebind_init, uldap_rebind_add): Define, using either the
callback or the (bad) APR-util versions.
(uldap_connection_unbind): Clear the rebind pool to remove rebind
references prior to destroying the LDAP *.
Omit for !USE_APR_LDAP_REBIND.
(uldap_connection_init): Use new wrappers, only create the rebind
pool if USE_APR_LDAP_REBIND.
* include/util_ldap.h: Don't include apr_ldap_rebind.h here.
projects / thirdparty / apache / httpd.git / log
