Stefan Eissing [Mon, 21 Mar 2022 08:19:28 +0000 (08:19 +0000)]
*) mod_http2: use pollset only for main connection and wakeups
for events on streams. Provide streams in INPUT pipe when
needed and supported on the platform.
Stefan Eissing [Fri, 18 Mar 2022 09:52:52 +0000 (09:52 +0000)]
*) core: adding a new hook and method to the API:
create_secondary_connection and ap_create_secondary_connection()
to setup connections related to a "master" one, as used in
the HTTP/2 protocol implementation.
*) mod_http2: using the new API calls to get rid of knowledge
about how the core handles conn_rec specifics.
Improvements in pollset stream handling to use less sets.
Using atomic read/writes instead of volatiles now.
Keeping a reserve of "transit" pools and bucket_allocs for
use on secondary connections to avoid repeated setup/teardowns.
Stefan Eissing [Tue, 15 Mar 2022 09:38:10 +0000 (09:38 +0000)]
*) mod_http2: disabling use of apr_pollsets for secondary
connections. Measurements show that their use leads to
up to 30% performance drop on busy scenarios.
Detailed numbers in the comment where this is disabled.
Yann Ylavic [Mon, 7 Mar 2022 13:36:10 +0000 (13:36 +0000)]
core: Make sure and check that LimitXMLRequestBody fits in system memory.
LimitXMLRequestBody can not exceed the size needed to ap_escape_html2() the
body without failing to allocate memory, so enforce this at load time based
on APR_SIZE_MAX, and make sure that ap_escape_html2() is within the bounds.
Document the limits for LimitXMLRequestBody in our docs.
Yann Ylavic [Tue, 1 Mar 2022 13:26:03 +0000 (13:26 +0000)]
mod_rewrite: URI-to-filename rewrites to transparently handle proxy mappings.
Since mod_rewrite works on r->filename and mod_proxy's mapping=servlet|decoded
sets its "proxy:" URL there at pre_translate_name stage (i.e. before
mod_rewrite's translate_name hook), users have to match the full proxy URL in
their RewriteRules to handle proxy mappings, which is not very friendly nor
consistent with how proxy non-mapping requests have to be matched.
Let's use r->filename = r->uri in hook_uri2file() for pre_trans'ed reverse
proxy requests, and restore r->filename to its original value if the request
was finally DECLINED (like in hook_fixup).
But if a proxy mapping gets rewritten to a non-proxy request, clear any
proxy specific r->proxyreq or r->handler so that processing continues
accordingly.
ap_lua_init_mutex() is not about thead only . It also calls
apr_global_mutex_child_init(). So I see no good reason to skip this hook
if !APR_HAS_THREADS.
Some #if APR_HAS_THREADS are also already in place in
ap_lua_init_mutex() anyway.
Stefan Eissing [Fri, 25 Feb 2022 13:18:51 +0000 (13:18 +0000)]
*) mod_watchdog: add assertions to cleanup code
*) core/mpm_preform: do not invoke the fancy new child_stopping/stopped
hooks when invoked from a signal handler. This is a stopgap to some
strange behaviour in need of some deeper insight.
Stefan Eissing [Fri, 25 Feb 2022 09:58:48 +0000 (09:58 +0000)]
*) test: assume that modules which do not appear in our list of DSO_MODULES
or MPM_MODULES from configure are statically linked and do not need a load.
Stefan Eissing [Thu, 24 Feb 2022 13:20:19 +0000 (13:20 +0000)]
*) mod_watchdog: do not call a watchdog instance for
AP_WATCHDOG_STATE_STOPPING outside its thread, as
watchdog instances are not prepared to be invoked
concurrently.
Stefan Eissing [Thu, 24 Feb 2022 11:56:01 +0000 (11:56 +0000)]
*) mod_watchdog: use the `child_stopping` and `child_stopped` hooks
to shutdown workers before pool destruction releases global
resources and libraries.
Stefan Eissing [Thu, 24 Feb 2022 11:53:53 +0000 (11:53 +0000)]
* core/mpm: add hook 'child_stopped` that gets called when the MPM has
stopped all processing in a child process. This is when all running
threads shall be stopped and joined.
[Stefan Eissing]
Ruediger Pluem [Wed, 16 Feb 2022 12:35:36 +0000 (12:35 +0000)]
* Change the logic to choose the maximum of both timeouts (front end socket,
backend socket) instead of the minimum as backend timeouts can be
configured more selectively (per worker if needed) as front end timeouts
and typically the backend timeouts reflect the application requirements
better.
Stefan Eissing [Thu, 10 Feb 2022 10:59:08 +0000 (10:59 +0000)]
*) mod_http2: :scheme pseudo-header values, not matching the
connection scheme, are forwarded via absolute uris to the
http protocol processing to preserve semantics of the request.
Checks on combinations of pseudo-headers values/absence
have been added as described in RFC 7540.
Fixes <https://github.com/icing/mod_h2/issues/230>.
Graham Leggett [Wed, 9 Feb 2022 18:23:23 +0000 (18:23 +0000)]
ab: Fix the detection for when the server performed a legitimate
connection close as per RFC7230 6.3.1. We must check whedther the
connection was previously kept alive, and not whether the current
closed request is keepalive.
Stefan Eissing [Tue, 8 Feb 2022 13:38:49 +0000 (13:38 +0000)]
*) mod_http2: when a h2 request carries a ':scheme' pseudoheader,
it gives a 400 response if the scheme does not match the
connection. Fixes <https://github.com/icing/mod_h2/issues/230>.
Yann Ylavic [Tue, 8 Feb 2022 13:07:32 +0000 (13:07 +0000)]
mod_http2: Fix possible beam bucket double free from session destroy.
When the session pool is destroyed, so is the beam's pool so we don't
want to run the beam cleanup twice.
ASan is reporting something like this (APR_POOL_DEBUG):
=================================================================
==81201==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000080ce8 at pc 0x7fdc78962cc9 bp 0x7fdc731ff4f0 sp 0x7fdc731ff4e8
READ of size 8 at 0x603000080ce8 thread T11
#0 0x7fdc78962cc8 in recv_buffer_cleanup ~httpd/modules/http2/h2_bucket_beam.c:279
#1 0x7fdc78962fdc in beam_cleanup ~httpd/modules/http2/h2_bucket_beam.c:306
#2 0x7fdc7896300c in beam_pool_cleanup ~httpd/modules/http2/h2_bucket_beam.c:313
#3 0x7fdc7c5a8239 in run_cleanups memory/unix/apr_pools.c:2689
#4 0x7fdc7c5a50f9 in pool_clear_debug memory/unix/apr_pools.c:1867
#5 0x7fdc7c5a562e in pool_destroy_debug memory/unix/apr_pools.c:1965
#6 0x7fdc7c5a5179 in pool_clear_debug memory/unix/apr_pools.c:1880
#7 0x7fdc7c5a562e in pool_destroy_debug memory/unix/apr_pools.c:1965
#8 0x7fdc7c5a5179 in pool_clear_debug memory/unix/apr_pools.c:1880
#9 0x7fdc7c5a562e in pool_destroy_debug memory/unix/apr_pools.c:1965
#10 0x7fdc7c5a5179 in pool_clear_debug memory/unix/apr_pools.c:1880
#11 0x7fdc7c5a562e in pool_destroy_debug memory/unix/apr_pools.c:1965
#12 0x7fdc7c5a5827 in apr_pool_destroy_debug memory/unix/apr_pools.c:2014
#13 0x7fdc789aeaa5 in h2_session_pre_close ~httpd/modules/http2/h2_session.c:1934
#14 0x7fdc7896a20e in h2_c1_pre_close ~httpd/modules/http2/h2_c1.c:188
#15 0x7fdc7896b538 in h2_c1_hook_pre_close ~httpd/modules/http2/h2_c1.c:308
#16 0x5596139aeb28 in ap_run_pre_close_connection ~httpd/server/connection.c:45
#17 0x5596139af353 in ap_prep_lingering_close ~httpd/server/connection.c:128
#18 0x5596139af3f2 in ap_start_lingering_close ~httpd/server/connection.c:154
#19 0x7fdc7835bdf0 in process_lingering_close ~httpd/server/mpm/event/event.c:1999
#20 0x7fdc78359ccb in process_socket ~httpd/server/mpm/event/event.c:1540
#21 0x7fdc783608d7 in worker_thread ~httpd/server/mpm/event/event.c:2756
#22 0x7fdc7c5d3e57 in dummy_worker threadproc/unix/thread.c:153
#23 0x7fdc7c441d7f in start_thread nptl/pthread_create.c:481
#24 0x7fdc7c337bde in clone (/lib/x86_64-linux-gnu/libc.so.6+0xfcbde)
0x603000080ce8 is located 8 bytes inside of 32-byte region [0x603000080ce0,0x603000080d00)
freed by thread T11 here:
#0 0x7fdc7c887f07 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:122
#1 0x7fdc7c5a5420 in pool_clear_debug memory/unix/apr_pools.c:1906
#2 0x7fdc7c5a562e in pool_destroy_debug memory/unix/apr_pools.c:1965
#3 0x7fdc7c5a5179 in pool_clear_debug memory/unix/apr_pools.c:1880
#4 0x7fdc7c5a562e in pool_destroy_debug memory/unix/apr_pools.c:1965
#5 0x7fdc7c5a5827 in apr_pool_destroy_debug memory/unix/apr_pools.c:2014
#6 0x7fdc789aeaa5 in h2_session_pre_close ~httpd/modules/http2/h2_session.c:1934
#7 0x7fdc7896a20e in h2_c1_pre_close ~httpd/modules/http2/h2_c1.c:188
#8 0x7fdc7896b538 in h2_c1_hook_pre_close ~httpd/modules/http2/h2_c1.c:308
#9 0x5596139aeb28 in ap_run_pre_close_connection ~httpd/server/connection.c:45
#10 0x5596139af353 in ap_prep_lingering_close ~httpd/server/connection.c:128
#11 0x5596139af3f2 in ap_start_lingering_close ~httpd/server/connection.c:154
#12 0x7fdc7835bdf0 in process_lingering_close ~httpd/server/mpm/event/event.c:1999
#13 0x7fdc78359ccb in process_socket ~httpd/server/mpm/event/event.c:1540
#14 0x7fdc783608d7 in worker_thread ~httpd/server/mpm/event/event.c:2756
#15 0x7fdc7c5d3e57 in dummy_worker threadproc/unix/thread.c:153
#16 0x7fdc7c441d7f in start_thread nptl/pthread_create.c:481
previously allocated by thread T11 here:
#0 0x7fdc7c8882b8 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144
#1 0x7fdc7c5a4d00 in pool_alloc memory/unix/apr_pools.c:1787
#2 0x7fdc7c5a507a in apr_palloc_debug memory/unix/apr_pools.c:1828
#3 0x7fdc7c4d8160 in apr_brigade_create buckets/apr_brigade.c:90
#4 0x7fdc7c4d82d8 in apr_brigade_split_ex buckets/apr_brigade.c:107
#5 0x7fdc78967f7c in h2_beam_receive ~httpd/modules/http2/h2_bucket_beam.c:729
#6 0x7fdc789b65f0 in buffer_output_receive ~httpd/modules/http2/h2_stream.c:847
#7 0x7fdc789bb655 in h2_stream_read_output ~httpd/modules/http2/h2_stream.c:1372
#8 0x7fdc789aa155 in on_stream_output ~httpd/modules/http2/h2_session.c:1313
#9 0x7fdc789956ba in mplx_pollset_poll ~httpd/modules/http2/h2_mplx.c:1299
#10 0x7fdc7898deb8 in h2_mplx_c1_poll ~httpd/modules/http2/h2_mplx.c:532
#11 0x7fdc789ae04b in h2_session_process ~httpd/modules/http2/h2_session.c:1863
#12 0x7fdc78969b0f in h2_c1_run ~httpd/modules/http2/h2_c1.c:138
#13 0x7fdc7896b302 in h2_c1_hook_process_connection ~httpd/modules/http2/h2_c1.c:286
#14 0x5596139ae4b6 in ap_run_process_connection ~httpd/server/connection.c:43
#15 0x7fdc78358d67 in process_socket ~httpd/server/mpm/event/event.c:1353
#16 0x7fdc783608d7 in worker_thread ~httpd/server/mpm/event/event.c:2756
#17 0x7fdc7c5d3e57 in dummy_worker threadproc/unix/thread.c:153
#18 0x7fdc7c441d7f in start_thread nptl/pthread_create.c:481
Thread T11 created by T2 here:
#0 0x7fdc7c7baa22 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208
#1 0x7fdc7c5d4534 in apr_thread_create threadproc/unix/thread.c:228
#2 0x7fdc7836273d in start_threads ~httpd/server/mpm/event/event.c:3035
#3 0x7fdc7c5d3e57 in dummy_worker threadproc/unix/thread.c:153
#4 0x7fdc7c441d7f in start_thread nptl/pthread_create.c:481
Thread T2 created by T0 here:
#0 0x7fdc7c7baa22 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cc:208
#1 0x7fdc7c5d4534 in apr_thread_create threadproc/unix/thread.c:228
#2 0x7fdc78363d9f in child_main ~httpd/server/mpm/event/event.c:3262
#3 0x7fdc7836483b in make_child ~httpd/server/mpm/event/event.c:3421
#4 0x7fdc78364b89 in startup_children ~httpd/server/mpm/event/event.c:3444
#5 0x7fdc78368abc in event_run ~httpd/server/mpm/event/event.c:3932
#6 0x5596139b6d18 in ap_run_mpm ~httpd/server/mpm_common.c:101
#7 0x55961399098b in main ~httpd/server/main.c:880
#8 0x7fdc7c2627ec in __libc_start_main ../csu/libc-start.c:332
SUMMARY: AddressSanitizer: heap-use-after-free ~httpd/modules/http2/h2_bucket_beam.c:279 in recv_buffer_cleanup
Shadow bytes around the buggy address:
0x0c0680008140: fa fa 00 00 00 00 fa fa fd fd fd fa fa fa fd fd
0x0c0680008150: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
0x0c0680008160: fd fd fd fd fa fa fd fd fd fd fa fa fd fd fd fd
0x0c0680008170: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
0x0c0680008180: fd fd fa fa fd fd fd fd fa fa fd fd fd fa fa fa
=>0x0c0680008190: fd fd fd fa fa fa fd fd fd fa fa fa fd[fd]fd fd
0x0c06800081a0: fa fa fd fd fd fd fa fa fd fd fd fd fa fa fd fd
0x0c06800081b0: fd fd fa fa fd fd fd fd fa fa fd fd fd fd fa fa
0x0c06800081c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c06800081d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c06800081e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==81201==ABORTING
Graham Leggett [Tue, 8 Feb 2022 12:34:52 +0000 (12:34 +0000)]
ab: Add an optional ramp delay when starting concurrent connections so
as to not trigger denial of service protection in the network. Report
levels of concurrency achieved in cases where the test completes before
full concurrency is achieved.
Stefan Eissing [Tue, 8 Feb 2022 12:18:45 +0000 (12:18 +0000)]
* mod_md) do not interfere with requests to /.well-known/acme-challenge/
resources if challenge type 'http-01' is not configured for a domain.
Fixes <https://github.com/icing/mod_md/issues/279>.
Stefan Eissing [Mon, 7 Feb 2022 14:40:46 +0000 (14:40 +0000)]
*) mod_md: the status description in MDomain's JSON, exposed in the
md-status handler (if configure) did sometimes not carry the correct
message when certificates needed renew.
[Stefan Eissing]
Stefan Eissing [Fri, 4 Feb 2022 12:22:26 +0000 (12:22 +0000)]
*) core/mod_ssl/mpm_event: reverting changes to nonblocing SSL handshakes
to stabilize CI tests again. Previous revision of trunk has been copied
to branches/trunk-ssl-handshake-unblocking to make those into a PR where
changes can be discussed and tested separately.
Yann Ylavic [Wed, 2 Feb 2022 10:02:26 +0000 (10:02 +0000)]
core: Follow up to r1897240: Opt-out for AP_HAS_THREAD_LOCAL and/or pcre's usage.
If the compiler's thread_local is not efficient enough on some platforms, or
not desired, have a way to disable its usage in httpd (at compile time).
Handle -DAP_NO_THREAD_LOCAL and/or -DAPREG_NO_THREAD_LOCAL as build opt-out for
thread_local usage in httpd gobally and/or in ap_regex only (respectively).
Stefan Eissing [Tue, 1 Feb 2022 10:40:30 +0000 (10:40 +0000)]
*) test cases
- disabling test_h2_105_02 due to handshake timeouts no longer working
- adjusting some log levels
- ignoring new handshake error AHLOGNOs
Yann Ylavic [Fri, 28 Jan 2022 12:32:25 +0000 (12:32 +0000)]
mod_cgi[d]: Close the connection after handling nph- scripts.
The HTTP output filters chain is bypassed for nph- scripts (whatever this
means), so there is no way to know whether the connection is reusable after
sending the cgi response (opaque header + body).
Set r->connection->keepalive = AP_CONN_CLOSE unconditionally.
For instance test frameork's "nph-foldhdr.pl" script responds with HTTP/1.0
and no Content-Length so closing the connection is the only way to detect the
end of response for the peer.
Yann Ylavic [Thu, 27 Jan 2022 15:06:55 +0000 (15:06 +0000)]
mpm_event: Use APR_POLLEXCL when available to prevent thundering hurd.
If APR_POLLEXCL is available, use it to prevent the thundering
herd issue. The listening sockets are potentially polled by all
the children at the same time, when new connections arrive this
avoids all of them to be woken up while most would get EAGAIN
on accept().
Yann Ylavic [Thu, 27 Jan 2022 12:34:53 +0000 (12:34 +0000)]
core: Follow up to r1897460: Provide ap_thread_main_create().
Replace ap_thread_current_create() by ap_thread_main_create() which is how
it's used by httpd. The former is now a local helper only to implement the
latter.
This allows to consolidate/factorize common code in the main() of httpd and
the unix MPMs.
Yann Ylavic [Tue, 25 Jan 2022 17:34:57 +0000 (17:34 +0000)]
core: Efficient ap_thread_current() when apr_thread_local() is missing.
#define ap_thread_create, ap_thread_current_create and ap_thread_current to
their apr-1.8+ equivalent if available, or implement them using the compiler's
thread_local mechanism if available, or finally provide stubs otherwise.
#define AP_HAS_THREAD_LOCAL to 1 in the two former case or 0 otherwise, while
AP_THREAD_LOCAL is defined to the compiler's keyword iff AP_HAS_THREAD_LOCAL.
Replace all apr_thread_create() calls with ap_thread_create() so that httpd
threads can use ap_thread_current()'s pool data as Thread Local Storage.
Bump MMN minor.
* include/httpd.h():
Define AP_HAS_THREAD_LOCAL, AP_THREAD_LOCAL (eventually), ap_thread_create(),
ap_thread_current_create() and ap_thread_current().
* server/util.c:
Implement ap_thread_create(), ap_thread_current_create() and
ap_thread_current() when APR < 1.8.
* modules/core/mod_watchdog.c, modules/http2/h2_workers.c,
modules/ssl/mod_ssl_ct.c:
Use ap_thread_create() instead of apr_thread_create.
* server/main.c:
Use AP_HAS_THREAD_LOCAL and ap_thread_current_create instead of APR's.
* server/util_pcre.c:
Use AP_HAS_THREAD_LOCAL and ap_thread_current instead of APR's.
* server/mpm/event/event.c, server/mpm/worker/worker.c,
server/mpm/prefork/prefork.c:
Use ap_thread_create() instead of apr_thread_create.
Create an apr_thread_t/ap_thread_current() for the main chaild thread usable
at child_init().
* server/mpm/winnt/child.c:
Use ap_thread_create() instead of CreateThread().
Create an apr_thread_t/ap_thread_current() for the main chaild thread usable
projects / thirdparty / apache / httpd.git / log
