Joe Orton [Thu, 12 Sep 2024 07:59:22 +0000 (07:59 +0000)]
Add Multipath TCP (MPTCP) support (Proxy)
Multipath TCP (MPTCP), standardized in RFC8684 [1],
is a TCP extension that enables a TCP connection to
use different paths.
Multipath TCP has been used for several use cases.
On smartphones, MPTCP enables seamless handovers between
cellular and Wi-Fi networks while preserving established
connections. This use-case is what pushed Apple to use
MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to
automatically use the best performing path, either IPv4
or IPv6. If one path fails, MPTCP automatically uses
the other path.
To benefit from MPTCP, both the client and the server
have to support it. Multipath TCP is a backward-compatible
TCP extension that is enabled by default on recent
Linux distributions (Debian, Ubuntu, Redhat, ...). Multipath
TCP is included in the Linux kernel since version 5.6 [3].
To use it on Linux, an application must explicitly enable
it when creating the socket. No need to change anything
else in the application.
Adding the possibility to create MPTCP sockets would thus
be a really fine addition to httpd, by allowing clients
to make use of their different interfaces.
This patch introduces the possibilty to connect to backend
servers using MPTCP. Note however that these changes are
only available on Linux, as IPPROTO_MPTCP is Linux specific
for the time being.
For proxies, we can connect using MPTCP by passing the
\"multipathtcp\" parameter:
Multipath TCP (MPTCP), standardized in RFC8684 [1],
is a TCP extension that enables a TCP connection to
use different paths.
Multipath TCP has been used for several use cases.
On smartphones, MPTCP enables seamless handovers between
cellular and Wi-Fi networks while preserving established
connections. This use-case is what pushed Apple to use
MPTCP since 2013 in multiple applications [2]. On dual-stack
hosts, Multipath TCP enables the TCP connection to
automatically use the best performing path, either IPv4
or IPv6. If one path fails, MPTCP automatically uses
the other path.
To benefit from MPTCP, both the client and the server
have to support it. Multipath TCP is a backward-compatible
TCP extension that is enabled by default on recent
Linux distributions (Debian, Ubuntu, Redhat, ...). Multipath
TCP is included in the Linux kernel since version 5.6 [3].
To use it on Linux, an application must explicitly enable
it when creating the socket. No need to change anything
else in the application.
Adding the possibility to create MPTCP sockets would thus
be a really fine addition to httpd, by allowing clients
to make use of their different interfaces.
This patch introduces the possibility to listen with MPTCP
sockets. Note however that these changes are only available
on Linux, as IPPROTO_MPTCP is Linux specific for the time being.
To do so, we extended the Listen directive to include
a \"multipathtcp\" option, allowing to create MPTCP sockets
instead of regular TCP ones:
Listen 80 options=multipathtcp
We then store this information in flags for the listen directive
and create sockets appropriately according to this value.
* Leave the proper escaping of the URL and the adding of r->args to the
proxy module which runs after us after r1920570.
Just take care to add r->args in case the proxy rule has the
[NE] flag set and tell the proxy module to not escape in this case.
mod_rewrite, mod_proxy: mod_proxy to cononicalize rewritten [P] URLs. PR 69235.
When mod_rewrite sets a "proxy:" URL with [P], it should be canonicalized by
mod_proxy still, notably to handle any "unix:" local socket part.
To avoid double encoding in perdir context, a follow up commit should remove the
ap_escape_uri() done in mod_rewrite since it's now on mod_proxy to canonicalize,
per PR 69260.
Before r1918550 (r1918559 in 2.4.60), "SetHandler proxy:..." configurations
did not pass through proxy_fixup() hence the proxy_canon_handler hooks, leaving
fcgi's SCRIPT_FILENAME environment variable (from r->filename) decoded, or more
exactly not re-encoded.
We still want to call ap_proxy_canon_url() for "fcgi:" to handle/strip the UDS
"unix:" case and check that r->filename is valid and contains no controls, but
proxy_fcgi_canon() will not ap_proxy_canonenc_ex() thus re-encode anymore.
Note that this will do the same for "ProxyPass fcgi:...", there is no reason
that using SetHandler or ProxyPass don't result in the same thing. If an opt
in/out makes sense we should probably look at ProxyFCGIBackendType.
Yann Ylavic [Thu, 1 Aug 2024 11:35:26 +0000 (11:35 +0000)]
mod_proxy: Fix selection of ProxyPassMatch workers with host/port substitution. PR 69233.
With "ProxyPassMatch ^/([^/]+)/(.*)$ https://$1/$2", ap_proxy_get_worker_ex()
should not consider the length of scheme://host part of the given URL because
of the globbing match on the host part.
Fix it by setting worker->s>is_host_matchable when creating a worker with host
substitution and avoiding the min_match check in worker_matches() in this case.
Ivan Zhakov [Tue, 30 Jul 2024 00:15:32 +0000 (00:15 +0000)]
Follow-up to r1919413: CMake: Use configure_file() instead of file(write)
to generate modules.c file because configure_file() doesn't change
timestamp of file if contents is the the same.
Joe Orton [Mon, 29 Jul 2024 08:40:42 +0000 (08:40 +0000)]
* modules/dav/fs/repos.c (dav_fs_get_resource): Return a 404 rather
than a 400 where r->path_info is not empty for a file; a valid but
unsatisfiable request to a path which cannot exist,
e.g. /dav/foo.txt/blah where /dav/foo.txt is not a directory.
mod_proxy: Allow for empty UDS URL hostname in ProxyPass workers too.
Using "unix:/udspath|scheme:" or "unix:/udspath|scheme://" for a ProxyPass URL
does not work currently, while it works for SetHandler "proxy:unix:...".
*) mod_proxy: Avoid AH01059 parsing error for SetHandler "unix:" URLs
in <Location> (incomplete fix in 2.4.62). PR 69160.
When SetHandler "unix:..." is used in a <Location "/path"> block, the path
gets appended (including $DOCUMENT_ROOT somehow) to r->filename hence the
current checks in fixup_uds_filename() to add "localhost" when missing don't
work. Fix them.
Joe Orton [Fri, 26 Jul 2024 09:14:40 +0000 (09:14 +0000)]
CI: Fix OpenSSL tarball download URLs after openssl.org site refresh
CI: Update to latest OpenSSL releases.
CI: Build OpenSSL with RPATH set so that the installed ./bin/openssl works
without LD_LIBRARY_PATH set.
Use LD_RUN_PATH during the httpd build to achieve the same with binaries
from the httpd build, but unset it after so that it doesn't affect running
e.g. php-fpm or perl later. Should fix warning from logs when php-fpm
is executed --
[26-Jul-2024 07:43:34] NOTICE: PHP message: PHP Warning: PHP Startup: Unable to load dynamic library 'curl.so' (tried: /usr/lib/php/20210902/curl.so (/lib/x86_64-linux-gnu/libcurl.so.4: undefined symbol: ENGINE_init, version OPENSSL_3.0.0), /usr/lib/php/20210902/curl.so.so (/usr/lib/php/20210902/curl.so.so: cannot open shared object file: No such file or directory)) in Unknown on line 0
CI: Test that php-fpm works if available before testing.
CI: For paranoia/future debugging, log the OpenSSL version from compile-time
and run-time as reported by mod_ssl.
Ivan Zhakov [Sun, 21 Jul 2024 07:40:54 +0000 (07:40 +0000)]
CMake: Use TARGET_COMPILE_DEFINITIONS to set preprocessor definitions.
TARGET_COMPILE_DEFINITIONS correctly handle values with spaces, so
DEFINE_WITH_BLANKS() macro can be removed.
Ivan Zhakov [Sat, 20 Jul 2024 13:36:20 +0000 (13:36 +0000)]
* build/build-modules-c.awk: Add AP_DECLARE_DATA to ap_prelinked_modules,
ap_prelinked_modules_symbols and ap_preloaded_modules to match declaration
in ap_config.h. It doesn't change behavior because AP_DECLARE_DATA is
empty on non-Windows platform.
Ivan Zhakov [Sat, 20 Jul 2024 12:15:48 +0000 (12:15 +0000)]
* os/win32/modules.c: Include "ap_config.h" to match the file generated by
build/build-modules-c.awk. It doesn't change the behaviour because ap_config.h
is included by httpd.h.
mod_rewrite: Better question mark tracking to avoid UnsafeAllow3F. PR 69197.
Track in do_expand() whether a '?' in the uri-path comes from a literal in
the substitution string or from an expansion (variable, lookup, ...).
In the former case it's safe to assume that it's the query-string separator
but for the other case it's not (could be a decoded %3f from r->uri).
This allows to avoid [UnsafeAllow3F] for most cases.
Stefan Eissing [Wed, 10 Jul 2024 10:55:23 +0000 (10:55 +0000)]
sync test code with mod-h2
- shutdown server at end of h2 tests
- adapt minimum httpd versions for some tests
- add test_700_20 for load on blocked connections,
disabled for now until mpm_event improves
- build websocket client automatically
mod_proxy: Don't mangle r->filename when ap_proxy_canon_netloc() fails.
ap_proxy_canon_netloc() called from canon_handler hooks modifies its given
url in pace, hence &r->filename[6] passed from ap_proxy_canon_url().
This is not an issue if every canon_handler hook succeeds (or declines)
since r->filename is usually completely rewritten finally, but on failure
it gets truncated.
Avoid this by passing a copy of r->filename from the start, the proxy *url
and r->filename don't need to point to the same data.
* proxy/proxy_util.c(ap_proxy_canon_url):
Pass a copy of r->filename to the canon_handler hooks.
mod_proxy: Fix canonicalisation and FCGI env (PATH_INFO, SCRIPT_NAME) for
"balancer:" URLs set via SetHandler, also allowing for "unix:"
sockets with BalancerMember(s). PR 69168.
* modules/proxy/proxy_util.h, modules/proxy/proxy_util.c:
Move proxy_interpolate() from mod_proxy.c to ap_proxy_interpolate(),
exported locally only (non public).
Move proxy_fixup() from mod_proxy.c to ap_proxy_canon_url(), exported
locally only too (non public).
Rollback ap_proxy_fixup_uds_filename() to a local fixup_uds_filename()
usable from proxy_util.c only. The public function will be removed in
a following commit.
* modules/proxy/mod_proxy.h:
Note that ap_proxy_fixup_uds_filename() is deprecated.
* modules/proxy/mod_proxy.c:
Just use ap_proxy_canon_url() from proxy_fixup() and proxy_handler()
for SetHandler URLs.
* modules/proxy/mod_proxy_balancer.c:
Do not canonicalize the path from proxy_balancer_canon() anymore but
rather from balancer_fixup() where the balancer URL is rewritten to
the BalancerMember URL.
The hostname part of the URL is not mandated for UDS though the canon_handler
hooks will require it, so add "localhost" if it's missing (won't be used anyway
for an AF_UNIX socket).
This can trigger with SetHandler "unix:" URLs which are now also fixed up.
Since r1908537 BIO_set_callback_ex is used with OpenSSL >= 3.0 instead of
BIO_set_callback to set the BIO callback. The meaning of parameters and
their range of values in the callback function set by BIO_set_callback_ex
has changed compared to the callback function set by BIO_set_callback
although parameters kept their names. Accommodate for this and adjust the
code accordingly.
Furthermore limit the size of dumps to APR_UINT16_MAX bytes. Given the length
of SSL records of 16k this should not have practical implications.
Yann Ylavic [Tue, 25 Jun 2024 23:49:09 +0000 (23:49 +0000)]
mod_proxy: Fixup UDS filename for mod_proxy called through r->handler.
* modules/proxy/proxy_util.c:
Export ap_proxy_fixup_uds_filename() from fix_uds_filename.
Call it from ap_proxy_pre_request() even for rewritten balancer workers.
projects / thirdparty / apache / httpd.git / log
