Mitigate DoS attacks that use client-initiated SSL/TLS renegotiation.
There is a well-known DoS attack using client-initiated SSL/TLS
renegotiation. The severety or uniqueness of this attack method
is disputed, but many believe it is serious/real.
There is even a (disputed) CVE 2011-1473:
https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2011-1473
The old Squid code tried to disable client-initiated renegotiation, but
it did not work reliably (or at all), depending on Squid version, due
to OpenSSL API changes and conflicting SslBump callbacks. That
code is now removed and client-initiated renegotiations are allowed.
With this change, Squid aborts the TLS connection, with a level-1 ERROR
message if the rate of client-initiated renegotiate requests exceeds
5 requests in 10 seconds (approximately). This protection and the rate
limit are currently hard-coded but the rate is not expected to be
exceeded under normal circumstances.
Fix "Source and destination overlap in memcpy" Valgrind errors
Before this patch, source and destination arguments in
log_quoted_string() could point to the same static memory area, causing
multiple Valgrind-reported errors. Fixed by creating another buffer to
store quoted-processed output string.
Amos Jeffries [Fri, 13 Jan 2017 05:14:03 +0000 (18:14 +1300)]
Remove USE_CHUNKEDMEMPOOLS compiler flag
The USE_CHUNKEDMEMPOOLS build-time setting is not very useful and adds
extra complexity to the build system. Even when set it does not always
enable chunked pools. The environment variable MEMPOOLS can easily be
used to enable or disable chunked pools as needed.
The helper protocol for external ACLs [1] defines three possible return values:
OK - Success. ACL test matches.
ERR - Success. ACL test fails to match.
BH - Failure. The helper encountered a problem.
The external acl helpers distributed with squid currently doesn't follow this
definition. For example, upon connection error, ERR is returned:
$ ext_ldap_group_acl ... -d
ext_ldap_group_acl: WARNING: could not bind to binddn 'Can't contact LDAP server'
ERR
This is does not allow to distinguish "no match" and "error" either and
therefore negative caches "ERR", also in the case of an error.
Moreover there are multiple problems inside squid when trying to handle BH
responses:
- Squid-5 and squid-4 retries requests for BH responses but crashes after the
maximum retry number (currently 2) is reached.
- If an external acl helper return always BH (eg because the LDAP server is
down) squid sends infinitely new request to the helper.
Reduce crashes due to unexpected ClientHttpRequest termination.
The underlying problem has been known since r13480: If a
ClientHttpRequest job ends without Http::Stream (and ConnStateData)
knowledge, then Squid is likely to segfault or assert. This patch does
not resolve the underlying issue (a proper fix would require
architectural changes in a consensus-lacking area) but makes an
unexpected ClientHttpRequest job destruction less likely.
BodyPipe and Adaptation-related exceptions are the major causes of
unexpected ClientHttpRequest job destruction. This patch handles them by
closing the client connection. Connection closure should trigger an
orderly top-down cleanup, including Http::Stream, ConnStateData, and
ClientHttpRequest destruction.
If there is no connection to close, then the exception is essentially
ignored with a level-1 error message disclosing the problem. The side
effects of ignoring such exceptions are unknown, but without a client
connection, it is our hope that they would be relatively benign.
Amos Jeffries [Sun, 8 Jan 2017 05:14:34 +0000 (18:14 +1300)]
Cleanup: remove --disable-inline and related macros
With the .cci files gone there is no longer any use of the SQUID_INLINE
macro. Which in turn removes the need for _USE_INLINE_ macro and then the
--disable-inline build option which sets that.
The relevant compiler specific *FLAGS option should be used instead to
disable compiler inlining functions without special macro usage.
Bug 4599 pt6: Restrict safe renegotiation disable to OpenSSL 1.0 and older
From OpenSSL 1.1.0 documentation:
"OpenSSL always attempts to use secure renegotiation as described in RFC5746.
This counters the prefix attack described in CVE-2009-3555 and elsewhere."
From OpenSSL changeLog:
Remove support for all 40 and 56 bit ciphers. This includes all the export
ciphers who are no longer supported and drops support the ephemeral RSA key
exchange. The LOW ciphers currently doesn't have any ciphers in it.
[Kurt Roeckx]
OpenSSL 1.1.0 drops EXPORT ciphers and ephemeral RSA. The ssl_temp_rsa_cb used
here. The OpenSSL manual for this function is:
https://wiki.openssl.org/index.php/Manual:SSL_CTX_set_tmp_rsa_callback(3)
Amos Jeffries [Fri, 30 Dec 2016 06:27:19 +0000 (19:27 +1300)]
Detect HTTP header ACL issues
rep_header and req_header ACL types cannot match multiple different
headers in one test (unlike Squid-2 appears to have done). Produce
an ERROR and ignore the extra line(s) instead of silently changing
all the previous regex to match the second header name.
Also detect and ERROR when header name is missing entirely. Ignore
these lines instead of asserting.
Amos Jeffries [Thu, 22 Dec 2016 04:08:55 +0000 (17:08 +1300)]
Bug 3940 pt2: Make 'cache deny' do what is documented
Instead of overriding whatever cacheability was previously set to
(including changing non-cacheables to be cacheable) actually
prevent both cache read and write.
Amos Jeffries [Wed, 21 Dec 2016 17:11:34 +0000 (06:11 +1300)]
Cleanup: Refactor libauth Config object(s)
Refactoring the Auth::Config objects to get v5 building again.
The object storing auth_param settings is renamed to SchemeConfig.
A new Auth::Config object is added. The objects holding auth_param,
auth_schemes and other authenticate_* directives settings are stored
there instead of just auth_params.
Lots of outdated doxygen docs that have more up to date copies elsewhere
are removed. The "\ingroup AuthAPI" are left for now since some auth
things are not yet in the Auth:: namespace, but other auth related
groups are dropped completely.
Code parsing and dumping auth_param lines has been de-duplicated. Along
with the keep_alive and utf8 settings, which are shared by multiple
schemes. This allows dropping several virtual methods from NTLM and
Negotiate scheme objects.
An auth/forward.h file is added for predefines. Some initial Auth::
symbols are listed there. It is not complete, but others can be added as
needed later.
Some unnecessary includes have been removed. There are probably more. I
just did the obvious ones related to the new auth/forward.h and
auth/Config.h files at this time.
Amos Jeffries [Mon, 19 Dec 2016 10:47:37 +0000 (23:47 +1300)]
Drop old Basic Authenticatio docs
The wiki these days contains a more detailed and correct description of
how Basic auth helpers work in Squid. This old document does not even
describe how the code works, so is no longer useful.
Amos Jeffries [Mon, 19 Dec 2016 10:27:54 +0000 (23:27 +1300)]
Shuffle auth_schemes config objects into Auth::Config
Also, move the directive definition down into "Access Controls" section
of squid.conf to avoid dependency issues with acl directive definitions
being done too late.
Do not share private responses with collapsed client(s).
This excessive sharing problem with collapsed forwarding code has
several layers. In most cases, the core CF code does not share
uncachable or private response with collapsed clients because of the
refreshCheckHTTP() check. However, some responses might not be subject
to that (or equivalent) check. More importantly, collapsed revalidation
code does not check its responses at all and, hence, easily shares
private responses.
This short-term fix incorrectly assumes that an entry may become private
(KEY_PRIVATE) only when it cannot be shared among multiple clients
(e.g., because of a Cache-Control:private response header). However,
there are a few other cases when an entry becomes private. One of them
is a DISK_NO_SPACE_LEFT error inside storeSwapOutFileClosed() where
StoreEntry::releaseRequest() sets KEY_PRIVATE for a sharable entry [that
may still be perfectly preserved in the memory cache]. Consequently, the
short-term fix reduces CF effectiveness. The extent of this reduction is
probably environment-dependent.
Also: do not re-use SET_COOKIE headers for collapsed revalidation slaves,
i.e., adhere to the same requirement as for regular response HITs.
Added auth_schemes to control schemes presence and order in 401s/407s.
The new squid.conf directive may be used to customize authentication
schemes presence and order in Squid's HTTP 401 (Unauthorized) and 407
(Proxy Authentication Required) responses. The defaults remain the same.
Garri Djavadyan [Wed, 7 Dec 2016 00:05:07 +0000 (13:05 +1300)]
Bug 3940 (partial): hostHeaderVerify failures MISS when they should be HIT
This fixes the critical condition leading to the HIT. However not all
code is correctly setting flags.noCache and flags.cacheable (see bugzilla).
So there may be other fixes needed after this.
Amos Jeffries [Sun, 20 Nov 2016 09:12:00 +0000 (22:12 +1300)]
C++11: Remove GnuRegex and all -lregex related code
Squid is now exclusively using the STL std::regex API provided on all
operating systems in a portable manner.
We no longer have any need of detecting if the system is providing a
libregex, or user has configured one, or if it actually works, or how
to call it, or use the GnuRegex code as a backup when one of those
complex details goes wrong.
Amos Jeffries [Fri, 18 Nov 2016 18:08:30 +0000 (07:08 +1300)]
Fix Null pointer dereferences after rev.14913
on_unsupported_protocol with non-HTTP requests occurs without
an HTTP request object. Some ACLs will not be possible to check
but that is not a reason to crash.
The r14945 patch has a major bug:
When the Http::One::Server::writeControlMsgAndCall fails to write the control
message, schedules a Comm::Write callback using just a ScheduleCallHere command.
The callback called withtout the CommIoCbParams details and squid is crashes.
This patch fixes the ConnStateData::writeControlMsgAndCall to return false if it
fails to write the control message and allow the caller to handle the failure.
The following sequence of events triggers this assertion:
- The server sends an 1xx control message.
- http.cc schedules ConnStateData::sendControlMsg call.
- Before sendControlMsg is fired, http.cc detects an error (e.g., I/O
error or timeout) and starts writing the reply to the user.
- The ConnStateData::sendControlMsg is fired, starts writing 1xx, and
hits the "no concurrent writes" assertion.
We could only reproduce this sequence in the lab after changing Squid
code to trigger a timeout at the right moment, but the sequence looks
plausible. Other event sequences might result in the same outcome.
To avoid concurrent writes, Squid now drops the control message if
Http::One::Server detects that a reply is already being written. Also,
ConnStateData delays reply writing until a pending control message write
has been completed.
ICAP trailers are currently specified by
https://datatracker.ietf.org/doc/draft-rousskov-icap-trailers/
This implementation complies with version -01 of that draft:
- Squid unconditionally announces ICAP Trailer support via the ICAP
Allow request header field.
- Squid parses the ICAP response trailer if and only if the ICAP server
signals its presence by sending both Trailer header and Allow/trailers
in the ICAP response.
Squid logs and ignores all parsed ICAP header fields (for now).
Also refactored HttpHeader parsing methods in order to reuse them for
ICAP trailer parsing.
Also simplified and fixed headers isolating code while dealing with
empty (i.e. zero header fields) headers. Old httpMsgIsolateHeaders()
tried to re-implement header end detection/processing logic that is
actually covered by headersEnd(). Old httpMsgIsolateHeaders() could
return success for some garbage input (e.g., a buffer of several CRs)
even if no end of headers was found.