Inside IdleConnList::findUseable the IdleConnList::removeAt call can delete
"this" IdleConnList object. The IdleConnList::clearHandlers called imediatelly
after the removeAt method, will try to use the invalid "this" object in
a comm_read_cancel function call, causing this assertion or other similar.
This patch fixes the IdleConnList::findUseable, IdleConnList::pop and
IdleConnList::findAndClose methods to call IdleConnList::clearHandlers before
the IdleConnList::removeAt is called.
The maximum buffer size for holding Server and Client SSL hello messages is only
16k which is not enough hold a Hello message which includes some extensions and
1-2 or more Certificates.
This patch increases the maximum size to 65535 and also adds some checks to
avoid squid crashes in the case the hello messages buffer overflows.
This patch adds support for ICAP services that require SSL/TLS transport
connections. The same options used for the cache_peer directive are used for
the icap_service directive, with similar certificate validation logic.
To mark an ICAP service as "secure", use an "icaps://" service URI scheme when
listing your service via an icap_service directive. The industry is using a
"Secure ICAP" term, and Squid follows that convention, but "icaps" seems more
appropriate for a _scheme_ name.
Squid uses port 11344 for Secure ICAP by default, following another popular
proxy convention. The old 1344 default for plain ICAP ports has not changed.
Technical Details
==================
This patch:
- Splits Ssl::PeerConnector class into Ssl::PeerConnector parent and two kids:
Ssl::BlindPeerConnector, a basic SSL connector for cache_peers, and
Ssl::PeekingPeerConnector, a peek-and-splice SSL connector for HTTP servers.
- Adds a third Ssl::IcapPeerConnector kid to connect to Secure ICAP servers.
- Fixes ErrorState class to avoid crashes on nil ErrorState::request member.
(Ssl::IcapPeerConnector may generate an ErrorState with a nil request).
- Modifies the ACL peername to use the Secure ICAP server name as value while
connecting to an ICAP server. This is useful to make SSL certificate
policies based on ICAP server name. However, this change is undocumented
until we decide whether a dedicated ACL would be better.
This patch changes pconn_lifetime (r13780) to abort only really idle
persistent connections (when they timeout). It removes some "extra" features
(added to pconn_lifetime during the feature review) because they break things
when aggressive timeouts is combined with picky clients. Specifically,
1. Squid closed connections with partially received requests when they
reached pconn_lifetime limit. We should only close _idle_ connections.
2. When connecting to Squid without sending anything for longer than
pconn_lifetime, the connection hangs if the request is sent after the
waiting period.
3. The connection also hangs if the initial request is starting to be
transmitted but then there is a longer pause before the request is
completed.
Most of the above problems are easy to trigger only when using very aggressive
pconn_lifetime settings that the feature was not designed for, but they still
can be considered bugs from admins point of view. Fixes:
* Do not stop reading a partially received request when we are timing out,
to avoid aborting that request.
* Do not set keepalive flag based on the pconn_lifetime timeout. We cannot
predict whether some new request data is going to be read (and reset the
idle timeout clock) before our Connection:close response is sent back.
HTTP clients are supposed to recover from such races, but some apparently
do not, especially if it is their first request on the connection.
Convert packing methods from Packer* to Packable* objects
Packaging code can now receive any object that implements the Packable
interface instead of requiring the Packer trampoline object. For now it
is still the only object implementing Packable.
Packer class model used C-style function pointers and a standalone
object to perform C-style trampoline for function/method calls.
C++ virtual methods offer to inline all that directly in the data store
objects and enforces type safety on the child object methods instead of
forcing manual type casting on developers.
Re-implement Packer as a wrapper class providing the Packable interface
for backward compatibility with Packer* code. Future code should inherit
objects directly from Packable and implement the interface.
comm_connect_addr() uses errno to determine whether library calls like connect()
are successful. Its callers also use errno for extra information on the cause
of any problem. However, after calling library functions like connect(),
comm_connect_addr() calls other library functions which can overwrite errno.
As the errno manpage explains, "a function that succeeds is allowed to change
errno". So even when nothing is wrong, comm_connect_addr() may return an error
flag if libc sets errno. And when something *is* wrong, incorrect error information
may be returned to the caller because errno was overwritten with a different code.
Correct this by using our own error code variable which is set only when a library
call fails. To avoid breaking callers, set errno before returning.
Fix 'access_log none' to prevent following logs being used
The documented behaviour of "access_log none" for preventing logging
using log lines following the directive has not been working in
Squid-3 for some time.
Since the 'none' type does not have a log module associated the entire
switch logic where its abort is checked for was being skipped.
The SSL_get_peer_certificate openSSL function increases the lock for X509
object it returns so X509 object retrieved using this function must be
released with X509_free after use.
This patch uses the Ssl::X509_Pointer TidyPointer to release X509 object
retrieved with the SSL_get_peer_certificate function inside the
Ssl::PeerConnector::handleNegotiateError method
The SSL_get_peer_certificate openSSL function increases the lock for X509
object it returns so X509 object retrieved using this function must be
released with X509_free after use.
This patch uses the Ssl::X509_Pointer TidyPointer to release X509 object
retrieved with the SSL_get_peer_certificate function inside the
Ssl::PeerConnector::handleNegotiateError method
Unexpected SQUID_X509_V_ERR_DOMAIN_MISMATCH errors while accessing sites with valid certificates
A "const char *" pointer retrieved using the SBuf::c_str() method may attached
to an SSL object using the SSL_set_ex_data method as server name used to
validate server certificates. This pointer may become invalid, causing
the SQUID_X509_V_ERR_DOMAIN_MISMATCH errors.
This patch changes the type of the ssl_ex_index_server index used with the
SSL_set_ex_data function to be an SBuf object.
Portability: Add hacks to define C++11 explicit N-bit type limits
Add cstdint and stdint.h to libcompat headers and ensure that type limits
used by Squid are always available. Mostly this involves shuffling
existing hacks into the compat headers but the UINT32_* limits are new.
Despite the "must match" comment, MAX_AUTHTOKEN_LEN in
auth/UserRequest.h got out of sync with similar constants in Negotiate helpers.
A 32KB buffer cannot fit some helper requests (e.g., those carrying Privilege
Account Certificate information in the client's Kerberos ticket). Each truncated
request blocks the negotiate helper channel, eventually causing helper queue
overflow and possibly killing Squid.
This patch increases MAX_AUTHTOKEN_LEN in UserRequest.h to 65535 which
is also the maximum used by the negotiate helpers. The patch also adds checks
to avoid sending truncated requests, treating them as helper errors instead.
The fix for Bug 3664 "ssl_crtd fails to build on OpenSolaris/OpenIndiana/Solaris 11"
introduced a regression on BSD and Linux where lockf() implementations appear not to
lock the entire file correctly or as reliably as flock().
Reverting the flock/lockf change for non-Solaris OS.
This patch adds code in squid to control SslBump behavior when dealing with
"resuming SSL/TLS sessions". Without these changes, SslBump usually terminates
all resuming sessions with an error because such sessions do not include
server certificates, preventing Squid from successfully validating the server
identity.
After these changes, Squid splices resuming sessions. Splicing is the right
because Squid most likely has spliced the original connections that the client
and server are trying to resume now.
Without SslBump, session resumption would just work, and SslBump behaviour
should approach that ideal.
Future projects may add ACL checks for allowing resuming sessions and may
add more complex algorithms, including maintaining an SMP-shared
cache of sessions that may be resumed in the future and evaluating
client/server attempts to resume a session using that cache.
This patch also makes SSL client Hello message parsing more robust and
adds an SSL server Hello message parser.
Also add support for NPN (next protocol negotiation) and ALPN (Application-Layer Protocol Negotiation) tls extensions, required to correctly bump web clients
support these extensions
Technical details
-----------------
In Peek mode, the old Squid code would forward the client Hello message to the
server. If the server tries to resume the previous (spliced) SSL session with
the client, then Squid SSL code gets an ssl/PeerConnector.cc "ccs received
early" error (or similar) because the Squid SSL object expects a server
certificate and does not know anything about the session being resumed.
With this patch, Squid detects session resumption attempts and splices
There are two mechanism in SSL/TLS for resuming sessions. The traditional
shared session IDs and the TLS ticket extensions:
* If Squid detects a shared ID in both client and server Hello messages, then
Squid decides whether the session is being resumed by comparing those client
and server shared IDs. If (and only if) the IDs are the same, then Squid
assumes that it is dealing with a resuming session (using session IDs).
* If Squid detects a TLS ticket in the client Hello message and TLS ticket
support in the server Hello message as well as a Change Cipher Spec or a New
TLS Ticket message (following the server Hello message), then (and only then)
Squid assumes that it is dealing with a resuming session (using TLS tickets).
The TLS tickets check is not performed if Squid detects a shared session ID
in both client and server Hello messages.
NPN and ALPN tls extensions
---------------------------
Even if squid has some SSL hello messages parsing code, we are relying to
openSSL for full parsing. The openSSL used in peek and splice mode to parse
server hello message, check for errors and verify server certificates.
If the openSSL, while parses the server hello message, find an extension enabled
in the server hello message, which is not enabled in its side, fails with an
error ("...parse tlsext...").
OpenSSL supports NPN tls extension and from 1.0.2 release supports also the
ALPN tls extensions. In peek mode we are forwading the client SSL hello message
as is, and if this message include support for NPN or ALPN tls extension is
possible that the SSL server support them and include related extensions
in its response. The openSSL will fail if support for these extensions is
not enabled in its side.
This patch handles the NPN (TLSEXT_TYPE_next_proto_neg) as follows:
Try to select the http/1.1 protocol from the server protocols list. If the
http/1.1 is not supported then the SSL bumping will fail. This is valid
because only http protocol we are supporting in squid.
Splicing is not affected.
Also add support for the ALPN TLS extension. This extension is a replacement
for the NPN extension. The client sends a list of supported protocols. In the
case of stare mode squid now sends only http as supported protocol. In the
case of server-first or client-first bumbing modes, squid does enable this
extension.
The NPN supported by chromium browser the ALPN supported by firefox.
Support for ALPN is added to openSSL 1.0.2 release.
These extensions are used to support SPDY and similar protocols.
Add server_name ACL matching server name(s) obtained from various sources such as CONNECT request URI, client SNI, and SSL server certificate CN.
During each SslBump step, Squid improves its understanding of a "true server
name", with a bias towards server-provided (and Squid-validated) information.
The server-provided server names are retrieved from the server certificate CN
and Subject Alternate Names. The new server_name ACL matches any of alternate
names and CN. If the CN or an alternate name is a wildcard, then the new ACL
matches any domain that matches the domain with the wildcard.
Other than supporting many sources of server name information (including
sources that may supply Squid with multiple server name variants and
wildcards), the new ACL is similar to dstdomain.
Fix HttpStateData::readReply to retry read from server in the case of EINPROGRESS, EAGAIN or similar errors
This bug mostly affects SSL bumped connections.
The HttpStateData::readReply will not retry read from server in the case of an
EINPROGRESS or similar comm errors and the connection will hang, until the
timeout handler called.
The Comm::ReadNow method calls ignoreErrno function to test if the comm error
should be ignored and in this case return Comm::INPROGRESS value.
In this case we need to set flags.do_next_read to true to force
HttpStateData::maybeReadVirginBody() method retry read.
Fix: An invalid request->clientConnectionManager object can be used inside Ssl::PeerConnector::handleNegotiateError method
This patch adds the Ssl::ServerBio::bumpMode() method to retrieve the configured
mode from a ServerBio object, and uses this method for checking the bumping
mode inside Ssl::PeerConnector::handleNegotiateError method
After a failed http_access acl check of an HTTP request, tunneled through a
SSL bumped connection, ssl bumping code try to re-setup the connection for a
client-first bumping mode to serve the error crashing squid.
Amos Jeffries [Tue, 31 Mar 2015 13:58:50 +0000 (06:58 -0700)]
Abort parse cleanly if there is nothing to do
ICAP behaviour on noteMoreBodySpaceAvailable() can result in a parse
being attempted even if the read I/O buffer has not yet received any
new content to fill the available space. Handle this cleanly instead of
throwing parse errors and causing ICAP service disconnect.
Amos Jeffries [Mon, 30 Mar 2015 11:43:06 +0000 (04:43 -0700)]
Refactor Http1::ChunkedCodingParser to Http1::Parser API with SBuf
* refactor the parse to accept SBuf input buffer using the Parser API
and shift the MemBuf output buffer accesses to a separate API method.
* refactor the parsing steps to use Parser methods where available and
Tokenizer for lexical tokenising.
This removes one temporary data copy and two performance regressions
from each chunked message payload processing action. It also removes
c-string operations and strign termination dependency from chunk parsing.
Amos Jeffries [Sat, 28 Mar 2015 14:53:59 +0000 (07:53 -0700)]
Parser-NG: Convert the ICAP read buffer to an SBuf.
* Remove the double-buffering hack used to comm_read() ICAP responses as
c-string then convert to MemBuf for parsing.
* Revert the HttpMsg parser API from MemBuf to c-string parameters.
The internals did not make much use of the MemBuf abilities and it is
simpler to retrieve c-string values directly from an SBuf than to go
via a MemBuf conversion.
When squid generated an error page which contains the "%m" formating code
but the authentication information is not available squid dies with
segfault.
Amos Jeffries [Sat, 21 Mar 2015 13:35:24 +0000 (06:35 -0700)]
Shuffle Ssl::parse_options to Security::ParseOptions
The function itself is generic, the options array entries are all
conditional so library agnostic.
Adjust the context creation functions to receive pre-parsed options
instead of the string to avoid circular dependency between libsquidssl
and libsecurity.
Amos Jeffries [Sat, 21 Mar 2015 08:25:19 +0000 (01:25 -0700)]
Crypto-NG: Move Ssl::PeerConnectorAnswer to Security::EncryptorAnswer
This class was not actually depending on OpenSSL API symbols and by
abstracting it out we can unify the callback handlers for encrypted and
non-encrypted logic paths for several classes that setup connections.
projects / thirdparty / squid.git / log
