Support http_access denials of SslBump "peeked" connections.
If an SSL connection is "peeked", it is currently not possible to deny it
with http_access. For example, the following configuration denies all plain
HTTP requests as expected but allows all CONNECTs (and all subsequent
encrypted/spliced HTTPS requests inside the allowed CONNECT tunnels):
http_access deny all
ssl_bump peek all
ssl_bump splice all
The bug results in insecure bumping configurations and/or forces admins to
abuse ssl_bump directive (during step1 of bumping) for access control (as a
partial workaround).
This change sends all SSL tunnels (CONNECT and transparent) through http_access
(and adaptation, etc.) checks during bumping step1. If (real or fake) CONNECT is
denied during step1, then Squid does not connect to the SSL server, but bumps
the client connection, and then delivers an error page (in response to the
first decrypted GET). The behavior is similar to what Squid has already been
doing for server certificate validation errors.
Technical notes
----------------
Before these changes:
* When a transparent SSL connection is being bumped, if we decide to splice
during step1, then we splice the connections without any http_access
checks. The (spliced) connection is always established.
* When a CONNECT tunnel is being bumped at step1, if peek/stare/server-first
mode is selected, and our http_access check fails, then:
1) We create an error page and proceeding with SSL bump, expecting
to serve the error after the client SSL connection is negotiated.
2) We start forwarding SSL Hello to the server, to peek/stare at (or
server-first bump) the server connection.
3) If we then decide to splice the connection during step2 or step3, then
we splice, and the error page never gets served to the client!
After these changes:
* During transparent SSL bumping, if we decide to splice at step1, do not
splice the connection immediately, but create a fake CONNECT request first
and send it through the callout code (http_access check, ICAP/ECAP, etc.).
If that fake CONNECT is denied, the code path described below kicks in.
* When an error page is generated during CONNECT or transparent bumping
(e.g. because an http_access check has failed), we switch to the
"client-first" bumping mode and then serve the error page to the client
(upon receiving the first regular request on the bumped connection).
Bug 4164: SEGFAULT when %W formating code used in errorpages
Squid will crash inside ErrorState::Dump if not authentication configured for
squid. In this case ErrorState::auth_user_request is NULL and trying to access
a method of this object will cause segfault to squid.
Hussam Al-Tayeb [Tue, 16 Dec 2014 12:23:58 +0000 (01:23 +1300)]
Bug 3826: pt 2: Provide a systemd .service file for Squid
Created with help from davidstrauss in #systemd channel and provided
as a working example for package distributors to use. It is not
installed by a 'make install' build of Squid.
For now SMP support is not available to Squid controlled by systemd.
That part of the bug 3826 issue has yet to be resolved.
Amos Jeffries [Thu, 11 Dec 2014 08:35:32 +0000 (00:35 -0800)]
Update Http::ProtocolVersion() to initializer functions
The Http::ProtocolVersion(*) does not work sufficiently well as a class
hierarchy.
Convert Http::ProtocolVersion to two functions:
* Http::ProtocolVersion() providing the default Squid HTTP version
level, and
* Http::ProtocolVersion(unsigned, unsigned) providing the HTTP version
details for the given level.
NP: using two overloaded functions instead of one with default
parameter values because with HTTP/0.x and HTTP/2.x we cannot safely
default just the minor value. ie. using two functions prevents
mistakenly using HTTP/2.1, HTTP/0.1 or HTTP/1.0 if the second
parameter is omitted.
All variables must now be of type AnyP::ProtocolVersion, and should be
constructed from an appropriate Foo::ProtocolVersion() function.
Amos Jeffries [Mon, 8 Dec 2014 11:25:58 +0000 (03:25 -0800)]
Update localnet definition for RFC 6890
RFC 6890 details updated IP address reservations for Carrier-Grade NAT
and confirms registration of the "this" network range legitimacy amongst
other non-relevant ddress range allocations.
Amos Jeffries [Fri, 5 Dec 2014 13:02:46 +0000 (05:02 -0800)]
HTTP/2: handle 'PRI' method found in HTTP/1.x traffic
draft-ietf-httpbis-http2-16 section 11.6 registers the method PRI.
"
This method is never used by an actual client.
This method will appear to be used when an HTTP/1.1 server or
intermediary attempts to parse an HTTP/2 connection preface.
"
If seen with a non-2.0 version number it means some client or proxy has
mishandled an HTTP/2.0 connection preface and corrupted the traffic.
The url_rewrite_timeout directive can accept the on_timeout argument to allow
user configure the action when the helper request times out.
The available actions are:
fail: squid return a ERR_GATEWAY_FAILURE error page
bypass: the url is not rewritten.
retry: retry the request to helper
use_configured_response: use a response which can be configured using the
the response= option
Example usage:
url_rewrite_timeout 30 seconds \
on_timeout=use_configured_response \
response="OK url=http://example.com/support"
Amos Jeffries [Wed, 3 Dec 2014 15:13:08 +0000 (07:13 -0800)]
HTTP/2: Support 421 (Misdirected Request) status code
Add support for status 421 responses. Squid is forbidden from generating
messages with this status in its role as proxy, however we expect to see
it being produced in responses by HTTP/2 servers in traffic from HTTP
2.0<->1.1 gateways.
We also MAY emit it on future reverse-proxy responses in the event of a
ERR_CANNOT_FORWARD message.
Amos Jeffries [Wed, 3 Dec 2014 14:12:12 +0000 (06:12 -0800)]
Bug 4135: Support \-escaped character in regex patterns
Squid cannot parse regex patterns as quoted strings since the pattern may
itself contain quote characters as part of the syntax.
Since we updated the squid.conf ConfigParser it is now possible to
handle regex patterns containing quoted-pair (\-escaped) characters
properly.
Add support for escaping by detecting the '\' characters as token
delimiters, and explicitly skipping the following character regardless
of whether it is a SP or not.
Escape detection is only added during parsing of regex tokens or
files listing regex patterns.
Some FTP severs respond to a FEAT command with 5xx status code. Squid sends
an invalid response in these cases which can confuse the client.
This patch fixes Squid to always send a valid 211 reply to client which
lists at least the EPSV and EPRT ftp commands which supported by Squid
regardless of the origin server support.
This patch also fixes a memory leak when FEAT replies processed.
The certificate db size file may become empty (for reasons beyond Squid
control such as server reboots, and possibly some unknown Squid bugs).
When it becomes empty, all ssl_crtd helpers (and then Squid) quit. This
change is required to make ssl_crtd more robust by recovering lost db
size information.
This patch:
- Adds the "size" rebuild operation in CertificateDB and ssl_crtd
daemon. Rebuild ssl_db/size file if it is empty:
* Inside Ssl::CertificateDb::check method
* When a CertificateDB operation try to read size from ssl_db/size file
- If no fs_block_size parameter given for CertificateDB then consider a
default value of 2048. Currently set to 0, which is may cause segfault to
ssl_crtd daemon.
Amos Jeffries [Fri, 21 Nov 2014 18:26:17 +0000 (10:26 -0800)]
Revert r12298 workaround for Bug 3613
Now that Squid is starting to utilize C++11 features we need to enable
them in all compilers. If the problem still exists then we need to find
a better solution to bug 3613.
Amos Jeffries [Fri, 21 Nov 2014 18:09:06 +0000 (10:09 -0800)]
HTCP: fix memory initialization errors
memset() used to initialize HTCP objects made sense when they were
structs. But now they are classes proper constructors need to be used
to avoid memset() erasing vtable and other important areas. It also
helps to reduce code and improve performance during init a tiny bit.
Amos Jeffries [Fri, 21 Nov 2014 04:33:21 +0000 (20:33 -0800)]
Fix build errors on CentOS 7 in rev.13712
It turns out the RHEL and CentOS 7 STL definition does not quite match
up to official documentation of what their GCC version supports. In
particular the use of const_iterator on std::list containers.
Use auto instead of an explicit iterator. Allowing the compiler to select
whichever iterator is available and will work.
In many cases HITs logged with zero response times. The logging entries are
correct, those transaction took less than a millisecond. However, to better
monitor Squid performance and to optimize things further, a user may want to
see more precise response time measurements logged.
Squid already computes response times with microsecond resolution
(timeval::tv_usec), which would be enough for any modern measurement, but
Squid loses that precision due to tvSubMs conversion.
This patch add configurable "precision" for time-related fields such as %tr,
using a .n syntax similar to the syntax used by the well-known printf(3) API
to mean "maximum field width".
Amos Jeffries [Wed, 19 Nov 2014 22:29:50 +0000 (14:29 -0800)]
ParserNG: fix typo in rev.13688
Squid xstrncpy() assumes \0-termination and enforces it by overwriting
the last byte of the output. Un-terminated strings must have +1 length
delivered or they will be truncated.
Amos Jeffries [Tue, 18 Nov 2014 10:13:39 +0000 (02:13 -0800)]
Alternate-Protocol is a hop-by-hop header
Google are pushing Alternate-Protocol header to arbitrarily move clients
from HTTP to their experimental and proprietary protocols.
When relayed through a proxy this can cause traffic bifurcation with
side effects ranging from client visible failures, to traffic increase
on the network as proxy caching is bypassed, to security vulnerabilities
or privacy information leaks over the experimental protocol.
Treating the Alternate-Protocol header as a hop-by-hop header only
relevant on client->origin connections we can both prevent the above
issues occuring on networks utilizing a proxy, and also participate in
the protocol exprimentation process ourselves.
Amos Jeffries [Tue, 18 Nov 2014 06:38:51 +0000 (22:38 -0800)]
Move the AnyP::ProtocolVersion stream operator into AnyP:: namespace
Clang compiler detects an issue determining which of multiple overloaded
operator<<() at global scope apply to the display of
AnyP::ProtocolVersion.
Moving the operator into the class itself is not possible due to similar
scoping issues with GCC failing to detect it as an inline member from
inside expanded templates.
To resolve both issues we can move the operator only to the AnyP::
namespace but no deeper.
Amos Jeffries [Mon, 17 Nov 2014 10:04:18 +0000 (02:04 -0800)]
Fix output from squid.conf parser
DBG_CRITICAL level messages before self_destruct() require a FATAL: tag
to highlight the reason for the crash and obscure "BUNGLED" text which
is about to follow.
Currently squid fails to handle correctly "100 Continue" requests/responses
when ICAP is used. The problems discussed in squid bugzilla:
http://bugs.squid-cache.org/show_bug.cgi?id=4067
A short discussion of the problem:
When a data upload request enters squid (eg a PUT request with
"Expect: 100-continue" header), squid sends ICAP headers and HTTP headers to
ICAP server and stucks waiting for ever the request body data.
This patch implements the "force_request_body_continuation" access list
directive which controls how Squid handles data upload requests from HTTP and
send the request body to Squid.
An allow match tells Squid to respond with the HTTP 100 or FTP 150
(Please Continue) control message on its own, before forwarding the
request to an adaptation service or peer. Such a response usually forces
the request sender to proceed with sending the body. A deny match tells
Squid to delay that control response until the origin server confirms
that the request body is needed. Delaying is the default behavior.
Amos Jeffries [Mon, 10 Nov 2014 08:42:54 +0000 (00:42 -0800)]
C++11: cleanup compiler flag detection logics
Annotate what GCC version is required to pass the -std=c++11 and
-std=c++0x feature detection tests.
Disable the tests for features we do not use that require GCC 4.5+.
This should allow Squid to continue building on older OS and compilers
at least until we start using those features.
Amos Jeffries [Mon, 10 Nov 2014 07:47:13 +0000 (23:47 -0800)]
RFC 6176 compliance
... prohibits use of SSLv2.
https://tools.ietf.org/html/rfc6176
Remove the documentation and support for configuring Squid with
SSLv2-only.
Explicitly enable the SSL_NO_SSLv2 option when provided by the library
to prevent implicit fallback.
Remove support for ssloptions= values which are for SSLv2-specific bugs.
Due to the way they are implemented with atoi() sslversion=N
configuration will still accept the values for SSLv2-only. But the
context creation will now unconditionally produce "SSLv2 not
supported" errors if the now undocumented values are attempted.
Make helper queue size configurable, with consistent defaults and better overflow handling.
This patch adds a queue-size=N option to helpers configuration. This
option allows users to configure the maximum number of queued requests
to busy helpers. We also adjusted the default queue size limits to be
more consistent across all helpers and made Squid more robust on some
queue overflows:
- external_acl helpers
Make the maximum queue size configurable via queue-size.
Default to 2*maximum-number-of-children.
If the queue overflows, then the ACL returns ACCESS_DUNNO.
Unpatched code uses the number of running children as the maximum
queue size. If the queue is overloaded, then the ACL returns ACCESS_DUNNO.
-redirector/storeID helpers
Make the maximum queue size configurable via queue-size.
Default to 2*maximum-number-of-children.
If the queue overflows and redirector_bypass configuration option
is set, then redirector is bypassed. Otherwise, if overloading
persists for more than 3 minutes squid quits with a FATAL message.
If the redirector_bypass/storeID_bypass is set then the default queue_size
is set to 0 for backward compatibility.
Unpatched code uses 2*number-of-running-children as the maximum queue size.
If the redirector_bypass/storeID_bypass is set then helper bypassed if all
of the children are busy.
If the queue is overloaded and redirector_bypass/storeID_bypass is not set
then squid quits with a FATAL message.
- ssl_crtd/ssl_crtd_validator helpers.
Make the maximum queue size configurable via queue-size.
Default to 2*maximum-number-of-children.
If the queue overflows, then helpers are bypassed. If overloading persists
for more than 3 minutes squid quits with a FATAL message.
The default size limit and overflow behavior has not changed.
- Authentication helpers
Make the maximum queue size configurable via queue-size.
Default to 2*maximum-number-of-children.
If the queue overflows and overloading persists for more than 3 minutes,
then squid quits with a FATAL message.
The default size limit and overflow behavior has not changed.
Amos Jeffries [Fri, 7 Nov 2014 08:41:47 +0000 (00:41 -0800)]
Parser-NG: HTTP Parser structural redesign
This update is to lay the groundwork for several planned future projects
implementing protocol specific parsers, seperating the currently
intertwinned client connection management and HTTP protocol parsing
logics, and for zero-copy processing of transactions using SBuf.
This patch renames the HttpParser class as RequestParser and moves it
into the Http::One:: namespace as child of an Http::Parser class with
generic API accessors for shared use by other HTTP message parsers.
The class API
- is updated to process both the request-line and HTTP mime headers,
returning an incomplete parse result until the entire headers parts of
the message have been received.
- now contains accessor methods for retrieving the method, URI,
protocol, mime headers block (as an SBuf) and some metrics about those.
- the old request_offsets structure and similar offset details are no
longer exposed.
- now emits 414 and 431 HTTP status codes as appropriate.
The parser is made partially incremental and some use of Tokenizer is
added for faster parsing. There is more that can be done to further
speed up parsing of slow or very large requests.
Much of the code from client_side.cc parseHttpRequest() and also the
header-field code from mime_headers.cc has been moved into the parser
class. The client_side.cc code now simply runs the main
Http1::RequestParser::parse() method then uses accessors to retrieve
and process the parse results.
A unit test for incremental parsing has been added to testHttpParser.
Also, the HttpRequestMethod class is moved into the Http:: namespace and
library to reduce dependencies on the parser class outside the library.
The gains made so far from incremental parse, reducing parser passes and
zero-copy SBuf are offset by several temporary performance regressions
added converting SBuf for legacy code use. These are marked for later
removal.
Amos Jeffries [Tue, 4 Nov 2014 08:47:03 +0000 (00:47 -0800)]
Bug 1961 part 2: redesign of URL handling.
Move the HttpRequest::login detail into class URL. Renaming to userInfo
as per the RFC 3986 defined name for this URI piece.
Convert the details to SBuf gaining several minor str*() removals in
the process and a simpler FTP login parser based on SBuf capabilities.
Also, updated the base64 encoder API used for converting between URI
userInfo and Basic authentication header token. Gaining better control
over the length of maximum token size and a small speedup from pre-known
input length.
Amos Jeffries [Sun, 2 Nov 2014 00:10:01 +0000 (17:10 -0700)]
Cleanup: Simplify CBDATA API and rename CBDATA_CLASS
There are no logic changes here.
* Rename CBDATA_CLASS2() to CBDATA_CLASS()
* Remove cbdata*Dbg() duplicate interface for CBDATA.
Inline it with the CBDATA_CLASS() API
* update public:/private: class section handling in CBDATA_CLASS() and
place all uses at the top of class definition as with MEMPROXY_CLASS()
and typedef coding style.
- this highlights TunnelStateData API as previously broken in regards
to private members. Leave it public: for now with TODO on fixing.
Amos Jeffries [Sat, 1 Nov 2014 05:04:05 +0000 (22:04 -0700)]
Docs: Add man(8) page for helper-mux tool
Convert the helper-mux.pl.README to inlineperl docs and generate a man(8)
page using pod2man when possible.
Also, shuffle the tool into a sub-directory of its own for easier build
management.
Also, add a automake conditional ENABLE_POD2MAN_DOC to allow optional
build of perl tools or helpers without depending on pod2man which is
only necessary for the documentation.
projects / thirdparty / squid.git / log
