Alex Rousskov [Wed, 14 Dec 2011 18:24:29 +0000 (11:24 -0700)]
Peek at the origin server SSL certificate when bumping intercepted HTTPS.
* Configuration changes:
Allow intercepted SSL connections to be bumped, in addition to the
tproxied SSL connections.
Honor and check ssl-bump flag in https_port. Earlier code apparently
assumed that the flag must be present in http_port and left
Ssl::TheGlobalContextStorage uninitialized if only https_port had the
flag.
* Client-side changes:
Added a new Ssl::ServerPeeker class to do client-side error handling
while peeking at the origin server certificate. Peeking is done at the
server-side. Server-side uses Store and store_client API to report
errors to the client-side. That works OK when the errors can be sent to
the client, but when we bump intercepted connections, the client does
not yet have a secure connection established with Squid so errors cannot
be sent (popular browsers do not display CONNECT-stage errors). Instead,
the errors must be accumulated and sent after the secure connection with
the client is established (in response to the first HTTP request on that
connection). Ssl::ServerPeeker needs work to support such accumulation.
Start Ssl::ServerPeeker job in ConnStateData::switchToHttps() and wait
for somebody to call the new ConnStateData::httpsPeeked() method back.
Needs more work to actually use the peeked certificate and handle
errors.
Changed ConnStateData::switchToHttps() profile to require destination
port number. Without it, we cannot switch intercepted SSL connections
because they do not have a proper request structure that can supply port
details.
Polished pinned connection cleaning code: If our Comm close handler for
the pinned connection was called, do no try to remove the handler. If
the pinned connection was closed (e.g., by a server-side error), do not
try to close it again. If we already called unpinConnection(), do not
try to close the pinned connection again.
Do not assume we have a request when pinning a connection to the server.
Intercepted connections do not have requests at the connection pinning
stage.
* Server-side changes:
Bug 3243 (CVE 2009-0801: Bypass of browser same-origin access control in
intercepted communication) fix always created a new connection to the
origin server. I think it is safe (and possibly even safer!) to reuse a
pinned connection instead (if one is available). We now do that in the
new FwdState::selectPeerForIntercepted() method. If bump-server-first
does not reuse a pinned connection (left from certificate peeking),
Squid would be opening and closing to-server connections just to learn
the certificate, which is not kosher.
Added a new internal protocol type (PROTO_SSL_PEEK) to allow FwdState to
detect peeking requests and end processing after the certificate is
received (instead of proceeding with to httpStart()).
Bug 3448: 204 response problem in adaptation chains
When the first ICAP service in a chain respond with 204 the next service
is aborted on Must(old_request->canonical) expression inside Adaptation::Icap::ModXact::encapsulateHead method.
Squid ICAP try to set the request::canonical member of the adapted request
inside Adaptation::Icap::ModXact::prepEchoing when the 204 response received.
The adapted.header->parse(..) call some lines after will set canonical member
to NULL.
This patch call the urlCanonical() function after parse() method
to build canonical member for the adapted request, instead of trying to copy
this member from the original request.
Author: Alex Rousskov <rousskov@measurement-factory.com>
Bug 3420: Request body consumption races and !theConsumer exception.
Also fixes endless waiting for HTTP client to send req body we no longer need.
Before these changes, the client side used a single "closing" state to
handle two different error conditions:
1. We stopped receiving request body because of some error.
2. We stopped sending response because of some error.
When a "directional" error occurred, we try to keep the transaction going in
the other direction (e.g., to give ICAP the entire request or to give HTTP
client the entire response). However, because there was just one "closing"
state, the code failed to correctly detect or process many corner cases,
resulting in stuck transactions and !theConsumer assertions/exceptions due to
races between enableAutoConsumption() and expectNoConsumption() calls.
This patch replaces the "closing" state with two direction-specific "we
stopped sending/receiving" flags.
Now, when the response sending code is done, it now checks whether the
receiving code stopped and closes the connection as needed. This is done both
when we encounter a sending error (ClientSocketContext::initiateClose) and
when we successfully sent the entire response to the client
(ClientSocketContext::keepaliveNextRequest).
Similarly, when the request body reading code is done, it now checks whether
the receiving code stopped and closes the connection as needed. This is done
both when we encounter a receiving error
(ConnStateData::noteBodyConsumerAborted) and when we successfully receive the
entire request body from the client (ClientSocketContext::writeComplete).
TODO: This patch focuses on various error cases. We might still have problems
when there is an early HTTP response and no errors of any kind. I marked the
corresponding old code with an XXX.
External ACL sometimes cannot find the credentials in ACL Checklist even
if they are attached to the HTTPRequest object.
This seems to happen when the checklist is created and the line match
started before the credentials are known. The credentials validation
updates the HTTP request state but is not aware of ACL checklists needing
to be updated so it never happens.
This patch:
* locate the %LOGIN value from either place where credentials can be found,
* updates the checklist if it was unset,
* passes '-' to the helper if no credentials at all were given.
Although the earlier logics forcing a lookup means this '-' case should
not happen it might if the external ACL were processed in 'fast' check.
Amos Jeffries [Sun, 4 Dec 2011 05:43:42 +0000 (22:43 -0700)]
Add FdeCbParams parameter object to CommCalls API.
The problem:
CommCalls API functionality is conflated with comm operational calls
created to do general FD handling (FD as pipe handle, FD as disk handle,
FD as pointer into the fd_table structure). Sometimes because they do
operations mirroring comm handlers and also use FD. None of this actually
requires the CommCalls layer to be involved though. The Comm::Connection
objects which CommCall TCP handlers pass around is also very inappropriate
for these FD types.
This adds FdeCbParams to CommCalls infrastructure, for use internally and
"lower" than comm API to pass around raw FD values. This should be avoided
on TCP socket FD, but may be used by callers needing FD where
Comm::Connection is inappropriate.
Amos Jeffries [Sun, 4 Dec 2011 05:39:39 +0000 (22:39 -0700)]
CBDATA call Dialer template
This adds a template for dialing Unary CBDATA wrapper functions with
type safety. Avoiding the casting that currently occurs in wrappers and
allowing the AsyncCall APIs to be used for these callbacks.
Alex Rousskov [Sat, 3 Dec 2011 22:45:38 +0000 (15:45 -0700)]
Honor ssl-bump option for https_port.
Initial ssl-bump handling logic mimics that of http_port: If the option is
set, check the slow ssl_bump ACL, and if there is a match, plug into
switchToHttps() code path, generating a dynamic certificate and establishing a
secure connection with the client. If there is no match, Squid becomes a TCP
tunnel for the intercepted connection.
For now, we use the destination IP address of the intercepted connection as
the host name for the certificate (which will trigger browser warnings, of
course).
Bug fix: HttpRequest::flags.intercepted, HttpRequest::flags.spoof_client_ip neve
r set
The request_flags::intercepted,request_flags::spoof_client_ip are 1 bit integers
so when you are try to set to an integer bigger than 1 will overflow and the
results will not be what you are expecting.
Amos Jeffries [Tue, 22 Nov 2011 12:00:59 +0000 (01:00 +1300)]
Cleanup: comm Close handlers
Make handlers take the CommCloseCbParams instead of series of expanded
variables.
Opening access to the other CommCommonCbParams fields with Connection/FD
data. Hiding the deprecated FD parameter from most handlers. Which seem
not to have actually needed it in most cases outside Comm.
The StoreEntry::write in the case of an empty write, calls the StoreEntry
handlers. It is possible one of these handlers will change the state of the
store entry or abort it. The next call of the StoreEntry::write will cause
this assertion.
The block of code which calls the StoreSntry handlers in the case of an empty
write, added to allow forward http headers to the client even if no body data
arrived yet (bug 1750). There is not need for this part of code in the latest
squid releases, so it is safe to be removed.
Alex Rousskov [Mon, 21 Nov 2011 16:49:34 +0000 (09:49 -0700)]
Avoid crashes when processing bad X509 common names (CN).
X509_REQ_get_pubkey() returns a refcounted object that we must clean after use.
X509_REQ_get_subject_name() does not; cleaning the result may cause segfaults.
How we are supposed to tell the difference is beyond me.
author: Martin Huter <mhuter@barracuda.com>, Alex Rousskov <rousskov@measurement-factory.com>, Christos Tsantilas <chtsanti@users.sourceforge.net>
Bug 2619: Excessive RAM growth due to unlimited adapted body data consumption
If the client does not read from the open connection (i.e. the user does not
confirm the browsers download-message-box in microsofts IE), squid keeps on
reading data from the ICAP server into the store entry, while no more data
can be delivered to the client.
Thus the store entry in memory is growing and squid may - in worst case -
consume memory up to the size of the users download.
This patch add API to StoreEntry to call the producer back when released
memory/space from the StoreEntry and add code to the ICAP client code to not
consume body data comes from the ICAP server when there is not available space
in the store entry.
projects / thirdparty / squid.git / log
