Alex Rousskov [Mon, 30 Jan 2012 17:13:52 +0000 (10:13 -0700)]
Bug 3441: Part 1: Minimize cache size corruption by malformed swap.state.
If swap.state gets corrupted, a single entry with bogus size value will screw
up Squid idea of the current cache size. A newly added StoreSwapLogData sane()
method attempts to minimize the chance of corruption by ignoring log entries
with obviously bogus values.
However, without expensive size checks (-S or "Does the log entry matches the
actual cache file size?"), it is not possible to reliably detect all bogus log
entries.
If Squid gets a wrong idea of the current cache size, it may either cache too
much (and possibly run out of space) OR delete everything.
where <signing algorithm> can be one of the signTrusted, signUntrusted or
signSelf
Default signing algorithm if the sslproxy_cert_sign is not configured is
signSelf, if the server certificate is self signed, signUntrusted if the server
certificate is untrusted (ERR_INVALID_CA, ERR_SELF_SIGNED_CERT_IN_CHAIN,
ERR_UNABLE_TO_VERIFY_LEAF_SIGNATURE, ERR_UNABLE_TO_GET_ISSUER_CERT,
ERR_UNABLE_TO_GET_ISSUER_CERT_LOCALLY and ERR_CERT_UNTRUSTED errors) and
signTrusted if the server certificate is trusted.
Notes:
- The signing algorithm passed as parameter to the ssl_crtd daemon
- A self signed certificate generated on squid startup to be used as
signing certificate for untrusted certificates, with CN =
"Squid CA for Untrusted Certificates"
- The configured certificates with http(s)_port squid.conf option chained to
the client SSL connection only when signTrusted signing algorithm selected.
When the signing algorithm is signSelf or signUntrusted no other certificate
send to the client browser.
- A small bug fixed which did not allow the sslproxy_cert_adapt option
to be enabled in squid if the icap client is not enabled
Properly initialize memory_cache_shared when not explicitly configured and
do not allocate shared memory segments if memory_cache_shared is off.
Prior to this change, shared memory was requested (during rrClaimMemoryNeeds)
before memory_cache_shared was initialized (during rrAfterConfig, unless
explicitly set in squid.conf). Moreover, MemStore::EntryLimit() calculation
ignored memory_cache_shared setting. All that resulted in an attempt to create
shared memory segments where none were needed or possible.
We now have a new Runners Registry (rrFinalizeConfig) that is dedicated to
finalizing complex configuration options, before those configurations options
are used to initialize modules, features, and services. As far as shared
memory is concerned, the initialization order is now:
* rrFinalizeConfig: finalize memory_cache_shared if needed
* rrClaimMemoryNeeds: request shared memory pages for the caches
* rrAfterConfig: create shared memory segments
The same bug 3449 also needed a fix for shared segment paths (trunk r11961).
Alex Rousskov [Thu, 26 Jan 2012 04:51:21 +0000 (21:51 -0700)]
Bug 3268: Squid cannot do anything else during ufs/diskd rebuild
Replaced 50-entries/step UFS rebuild logic with 50msec/step logic,
while also ensuring that other events can be processed between steps.
During a swap.state rebuild, progress reporting will now happen
every 4000 entries instead of every 0xFFF entries because humans do not
think in hex. Still no progress reporting for dirty rebuilds based on the
cache directory scans.
The code is based on Rock Store rebuild code. Eventually, the common rebuild
pacing logic among all stores should be merged to avoid code duplication.
Also removed RebuildState::next methods as unused. A copy-paste error?
Henrik Nordstrom [Sat, 21 Jan 2012 23:04:54 +0000 (00:04 +0100)]
Disable OpenSSL SSL/TLS bug workarounds by default
On a closer inspection the set of "harmless" SSL/TLS bug workarounds
set by SSL_OP_ALL is not all of them harmless and reduces the SSL/TLS
strength to some attacks.
To revert to the older mode the ALL option can be set explicitly, but
it's better to understand which bug is encountered and enable only that
specific workaround if needed.
This patch add the sslproxy_cert_adapt option to squid.conf which gives to
squid administrators the required functionality to "fix" a known broken
certificate using acls.
Currently only the "Not After", "Not Before" and "Common Name" fields of a
certificate can be modified/fixed.
The sslproxy_cert_adapt option has the form:
sslproxy_cert_adapt <adaptation algorithm> acl ...
where <adaptation algorithm> is one of the setValidAfter, setValidBefore and
setCommonName.
setValidAfter: sets the "Not After" property to the signing cert's
"Not After" property.
setValidBefore: sets the "Not Before" property to the signing cert's
"Not After" property.
setCommonName: sets certificate Subject.CN property to the host name
from specified as a CN parameter (setCommonName{CN}) or,
if no explicit CN parameter was specified, extracted from
the CONNECT request
When the acl(s) match, the corresponding adaptation algorithm is applied to
the fake/generated certificate. Otherwise, the default mimicking action takes
place.
Alex Rousskov [Fri, 20 Jan 2012 23:28:14 +0000 (16:28 -0700)]
Do not add HTTP 110 and 111 Warnings to TCP_REFRESH_UNMODIFIED responses.
The old "stale if hit" logic did not account for cases where the stored stale
response became fresh due to a successful revalidation with the origin server.
When the stored response was stale at the time of the request, we were adding
110 "Response is stale" and even 111 "Revalidation failed" Warning headers to
responses while logging TCP_REFRESH_UNMODIFIED, which is considered a hit.
Added StatHist unit tests, and moved algorithm consistency checks there.
Expanded storage for histograms to 64bit unsigned.
Inlined StatHist constructor, destructor and assignment operator.
Implemented stubs for StatHist.cc and mem.cc
Henrik Nordstrom [Mon, 16 Jan 2012 17:30:16 +0000 (18:30 +0100)]
Send DNS A and AAAA queries in parallel
This implements sending DNS A & AAAA queries in parallel by
creating "slave" idns_query requests. Current implementation
uses the A lookup as "master" and AAAA as "slave" query.
Long term this should probably be restructured to separate "lookup
state" and "query", or even better yet to defer the DNS lookups
until connect time and perform A respective AAAA as needed only and
not look up both before attemting to connect.
This also drops the dns_v4_fallback directive as it have no effect with
parallel DNS lookups. This directive should be reinstanciated in future
to enable pure IPv6 usage.
Inside StoreEntry::swapOut() the StoreEntry::trimMemory() method called to
release unused MemObjects memory. The trimMemory method must called for all
store entries, but current code blocks that call for not-swappable objects,
at least.
This patch trying to fix this bug implementing the following simple logic:
{
bool weAreOrMayBeSwappingOut =
swappingOut() || mayStartSwapout();
trimMemory(weAreOrMayBeSwappingOut);
if (!weAreOrMayBeSwappingOut)
return; // nothing else to do
}
In the case where the squid-to-server connection closed and we re-connect to
server we need to check if the server certificate changed.
Moreover we need to check stored certificates with the web server certificate
to handle the case the web server certificate is updated. In this case a new
certificate must generated to mimic the updated server certificate fields
Amos Jeffries [Sat, 7 Jan 2012 10:15:40 +0000 (03:15 -0700)]
Cleanup: update most of the existing stub files to use the STUB.h framework
There are still several sections to be done. Including adding library API
stubs. However these are the ones which can be done immediately without
breaking or re-writing existing unit tests.
This implements sending DNS A & AAAA queries in parallel by
creating "slave" idns_query requests. Current implementation
uses the A lookup as "master" and AAAA as "slave" query.
Long term this should probably be restructured to separate "lookup
state" and "query", or even better yet to defer the DNS lookups
until connect time and perform A respective AAAA as needed only and
not look up both before attemting to connect.
Bump-server-first fails with SQUID_X509_V_ERR_DOMAIN_MISMATCH error
When the bump-server-first used we do not always know the hostname of the server
we are connecting to. Thus the the hostname validity check for the
SSL server certificate will fail with SQUID_X509_V_ERR_DOMAIN_MISMATCH error.
This patch does not check if server certificate is valid for the hostname on
bump-server-first case and move this check after the http request received
from the web client
Amos Jeffries [Sun, 1 Jan 2012 04:48:49 +0000 (17:48 +1300)]
Cleanup: unlinkd is self-contained
This helper is very minimal. It does not use IP:: or Ipc:: symbols.
If we get linkage issues against it for them those need to be resolved
without pulling in dependencies from squid internal libraries.
Amos Jeffries [Fri, 30 Dec 2011 16:01:37 +0000 (05:01 +1300)]
Cache Manager migration support
* Add a little bit of XHR script to the CGI cachemgr front page which
probes each of the managed proxies for http:// and https:// capabilities
and produces web links to their internal managers.
* Reserve the template name MGR_INDEX for use by cachemgr scripts.
But do not distribute any preset template. This allows manager apps to
provide their own static template with linked scripts and objects.
* The error page system is updated to create a blanket message
indicating missing template instead of aborting Squid if a template is
not even installed.
Amos Jeffries [Fri, 30 Dec 2011 03:42:50 +0000 (16:42 +1300)]
Support Cross-Origin Requests for the cache manager API
Now that tools are being implemented to access the cache manager via
http:// scheme we need to accomodate the browser XSS protection
mechanisms which limit XHR based scripts abilities.
This adds CORS headers to manager responses. Permitting XHR to view the
Server header (to detect squid version for known capabilities) and to
flag that the XHR request may need access to credentials for
authenticating with the manager.
This also closes the feature bug 3407 requesting we support the
non-standard "Origin:" header, which is used by the CORS mechanisms.
Future work:
Support the OPTIONS request used by CORS to detect requirements before
POSTing. We do not yet use POST in the Squid code so that is left until
needed.
Amos Jeffries [Fri, 30 Dec 2011 01:24:57 +0000 (18:24 -0700)]
SourceLayout: shuffel UserRequest into Auth:: namespace
No logic changes, symbols and debug only.
* shuffle AuthUserRequest to Auth::UserRequest
* shuffle AuthBasicUserRequest to Auth::Basic::UserRequest
* shuffle AuthDigestUserRequest to Auth::Digest::UserRequest
* shuffle AuthNegotiateUserRequest to Auth::Negotiate::UserRequest
* shuffle AuthNTLMUserRequest to Auth::NTLM::UserRequest
* polish and add documentation for several parts of the hierarchy
The error pages of failed "ssl bump first" requests contains urls in the form:
ssl_peek://hosrtname:443
This patch fixes this problem adding code which creates a new fake HttpRequest
which is a clone of the "ssl_peek" request but sets the hostname to the CN
retrieved from server certificate and protocol to Https.
The fake HttpRequest object passed to the ErrorState object.
Implement the Ssl::CommonHostName name to recurn the CN from a certificate,
suitable for use as a host name.
Use this function to set the ConnStateData::sslHostName in ConnStateData::httpsPeeked method
Amos Jeffries [Thu, 22 Dec 2011 04:49:24 +0000 (17:49 +1300)]
Bug 3370: external ACL sometimes skipping
Emit tag/user/log/message/pass details to the request in the case where
the external ACL entry has expired but within graceful revalidate period.
The result of this bug appears as incorrect matches later down the ACL
processing in any config relying on the external ACL output values.
Example; for bypassing auth login, or for filtering tagged traffic.
Forward a StoreEntry holding the bumped secure connection establishment
error page from the server-side code (which generates the error) to the
client-side code (which delays the error until the first encrypted
request comes). This allows Squid to display the error page to the user
(using secure connection) when bumping intercepted SSL connections. The
code still needs more polishing, including generating errors with host
names and not IP addresses (when possible).
The peeked server certificate is now stored in
ConnStateData::bumpErrorEntry. This allows us to mimic the certificate
of dropped server connections.
Load signing certificate and key when initializing a tproxy-enabled
https_port.
Fixed debugging when opening a peeking connection.
This patch try to mimic true server certificate properties when generating
a fake SSL certificate for SslBump. If ssl_crtd is enabled, it receives the
true server certificate and mimic its properties. Otherwise, the certificate
mimicking code will run in the worker.
Currently the following properties mimicked: subject name, not before/after,
and subject alternate name.
Moved some more typedefs to StatHist.h
Improved documentation
Removed squid.h include
Changed StatHist::operator= to almost be an actual assignment operator
Fixed findBin corner-case
When the ssl_bump and authentication iare both enabled for an ssl-bumped port
all SSL enabled websites prompt the user for authentication information once
per FQDN.
Tis patch inherits the authentication info from the CONNECT request to the
sslbumped requests.
Alex Rousskov [Wed, 14 Dec 2011 18:24:29 +0000 (11:24 -0700)]
Peek at the origin server SSL certificate when bumping intercepted HTTPS.
* Configuration changes:
Allow intercepted SSL connections to be bumped, in addition to the
tproxied SSL connections.
Honor and check ssl-bump flag in https_port. Earlier code apparently
assumed that the flag must be present in http_port and left
Ssl::TheGlobalContextStorage uninitialized if only https_port had the
flag.
* Client-side changes:
Added a new Ssl::ServerPeeker class to do client-side error handling
while peeking at the origin server certificate. Peeking is done at the
server-side. Server-side uses Store and store_client API to report
errors to the client-side. That works OK when the errors can be sent to
the client, but when we bump intercepted connections, the client does
not yet have a secure connection established with Squid so errors cannot
be sent (popular browsers do not display CONNECT-stage errors). Instead,
the errors must be accumulated and sent after the secure connection with
the client is established (in response to the first HTTP request on that
connection). Ssl::ServerPeeker needs work to support such accumulation.
Start Ssl::ServerPeeker job in ConnStateData::switchToHttps() and wait
for somebody to call the new ConnStateData::httpsPeeked() method back.
Needs more work to actually use the peeked certificate and handle
errors.
Changed ConnStateData::switchToHttps() profile to require destination
port number. Without it, we cannot switch intercepted SSL connections
because they do not have a proper request structure that can supply port
details.
Polished pinned connection cleaning code: If our Comm close handler for
the pinned connection was called, do no try to remove the handler. If
the pinned connection was closed (e.g., by a server-side error), do not
try to close it again. If we already called unpinConnection(), do not
try to close the pinned connection again.
Do not assume we have a request when pinning a connection to the server.
Intercepted connections do not have requests at the connection pinning
stage.
* Server-side changes:
Bug 3243 (CVE 2009-0801: Bypass of browser same-origin access control in
intercepted communication) fix always created a new connection to the
origin server. I think it is safe (and possibly even safer!) to reuse a
pinned connection instead (if one is available). We now do that in the
new FwdState::selectPeerForIntercepted() method. If bump-server-first
does not reuse a pinned connection (left from certificate peeking),
Squid would be opening and closing to-server connections just to learn
the certificate, which is not kosher.
Added a new internal protocol type (PROTO_SSL_PEEK) to allow FwdState to
detect peeking requests and end processing after the certificate is
received (instead of proceeding with to httpStart()).
Bug 3448: 204 response problem in adaptation chains
When the first ICAP service in a chain respond with 204 the next service
is aborted on Must(old_request->canonical) expression inside Adaptation::Icap::ModXact::encapsulateHead method.
Squid ICAP try to set the request::canonical member of the adapted request
inside Adaptation::Icap::ModXact::prepEchoing when the 204 response received.
The adapted.header->parse(..) call some lines after will set canonical member
to NULL.
This patch call the urlCanonical() function after parse() method
to build canonical member for the adapted request, instead of trying to copy
this member from the original request.
projects / thirdparty / squid.git / log
