Abort with an error when a wildcard entry is going to be
discarded because of a sub-domain entry.
Also whenever there is a mixup between a domain and its
sub-domain wildcard alternative.
Raise a non-fatal warning when a useless subdomain entry
is being discarded and its super-set wildcard kept.
Care is taken to present the singular subdomain for
possible removal and keep the wildcard.
Amos Jeffries [Wed, 30 Mar 2011 12:38:06 +0000 (06:38 -0600)]
squidclient: send cachemgr password via -w option
Preparation for internal cachemgr updates to use real proxy-auth.
The cachamgr password may now be sent in three ways:
Deprecated: mgr:info@password
Current Option: -w password mgr:info
Preferred: -u username -w password mgr:info
The old explicit @ syntax is now deprecated for visible use. The background
systems will still send it that way for cache_object: URLs. Use of this
overrides any -w option set. So it is still possible to login to a proxy
with one set of credentials and pass a separate password to the cachemgr.
The long-term plan is to drop @ completely in future.
The current option of just -w will convert the password to @ syntax in the
background but not add Proxy-Authentication headers. This may die in future.
The preferred alternative is to use -u and -w which triggers addition of real
Proxy-Authenticate headers. The username is not yet used by cachemgr but
may be required by the proxy ACL configuration.
Alex Rousskov [Wed, 30 Mar 2011 12:02:11 +0000 (06:02 -0600)]
Bug 2621: Provide request headers to RESPMOD when using cache_peer.
A short-term fix.
When FwdServer::_peer is set, HttpStateData constructor creates a new special
HttpRequest, overwriting the request pointer set in the parent
(ServerStateData) constructor to fwd->request.
To make matters worse, this special peer request has no headers at all (even
though flags and some cached/computed header values are copied). We initialize
it with the right URL, method, and protocol. We copy flags and a few other
random properties from the original request. We never copy the original
headers.
Furthermore, regardless of the peering, when we create the headers to send to
the next hop, those headers are temporary and not stored in any request
structure (see HttpStateData::buildRequestPrefix). The non-peering code
survives this because the request member points to fwd->request, which has the
headers. The peering code fails as illustrated by this bug.
I believe both cases are buggy because server-side adaptation and core code
should have access to the request headers we sent rather than the request
headers we received and adapted (or no headers at all). After all, it is the
sent headers that determine the next hop view of our Squid and adaptation
services should see a pair of _matching_ request and response headers.
I am pretty sure there are other bugs related to HttpStateData using a special
peer request structure instead of fwd->request. Please note that FwdState has
no idea that this substitution is going on.
This quick short-term fix uses the original request and its headers when
checking RESPMOD ACLs. This is what the patch in bug #2562 did for Squid v3.0.
For the reasons described above, this patch may be either insufficient or
wrong for the long-term fix.
Alex Rousskov [Wed, 30 Mar 2011 11:53:48 +0000 (05:53 -0600)]
Bug 3173: Assertion bodyPipe!=NULL on SslBump CONNECT response writing failure
Do not call ConnStateData::startClosing() when we fail to write our CONNECT
response while bumping a connection. startClosing() can only be used when we
handle response bodies. Just close the connection, in hope that the connection
close handler kicks in and cleans up.
Frank Schmirler [Mon, 28 Mar 2011 11:31:28 +0000 (05:31 -0600)]
Bug 2330: AuthUser objects are never unlocked
This is a partial port of the Buug 2305 auth fixes.
These changes involve combining the auth operation state links to
credentials data such that the shared code can lock/unlock them properly.
Amos Jeffries [Tue, 22 Mar 2011 12:04:26 +0000 (06:04 -0600)]
Bug 2976: invalid URL on intercepted requests during reconfigure
Listening ports abuse the cbdata type as a pseudo refcount. This breaks
during reconfigure when the config is erased and the active requests
handles all become invalid pointers.
Interception only works on HTTP protocol. We can hard-code the scheme
and avoid this problem until a complete fix is written.
Amos Jeffries [Mon, 7 Feb 2011 12:16:27 +0000 (05:16 -0700)]
Author: Jonathan Wolfe <jonathan.wolfe@gmail.com>
Bug 3149: not caching ecap adapted body
eCAP adapters may remove or edit the Content-Length header, and to allow
caching of the adapted body squid must reflect that new (or unspecified)
content length when deciding whether or not the adapted body can be cached.
Amos Jeffries [Tue, 1 Feb 2011 02:31:49 +0000 (19:31 -0700)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Fix IP/FQDN cache accounting to avoid idle caches on busy servers.
When maintaining the IP/FQDN cache size, use the number of entries in the cache
rather than the number of allocated and not freed MEM_IPCACHE_ENTRY and
MEM_FQDNCACHE_ENTRY objects. These objects are used outside the cache
for DNS queries. If queries leak (or perhaps when there are just a lot of them),
the memory-pool-based count overestimates the cache size, sometimes to
such a degree that the cache remains nearly empty despite lots of misses.
Use memory-pool-based counter to estimate cache size also violates IP/FQDN cache
encapsulation boundaries because it effectively prevents others from using
the same memory pool.
Amos Jeffries [Tue, 1 Feb 2011 02:17:05 +0000 (19:17 -0700)]
Author: Fabian Hugelshofer <fh@open.ch>
Allow persistent connections for Mozilla/3.0 User-Agents
This fixes NTLM and Negotiate authentication for these agents.
History:
In 1998 a hack was added to HttpMsg::persistent() that disables
persistent connections for HTTP/1.0 User-Agents starting with
"Mozilla/3." and "Netscape/3.".
According to the thread on squid-dev
(http://www.eu.squid-cache.org/mail-archive/squid-dev/199805/0087.html),
this was necessary to make some versions of Netscape browsers work that
had a broken implementation of persistent connections. It was said that
"NS 3.01 is ok. NS 3.02 is bad. NS 3.04 is good." Netscape 4 was ok, too.
Amos Jeffries [Tue, 1 Feb 2011 02:15:16 +0000 (19:15 -0700)]
Fix external_acl_type grace= option
Due to race conditions between concurrent requests this is still not a
complete fix. But reduces the unwanted re-use of helper responses from
all connections arriving in a whole second which match the ACL key down
to just those that arrive within the reply lag time of the helper.
Amos Jeffries [Tue, 1 Feb 2011 01:50:29 +0000 (18:50 -0700)]
Author: Henrik Nordstrom <henrik@henriknordstrom.net>
Simplify request parsing to not check request method when determining entities
Requests containing a request-entity or not is signalled entirely by
Content-Length/Transfer-Encoding regardless of method.
Also drops the requirement that PUT/POST requests must have a request-entity.
The RFC do not explicitly state this requirement even if the wording for those
methods do assume there is a enclosed request-entity.
The administrative "request_entities" config flag is kept for security
reasons, even if not really RFC compliant. (RFC meaning of request-entity
in GET/HEAD is just undefined or "ignored", not forbidden)
Amos Jeffries [Tue, 1 Feb 2011 01:37:14 +0000 (18:37 -0700)]
ftp_eprt directive to disable EPRT extensions in FTP
This allows admin to resolve compatibility problems with old devices which
encounter a range of problems when FTP extensions are used by selectively
disabling any of the extensions individually.
The other EPSV extensions already have enable/disable directives.
Amos Jeffries [Tue, 1 Feb 2011 01:33:52 +0000 (18:33 -0700)]
Bug 2959: remove SAMBAPREFIX dependency
This removes the tricky SAMBAPREFIX variable which passes full-path
information from the squid build machine down to the run-time host
helper.
Such information is not always correct when crossing machines, and the
binaries being run can easily be added to PATH in the run-time host
environment instead.
The net result of doing this is removal of Samba from the build
dependencies and increased availability of the smb_auth and
wbinfo_group.pl helpers.
Amos Jeffries [Sun, 30 Jan 2011 05:24:12 +0000 (22:24 -0700)]
Compat: static functions cannot be passed externally with some compilers
Solaris StudioCC, HPUX and old GCC complain about statics being passed
by pointers outside the current file.
Elected to drop 'static' instead of adding a bunch of specific wrapper hacks.
Amos Jeffries [Sun, 30 Jan 2011 05:20:41 +0000 (22:20 -0700)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Handle early eCAP transaction failures better.
Do not throw an exception if eCAP transaction had to deal with a virgin body
but was not consuming it at swangSong() time. This may happen if the eCAP
adapter throws an exception before the adapter requests the virgin body
transmission or after it stops the transmission. In other words, the
transaction wrapper consumes only if proxyingVb is on.
Amos Jeffries [Fri, 14 Jan 2011 06:30:28 +0000 (23:30 -0700)]
HTTP/1.1 support: Send 307 status on deny_info redirection
This makes Squid send an HTTP/1.1 307 status response to 1.1+ clients if
the deny_info directive is used to redirect non-GET/HEAD requests.
Current behaviour is to use a 302, which browsers will prevent
displaying for security protection against injection attacks. Using 307
will give browsers a better chance to identify the redirects and handle
them safely.
Amos Jeffries [Fri, 14 Jan 2011 05:02:13 +0000 (22:02 -0700)]
Author: Henrik Nordstrom <hno@squid-cache.org>
Support RFC 5861 Cache-Control: stale-if-error option
The default behaviour for Squid is to present the stale object when
revalidation fails with a 5xx error.
stale-if-error places a maximum limit on how long this stale object may
be sent. After the limit has passed Squid is required to present the 5xx
message to the client.
Original code for Squid-2 was sponsored by Yahoo!.
Amos Jeffries [Tue, 21 Dec 2010 00:53:56 +0000 (17:53 -0700)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Avoid comm_read "!fd_table[fd].closing()" assertion after adaptation ACL check
The assertion was hit if Server fd was closed while we were checking
adaptation ACLs, and we have not been notified of the closure yet (because the
Adaptation::AccessCheck callback is not async while closure notification is).
Amos Jeffries [Tue, 21 Dec 2010 00:52:56 +0000 (17:52 -0700)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Polished HttpStateData::persistentConnStatus() code. No functionality changes.
Do not check for flags.headers_parsed. The removed check was:
- misplaced: connection-related conditions such as eof must be checked first;
- wasteful: we never call persistentConnStatus() unless we parsed headers.
Moreover, calling persistentConnStatus() before we parse headers would trigger
and assertion because the method uses virginReply() which does not exist until
the headers are parsed.
Moved virginReply() call closer to the first virgin reply use. This will help
re-adding "did we parse the header yet" check if we ever need it again. It
also saves a couple of CPU cycles for some transactions.
Amos Jeffries [Tue, 21 Dec 2010 00:42:37 +0000 (17:42 -0700)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Prevent memory leaks when Adaptation::AccessCheck callback ends the job.
The AccessCheckCallbackWrapper is used in nonBlockingCheck() and is called
from the ACL code, using legacy function-based API. If the job ends during
the callback processing, there are no AsyncCall wrappers to destroy the job
object. We now convert legacy to async call to enable proper wrapping and job
destruction.
These kind of job leaks are invisible to valgrind, but that is another bug.
Amos Jeffries [Mon, 20 Dec 2010 23:44:21 +0000 (16:44 -0700)]
Reduce debug level on bodypipe re-write change
- the original reason for adding is unknown
- it is an annoyance for some
- there have been no big problems tracked down to this bodypipe change
over the last few years. It appears relatively harmless.
Amos Jeffries [Mon, 20 Dec 2010 13:37:07 +0000 (06:37 -0700)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Author: Dmitry Kurochkin <dmitry.kurochkin@measurement-factory.com>
Bug 427: HTTP Compliance: Support If-Match and If-None-Match requests.
Add support for If-Match and If-None-Match headers as described in RFC 2616
(sections 14.24 and 14.26 in particular).
Moved IMS handling from clientReplyContext::cacheHit() to
clientReplyContext::processConditional() while preserving the original IMS
logic, except for the case when a request has both IMS and If-None-Match.
Co-Advisors test cases:
test_clause/rfc2616/ifMatch-mismatch-strong
test_clause/rfc2616/ifMatch-mismatch-weak
test_clause/rfc2616/ifNoneMatch-match-imsNone
and many more
Amos Jeffries [Mon, 20 Dec 2010 06:12:57 +0000 (23:12 -0700)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
HTTP/1.1: do not forward TRACE with Max-Forwards: 0 after REQMOD
Before the change, Max-Forwards request value was cached in
HttpRequest::max_forwards member. It was set once in
clientProcessRequest() function. This works fine as long as no request
adaptation is performed. Otherwise original HTTP request may be
replaced with adopted one in ClientHttpRequest::noteAdaptationAnswer()
method and max_forwards value is lost.
This change removes HttpRequest::max_forwards member and gets the value
directly from HttpHeader when needed. This adds another string-to-int
conversion for TRACE and OPTIONS requests, but those are rare, and we
save a little in the other, far more common cases by removing the
HttpRequest::max_forwards member.
Removed assertion from clientReplyContext::traceReply() since it is
called from a single place and the condition is checked right before
the call.
Co-Advisors test cases:
test_case/rfc2616/maxForwardsZero-TRACE-asterisk
test_case/rfc2616/maxForwardsZero-TRACE-absolute
and response_is_fresh is always false if freshness_lifetime is zero.
The check code was introduced in r5998 with a "Import of fix-ranges
branch" message. The code was commented out at the time of that
commit, for reasons unknown.
Test case:
test_case/rfc2616/noSrv-hit-stale-max-age-req
Amos Jeffries [Sat, 18 Dec 2010 12:46:13 +0000 (05:46 -0700)]
Author: Henrik Nordstrom <henrik@henriknordstrom.net>
Make bootstrap.sh use system default autotools versions instead of searching
- can be overridden on commandline if needed. See bootstrap.sh for details.
- Update autoconf to 2.68
- Rename configure.ac to match current autotools standards
Amos Jeffries [Fri, 17 Dec 2010 19:46:13 +0000 (12:46 -0700)]
Upgrade process for obsolete options
One problem we currently have with upgrades is leaving the parser able
to avoid its bungled/unknown option message for directives which have
been fully removed or massively syntax altered.
We are able to handle this for flags and option syntax easily but the
parser has been particularly dense and strict on the directives (first
word of each line).
This patch updates the cf_* and cfgman code to allow a special directive
type "obsolete" which causes these directives to be handled specially
without causing the directives to remain in the publicly visible
squid.conf documentation.
It allows DOC_START / DOC_END comments to be written in cf.data.pre
describing the upgrade actions that need to be taken. This text is
dumped to cache.log verbatim when the configuration option is sighted.
If "-k parse" is used the text is displayed at debug level 0, otherwise
displayed at debug level 1. One line indicating a generic "directive X
is obsolete" is always displayed at level 0 for backwards compatibility
with admin expectations of a high level "bungled" message.
After all this text display, parse_obsolete(char*) is called with the
directive name. This function exists in cache_cf.cc and can be coded to
selectivey do more complex handling of the directive. ie for upgrade
actions deeper than removal.
* cf.data.pre has entries added for all the 2.6-3.1 directives I could
find that were removed.
Amos Jeffries [Fri, 17 Dec 2010 18:56:56 +0000 (11:56 -0700)]
Author: Graham Keeling <graham@equiinet.com>
Bug 3113: Squid can eat far too much memory when uploading files
Problem description:
Uploading a large file to a web site on the internet, squid's client
input buffer will increase far faster than it can be emptied to
the target website, and the machine will swiftly run out of memory.
This patch adds the client_request_buffer_max_size configuration
parameter which specifies the maximum buffer size of a client request.
Amos Jeffries [Mon, 6 Dec 2010 02:14:40 +0000 (19:14 -0700)]
Author: Stefan Fritsch <sf@sfritsch.de>
Bug 3096: Squid destroys CbDataList<DeferredRead> objects too late
When server download speed exceeds client download speed, Squid creates a
CbDataList<DeferredRead> object and associates a comm_close handler with it.
When the server kicks the deferred read, the comm_close handler is canceled.
This create/cancel sequence happens every time the server-side code wants to
read but has to wait for the client, which may happen hundreds of times per
second.
Before this change, those canceled comm_close handlers were not removed from
Comm until the end of the entire server transaction, possibly accumulating
thousands of CbDataList<DeferredRead> objects tied to the socket descriptor
via the canceled but still stored close handler.
comm_remove_close_handler now immediately removes canceled close handlers to
avoid their accumulation.
Amos Jeffries [Sun, 7 Nov 2010 10:04:54 +0000 (03:04 -0700)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Bug 3091: Bypassed ICAP errors are not counted as service failures.
Notify ICAP service about the failure even if we can bypass it. Otherwise,
a failing service may continue to stay "up", preventing Squid from using a
healthy backup alternative in a service_set (or bypassing ICAP completeley).
Amos Jeffries [Mon, 1 Nov 2010 05:41:38 +0000 (23:41 -0600)]
Bug 3090: Polish FTP login error handing
Reverts a regression added recently that blocked the challenge events.
Fixes another potential nul-pointer dereference bug.
* 421/426 server overload equate to HTTP overload. But do special such that
the credentials are asked of the browser on retries.
* 43x and 53x FTP status are all credentials failures of various types.
Other failures are not credential related.
This leaves the other non-credential errors as general failures.
Amos Jeffries [Sat, 23 Oct 2010 13:54:37 +0000 (07:54 -0600)]
Author: Christophe Saout <christophe@saout.de>
Bug 3084: IPv6 without Host: header in request causes connection to hang
accel and intercept mode URL re-generation used NtoA instead of ToHostname.
This results in the URL incorrectly wrapping the raw-IPv6 and problems
connecting to non-existent addresses in some cases.
projects / thirdparty / squid.git / log
