* Mostly adding DEFAULT_DOC directive to hide strange default values.
This wll help us move to different internal values for no-limit etc
at some point in the future.
* Migrates some access controls to "DEFAULT: none" instead of "deny all".
This reduces run-time CPU cycles running useless ACL tests.
NP: some access controls have been left unchanged due to complexity in
the code testing them (ie icap_access).
* added documentation for several directives which were missing text
* corrected buffered_logs documentation (text by Alex Rousskov)
* updated cf_gen tool to produce more descriptive error messages
* corrected icp_access default to match documented 'deny all' permission
Alex Rousskov [Thu, 25 Apr 2013 15:47:40 +0000 (09:47 -0600)]
Prevent external_acl.cc "inBackground" assertion on queue overloads.
The enqueue check for external ACL lookups was inconsistent with the final
queue length check in ExternalACLLookup::Start(). The former allowed adding to
the already full (but not yet overflowing) queue while the latter
rightfully(?) asserted that the queue should not overflow.
The SSL_get_certificate implementation in OpenSSL 1.0.1d and 1.0.1e releases,
will crash if called before the certificate sent to the client.
This patch add a hack when one of the problematic OpenSSL versions used to
retrieve the certificate directly from SSL_CTX object, instead of creating
a temporary SSL object, and call SSL_get_certificate.
Some systems like GNU Hurd provide the mmap() API but lack MAP_NORESERVE
support. This option is an optimization, so we can define the macro
ourselves to nil and apparently not suffer (many) bad side effects.
Bug 3817: Memory leak in SSL cert validate for alt_name peer certs
Inside function Ssl::matchX509CommonNames which checks a domain name against
certificate common name and alternate names, if the domain matches any of the
alternate names the function return without releasing allocated data.
Michal Luscon [Thu, 18 Apr 2013 05:30:47 +0000 (23:30 -0600)]
Bug 3825: basic_ncsa_auth segfaulting with glibc-2.17
It appears the crypt() function may return NULL strings. Check for those
before all strcmp() operations.
NOTE: The MD5 output checks are probably not needed but since SquidMD5 is
an object build-time switched between several encryption library API
definitions it is better to be safe here as well.
Fix enter_suid/leave_suid build errors in ip/Intercept.cc
Intercept.cc:210: error: 'enter_suid' was not declared in this scope
Intercept.cc:217: error: 'leave_suid' was not declared in this scope
We should just be including tools.h, but on some GCC the src/tools.h and
src/ip/tools.h include locations clash when building inside src/ip/.
For now we must reference the src/tools.h by its long path.
This adds support for the PF 'divert-to' target which presents the
client and remote IPs directly to Squid in accept() parameters the
way Linux TPROXY target does.
It also adds support for the SO_BINDANY option on outgoing traffic for
client IP address spoofing which completes the TPROXY behaviour.
To enable these features Squid built with --enable-pf-transparent can
be configured with:
http_port 1234 tproxy
There is no reason why manager access should be excluded from CONNECT and
Safe_ports security checks. Also, under the new design manager ACL is a
REGEX pattern test, which may be quite slow.
Overall there should be better performnce under certain DoS condtions
having the manager tests after the port tests, with no change under the
other more common traffic.
Amos Jeffries [Fri, 29 Mar 2013 05:59:17 +0000 (23:59 -0600)]
Fix memory leaks in ConnStateData pinning
ConnStateData does not cleanup any of the pinning child structure fields.
This results in a leak of pinned host and peer details.
Run the full un-pin cleanup operation from ConnStateData::swanSong in
order to clean up all the pinning state instead of just the release the
Comm::Connection.
This fix keeps Rock cache_dirs visible to Cache Manager after reconfigure.
We still lack proper support for complex reconfiguration changes involving
cache_dirs.
Amos Jeffries [Fri, 29 Mar 2013 05:55:45 +0000 (23:55 -0600)]
HTTP/1.1: partial support for no-cache and private= controls with parameters
Since we now support HTTP/1.1 storage and revalidation of
Cache-Control:no-cache it is important that we at least detect the cases
where no-cache= and private= contain parameters.
These are likely still rare occurances due to the historic lack of
support. So for now Squid just detects and exempts these responses from
the caching performed. The basic framework for adding handling of the
header lists is made available but not at this time used.
Amos Jeffries [Thu, 14 Mar 2013 11:32:19 +0000 (05:32 -0600)]
Fix SSL Bump bypass for intercepted traffic
The SSL-bump bypass code on intercepted HTTPS traffic generates a fake
CONNECT request from the original destination IP:port in an attempt to
trigger a TCP tunnel being opened for the un-bumped data to be
transferred over.
The current implementation breaks in two situations:
1) when IPv6 traffic is intercepted
The URL field generated does not account for the additional []
requirements involved when IPv6+port are combined.
The resulting fake requests look like:
CONNECT ::1:443 HTTP/1.1
Host: ::1
... which are both invalid, and will fail to parse. Breaking IPv6 HTTPS
interception bypass.
Resolve this by using Ip::Address::ToURL() function which was created
for the purpose of generating URL hostnames from raw-IP + port with
the bracketing inserted when required.
2) when a non-443 port is being intercepted
The Host: header generated is missing the port and Squid Host: header
validity will reject the outbound
CONNECT 127.0.0.1:8443 HTTP/1.1
Host: 127.0.0.1
... this is an invalid request. Squid is currently ignoring the Host
header. However Squid tunnel.cc does make use of peering and may relay
the fake request Host: to upstream peers where we cannot be so sure what
will happen.
Resolve this issue by re-using the generated IP:port string for both URL
and Host: fields, which preserves teh port in Host: regardless of value.
This also means there is an unnecessary :443 tagged on for most HTTPS
traffic, however the omission of port from the Host: header is only a MAY
and this should not cause any issues.
Amos Jeffries [Fri, 8 Mar 2013 02:05:39 +0000 (19:05 -0700)]
Regression fix: Accept-Language header parse
When handling error page negotiation the header parse to detect language
code can enter into an infinite loop. Recover the 3.1 series behaviour
and fix an additional pre-existing off-by-1 error.
The errors were introduced in trunk rev.11496 in 3.2.0.9.
Amos Jeffries [Mon, 4 Mar 2013 20:08:47 +0000 (13:08 -0700)]
Fix authentication headers sent on peer digest requests
Cache digest fetches have been sending the cache_peer login= option
value without sanitizing it for special-case values used internally
by Squid. This causes authentication failure on peers which are checking
user credentials.
The cases which were earlier causing a lot of RAM 'leaks' have been
resolved already and the remaining causes appear to all be in components
with short packet handling pathways where the orphan is not wasting much
in the way of RAM bytes or FD time.
The trace is left at level-4 for future debugging if necessary.
Amos Jeffries [Fri, 1 Mar 2013 10:01:57 +0000 (03:01 -0700)]
MacOS: reduce the testRock unit test UDS path
On MacOS shm_open() requires the name entry to be less than 31 bytes
long. The garbage name used by testRock was 35 bytes and not really
describing what it was used for in the test anyway.
TODO: find out and fix why MacOS still responds EINVAL once the path
is set to a usable length.
Amos Jeffries [Tue, 26 Feb 2013 22:21:22 +0000 (15:21 -0700)]
Bug 3720: SourceLayout: shuffle fd_table definition into fde.h
Shift the definition out of globals.h into fde.h where the type class
is defined, and the instance into fde.cc. Fixing bug 3720; build errors
on OpenIndiana and Solaris.
Also, move it into the fde class scope as a static Table member.
Provides wrapper definition of fd_table to reduce patch impact.
Amos Jeffries [Mon, 25 Feb 2013 03:47:25 +0000 (20:47 -0700)]
Bug 3794: MacOS: workaround compiler errors and case-insensitivity
MacOS GCC version implicitly searches the local directory for .h
includes despite the absence of -I. in the provided options.
Furthermore it searches with case-insensitive filenames due to the
underlying case-insensitive filesystem.
The combined result is that libacl .cc files include their local copy of
acl/Url.h instead of the base directories src/URL.h which was needed.
The long term fix will be to shuffle URL.h and its related code into
a convenience library. For now we can avoid issues by prefixing the full
src/ path to the includes.
Amos Jeffries [Mon, 25 Feb 2013 03:42:35 +0000 (20:42 -0700)]
Bug 3753: Removes the domain from the cache_peer server pconn key
Under the squid-3.2 pconn model the IP:port specifying the destination
are part of the key and can be used to strictly filter selection when
locating pconn. This means the domain is no longer a necessary part
of the key.
Squid using cache_peer can see a large number of wasted idle connections
to their peers due to the key domain value if the peer hostname is not
substituted properly. There is also a similar affect when contacting
servers with virtual hosted domains.
Also bug 3753 was located with peer host and name= values being used
inconsistently as the domain marker. Resulting in failed pop()
operations and extra FD usage.
This has been tested for several months now with only socket usage
benefits seen in several production networks.
NOTE: previous experience some years back with pconn has demonstrated
several broken web servers which assume all requests on a persistent
connection are for the same virtual host. For now this change avoids
altering the behaviour on DIRECT traffic for this reason.
Amos Jeffries [Thu, 14 Feb 2013 07:34:42 +0000 (00:34 -0700)]
Bug 3686: cache_dir max-size default fails
If some cache_dir are configured with max-size and some not the default
maximum_object_size limit fails.
This refactors the max-size management code such that each SwapDir always
has a value maxObjectSize(). This value is calculated from the SwapDir
local setting or global limit as appropriate.
The global maximum_object_size directive is migrated to simply be a default
for cache_dir max-size= option.
The global store_maxobjsize variable is altered to be the overall global
limit on how big an object may be cache by this proxy. It now takes into
account the max-size for all cache_dir and cache_mem limitation.
NP: The slow accumulation of these and earlier changes means Squid no
longer immediately caches unknown-length objects. The unit-tests are
therefore changed to test using explicit 0-length objects to ensure the
test is on a cached object not bypassing the apparently ested logic.
They are also provided with a large global store_maxobjsize limit in order
to do a weak test of the SwapDir types max-size in the presence of other
larger cache_dir or maximum_object_size settings.
Alex Rousskov [Sat, 9 Feb 2013 12:48:22 +0000 (05:48 -0700)]
Bug 3752: objects that cannot be cached in memory are not cached on disk if cache_dir max-size is used.
This fix contains four related changes:
1) When fixing "trimMemory for unswappable objects" (trunk r11969), we
replaced swapoutPossible() with swappingOut()||mayStartSwapOut() but missed
the fact that swapoutPossible() had "possible now" semantics while
mayStartSwapOut() has "may start now or in the future" semantics. When all
cache_dirs had max-size set, mayStartSwapOut() returned false for objects of
unknown size and even for smaller-than-maximum but not-yet-received objects,
despite the fact that those objects could be swapped out later.
That false mayStartSwapOut() result allowed maybeTrimMemory() to trim those
objects memory and mark the objects for release, preventing their subsequent
disk caching.
2) To fix (1) above, mayStartSwapOut() had to return true for not-yet-received
objects of unknown size. However, returning true is correct only if no
subsequent check can return false. Thus, we had to move all lower/later checks
that could return false up, placing them before the maximum-of-all-max-sizes
check.
3) Once (2) was done, the end of mayStartSwapOut() had (a) a loop that could
return true while setting decision to MemObject::SwapOut::swPossible and (b)
an unconditional code that did ... the same thing. Thus, the loop could no
longer change the method outcome. The loop also had a lot of doubts and XXXs
attached to it. We removed it. If that loop is needed, it is needed and must
be resurrected elsewhere.
4) Since mayStartSwapOut() returns true if swapout is possible in the future
(but not necessarily now), we cannot rely on its return value to initiate
swapout code. We need to test whether swapout.decision is swPossible instead.
Alex Rousskov [Fri, 8 Feb 2013 11:25:14 +0000 (04:25 -0700)]
Make squid -z for cache_dir rock work like UFS instead of like COSS.
Also, Polish -z documentation and cache.log reporting.
When a startup script runs squid -z by mistake against a cache_dir that is
already initialized and full of cached entries, some admins prefer that
nothing happens. Rock store now skips reinitialization if both the cache_dir
directory and the db file in that directory exist. If one or both are missing,
the missing pieces are created.
UFS does something similar because it creates missing L1 and L2 directories
but does not erase any entries already present in the cache_dir path. COSS,
OTOH, re-initializes the existing db. Rock behavior will now be closer to UFS.
To clean a corrupted cache_dir, the admin must remove its top-level directory
before running squid -z.
Squid now logs "Creating missing swap directories" instead of "Creating Swap
Directories", and our documentation now reflects the "if missing" part of the
-z algorithm.
Also documented that recent Squid versions run -z in daemon mode (so that SMP
configuration macros continue to work).
Amos Jeffries [Sun, 3 Feb 2013 14:05:11 +0000 (07:05 -0700)]
Bug 3515: crash in FtpStateData::ftpTimeout
Since revision squid-3.2-11174 restructured teh TCO connection handling
in FTP the data connection has had separate 'opener' connection.
Meaning the conn->fd state is always NULL when the setup times out.
This cleans up the data connection opener state and allows the error
message generation to take place.
Alex Rousskov [Sun, 3 Feb 2013 14:03:29 +0000 (07:03 -0700)]
Fixed several ConnOpener problems
... by relying on AsyncJob protections and comm_close(), while maintaining a
tighter grip on various I/O and sleep states.
Problems addressed:
* Connection descriptor was not closed when attempting to reconnect after
failures. We now properly close on failures, sleep with descriptor closed,
and then reopen.
* Timeout handler was not cleaned up properly in some cases, causing memory
leaks (for the handler Pointer) and possibly timeouts that were fired (for
then-active handler) after the connection was passed to the initiator.
* Comm close handler was not cleaned up properly.
* statCounter.syscalls.sock.closes counter was not updated on FD closure.
* Waiting pending accepts were not kicked on FD closure.
* Connection timeout was enforced for each connection attempt instead of
applying to all attempts taken together.
and possibly other problems. The full extent of all side-effects of mishandled
race conditions and state conflicts is probably unknown.
There are cases where the generated certificates do not mimic enough
properties
and secure connection with the client fails. For example, Squid does not
mimic
Key Usage extensions. Clients using GnuTLS (or similar libraries that
validate
server certificate using those extensions) fail to secure the connection
with
Squid.
This patch add mimicking for the following extensions, which are
considered
as safe to mimic:
* X509v3 Key Usage
* X509v3 Extended Key Usage,
* X509v3 Basic Constraints CA.
We would be happy to add more "safe to mimic" extensions if users
request (and
vouch for) them.
Tianyin Xu [Mon, 28 Jan 2013 04:15:03 +0000 (21:15 -0700)]
Polish squid.conf parsing validation
* Updates ato*() functiosn to safer xato*() alternatives provided by
the Squid compat library. Along with error messages on invalid
configuration values detected by these.
* Add protection against integer overflow on most options
* Add parse deprecation messages on enable/disable for boolean and
and trilean options.
* Add 'wrong-value' error messages on most options.
Alex Rousskov [Mon, 28 Jan 2013 04:13:51 +0000 (21:13 -0700)]
Bug 3111: Mid-term fix for the forward.cc "err" assertion.
The assert is triggered when a close handler for the server connection
destroys FwdState before we have received anything from the origin
server.
Instead of asserting, we now respond with a 502 (Bad Gateway)
ERR_READ_ERROR.
That error seems the most appropriate single choice among available
ones, but
it may be misleading (in access.log) when the close handler was called
due to
client problems. Hopefully, another error will be logged in most of
those
cases.
Propagate pinned connection persistency and closures to the client.
Squid was trying hard to forward a request after pinned connection
failures
because some of those failures were benign pconn races. That meant
re-pinning
failed connections. After a few iterations to correctly handle
non-idempotent
requests, the code appeared to work, but the final design, with all the
added
complexity and related dangers was deemed inferior to the approach we
use now.
Squid now simply propagates connection closures (including pconn races)
to the
client. It is now the client responsibility not to send non-idempotent
requests
on idle persistent connections and to recover from pconn races.
Squid also propagates HTTP connection persistency indicators from client
to
server and back, to make client job feasible. Squid will send
Connection:close
and will close the client connection if the pinned server says so, even
if
Squid could still maintain a persistent connection with the client.
These changes are not mean to affect regular (not pinned) transactions.
In access.log, one can detect requests that were not responded to (due
to race
conditions on pinned connections) by searching for ERR_ZERO_SIZE_OBJECT
%err_code with TCP_MISS/000 status and zero response bytes.
Amos Jeffries [Mon, 28 Jan 2013 04:07:58 +0000 (21:07 -0700)]
squidpurge: display friendly errors on missing command line options
Currently the tool will crash with a segmentation fault if any one of
several command switches which are expected to have a mandatory argument
are in fact followed by nothing.
Detect these cases and display a message about what is missing.
Amos Jeffries [Tue, 22 Jan 2013 11:08:52 +0000 (04:08 -0700)]
Bug 3732: Fix ConnOpener IPv6 awareness
When updating IPv6 support for split-stack one USE_IPV6 wrapper was
omitted conversion to the EnabledIpv6 stack auto-detect mechanism.
This resulted in IPv6 addresses being mis-converted on split-stack
systems or with IPv6 disabled in the kernel and --enable-ipv6 built.
The visible symptom is "assert(FALSE)" in Ip::Address::GetAddrInfo().
Amos Jeffries [Tue, 22 Jan 2013 11:07:36 +0000 (04:07 -0700)]
Initialize mem_node fully
Experience in squid-2.7 shows that with MemPools use of malloc instead
of calloc mem_node requires full initialization in order to get accurate
memory accounting.
Alex Rousskov [Tue, 22 Jan 2013 11:06:18 +0000 (04:06 -0700)]
Fix "address.GetPort() != 0" assertion for helpers on FreeBSD (at least).
The order (or set of?) #include files used by src/ssl/helper.cc (and
probably by other helper source files) has changed recently, exposing a
defines.h dependency on sys/socket.h where that system header is
required to define AF_UNIX. With AF_UNIX incorrectly undefined,
IPC_STREAM was set to IPC_TCP_SOCKET instead of IPC_UNIX_STREAM, and
helpers that do not have a notion of a listening port, were trying to
create communication sockets using TCP streams, triggering a "must have
a port" assertion in comm_connect_addr() called from ipcCreate().
TODO: Moving IPC_* defines into an IPC-specific header file may be a
better solution then exposing all defines.h users to sys/socket.h.
Amos Jeffries [Tue, 22 Jan 2013 04:39:33 +0000 (21:39 -0700)]
WCCP: Fix memory leak in mask assignment, improve debuggsing.
* Release temporary weight array allocated on each HERE_I_AM packet sent
by Squid. For mask assignment these were not released properly and may
build up to a significant size of memory over time.
* Add debug traces to send() events to report failures sending packets
* Also, on HERE_I_AM event send() failure, reduce the timeout to 2sec
for the retry in a crude attempt to prevent router state flapping.
* Silence compiler warnings on use of connect() to disconnect a socket.
Inconsistent OS behaviour makes the result useless in this case.
Amos Jeffries [Wed, 9 Jan 2013 00:19:44 +0000 (17:19 -0700)]
Fix memory leaks in ICMP
InitAddrInfo() and GetAddrInfo() both allocate addrinfo structs which
must be freed by FreeAddrInfo(). Several places in ICMP were not doing
that free step.
Detected by Coverity Scan. Issues 740434, 740435, 740436, 740437.
Amos Jeffries [Wed, 9 Jan 2013 00:12:02 +0000 (17:12 -0700)]
Bug 3731: TOS setsockopt() requires int value
FreeBSD is confirmed errors on 8-bit variable size. Other BSD are
documented in a way that implies they do as well, although not at this
stage confirmed to be failing.
Linux seems to be the only confirmed system working with 8-bit size sent
to setsockopt(). So we revert this to 'int' (32-bit or 64-bit) as was
working in Squid 3.1.
Address issues for the defect classes:
- Array compared against 0
- Copy into fixed size buffer
- Dereference after null check
- Dereference before null check
projects / thirdparty / squid.git / log
