Amos Jeffries [Fri, 6 May 2016 09:46:14 +0000 (21:46 +1200)]
Fix SIGSEGV in ESIContext response handling
HttpReply pointer was being unlocked without heving been locked.
Resulting in a double-free. Make it use RefCount instead of
manual locking to ensure locked/unlock is always symmetrical.
cachemgr.cgi: use dynamic MemBuf for internal content generation
Using a fixed size buffer limits how big content lines can be. Modern
HTTP is fast reaching the point where such limits are problematic.
Also fixes incorrect uses of snprintf() by removing them.
Amos Jeffries [Sat, 1 Aug 2015 01:00:12 +0000 (18:00 -0700)]
AUFS: Raise I/O queue congestion limits
... from 8 to 8196 before initial congestion message appears.
Modern networks can be quite busy and even amateur installations have a
much higher I/O throughput than Squid was originally designed for. This
often results in a series of "Queue congestion" warnings appearing on
startup before Squid learns what the local environment requires.
The new limit helps to cater for this and reduce teh frequency of
unnecessary warnings. They may still occur, so debug output is also
updated to show what the queue length has grown to with each warning.
Also updating the congestion counter from 32-bit to 64-bit unsigned
since the new limit already consumes half the available growth bits in
32-bit integer.
NP: this update was triggered by reports from admin with proxies needing
to expand AIO queues to over 4K entries on startup.
Alex Rousskov [Thu, 9 Jul 2015 03:21:33 +0000 (20:21 -0700)]
Do not blindly forward cache peer CONNECT responses.
Squid blindly forwards cache peer CONNECT responses to clients. This
may break things if the peer responds with something like HTTP 403
(Forbidden) and keeps the connection with Squid open:
- The client application issues a CONNECT request.
- Squid forwards this request to a cache peer.
- Cache peer correctly responds back with a "403 Forbidden".
- Squid does not parse cache peer response and
just forwards it as if it was a Squid response to the client.
- The TCP connections are not closed.
At this stage, Squid is unaware that the CONNECT request has failed. All
subsequent requests on the user agent TCP connection are treated as
tunnelled traffic. Squid is forwarding these requests to the peer on the
TCP connection previously used for the 403-ed CONNECT request, without
proper processing. The additional headers which should have been applied
by Squid to these requests are not applied, and the requests are being
forwarded to the cache peer even though the Squid configuration may
state that these requests must go directly to the origin server.
This fixes Squid to parse cache peer responses, and if an error response
found, respond with "502 Bad Gateway" to the client and close the
connections.
Amos Jeffries [Fri, 1 May 2015 07:11:04 +0000 (00:11 -0700)]
Fix 'access_log none' to prevent following logs being used
The documented behaviour of "access_log none" for preventing logging
using log lines following the directive has not been working in
Squid-3 for some time.
Since the 'none' type does not have a log module associated the entire
switch logic where its abort is checked for was being skipped.
The fix for Bug 3664 "ssl_crtd fails to build on OpenSolaris/OpenIndiana/Solaris 11"
introduced a regression on BSD and Linux where lockf() implementations appear not to
lock the entire file correctly or as reliably as flock().
Reverting the flock/lockf change for non-Solaris OS.
Amos Jeffries [Sun, 1 Feb 2015 09:43:41 +0000 (01:43 -0800)]
Stop emitting (Proxy-)Authentication-Info for Negotiate
This header is not defined for use by RFC 4559, and there seem to
be no clients actually using it.
The syntax Squid was using to emit the details was also clashing
with the syntax defined for use in Digest which is becoming the
standardized ABNF syntax for the header in general.
Amos Jeffries [Sun, 1 Feb 2015 09:42:12 +0000 (01:42 -0800)]
Remove dst ACL dependency on HTTP request message existence
The ACL checklist dst_addr member can be used in cases where the HTTP
message is not provided. Such as ssl_bump, ICAP outgoing IP, or peer
selection probes.
Set cap_net_admin capability when Squid sets TOS/Diffserv packet values.
In capabilities-capable environments (e.g., Linux with libcap), CAP_NET_ADMIN
capability is required to honor clientside_tos and tcp_outgoing_tos
directives. The code was setting that capability when Netfilter marks or
tproxy was enabled, but missed the clientside_tos and tcp_outgoing_tos cases.
Amos Jeffries [Sun, 18 Jan 2015 11:02:13 +0000 (03:02 -0800)]
Bug 3997: Excessive NTLM or Negotiate auth helper annotations
With the transaction annotations feature added in Squid-3.4 auth
helper response values get recorded as annotatiions. In the case
of NTLM and Negotiate authentication the helper response contains
a large credentials token which changes frequently.
Also, user credentials state is cached. In the case of NTLM and
Negotiate the active credentials are cached in the TCP connection
state data, but also for the cache mgr helper reports make use of
caching in a global username cache.
When these two features are combined, the global username cache
for mgr reporting accumulates all TCP connection specific
token= values presented by the client on all its connections, and
any changes to the token over its lifetime.
The result is that for users performing either many transactions,
or staying connected for long periods the memory consumption from
unnecesarily stored tokens is excessive. When clients do both the
machine memory can be consumed, and the CPU can reach 100%
consumption just walking the annotations lists during regular
operations.
To fix this we drop the security credentials tokens from cached
annotations list in NTLM and Negotiate. Digest is also included
though its HA1 token value is static it has similar privacy issues
related to storage.
Amos Jeffries [Fri, 9 Jan 2015 10:24:41 +0000 (02:24 -0800)]
Bug 3754: configure doesnt detect IPFilter 5.1.2 system headers
Solaris 10+ bundles IPFilter code natively, but the IPFilter
headers contain a duplicate definition of minor_t which does
not match the existing OS definition.
The result is that no applications (such as Squid) will build
on Solaris with the natively provided headers.
Also, the upstream IPFilter code separate from Solaris contains
the same minor_t definition so building against a separate
install of IPFilter does not fix the issue.
We must instead play fancy games #define'ing minor_t to a
different real-name for the OS headers and its own name for
the IPFilter headers.
Thanks to Yuri Voinov for sponsoring the Solaris 10
machine and environment resources for this fix.
Bug 4164: SEGFAULT when %W formating code used in errorpages
Squid will crash inside ErrorState::Dump if not authentication configured for
squid. In this case ErrorState::auth_user_request is NULL and trying to access
a method of this object will cause segfault to squid.
Alex Rousskov [Sun, 14 Dec 2014 11:02:41 +0000 (03:02 -0800)]
Bug 4057: Avoid on-exit crashes when adaptation is enabled.
After trunk r13269 (Vector refactor) destroyed vector objects still have
positive item counts. This exposes use-after-delete bugs. In this particular
case, global adaptation rule/group/service arrays are destructed by global
destruction sequence first and then again by Adaptation::*::TheConfig objects
destructors.
This change avoiding static destruction order dependencies by storing those
global adaptation arrays on heap.
Amos Jeffries [Mon, 8 Dec 2014 12:05:17 +0000 (04:05 -0800)]
HTTP/2: Support 421 (Misdirected Request) status code
Add support for status 421 responses. Squid is forbidden from generating
messages with this status in its role as proxy, however we expect to see
it being produced in responses by HTTP/2 servers in traffic from HTTP
2.0<->1.1 gateways.
We also MAY emit it on future reverse-proxy responses in the event of a
ERR_CANNOT_FORWARD message.
Amos Jeffries [Wed, 3 Dec 2014 12:06:59 +0000 (04:06 -0800)]
Alternate-Protocol is a hop-by-hop header
Google are pushing Alternate-Protocol header to arbitrarily move clients
from HTTP to their experimental and proprietary protocols.
When relayed through a proxy this can cause traffic bifurcation with
side effects ranging from client visible failures, to traffic increase
on the network as proxy caching is bypassed, to security vulnerabilities
or privacy information leaks over the experimental protocol.
By treating the Alternate-Protocol header as a hop-by-hop header only
relevant on client->origin connections we can both prevent the above
issues occuring on networks utilizing a proxy, and also participate in
the protocol exprimentation process ourselves.
The certificate db size file may become empty (for reasons beyond Squid
control such as server reboots, and possibly some unknown Squid bugs).
When it becomes empty, all ssl_crtd helpers (and then Squid) quit. This
change is required to make ssl_crtd more robust by recovering lost db
size information.
This patch:
- Adds the "size" rebuild operation in CertificateDB and ssl_crtd
daemon. Rebuild ssl_db/size file if it is empty:
* Inside Ssl::CertificateDb::check method
* When a CertificateDB operation try to read size from ssl_db/size file
- If no fs_block_size parameter given for CertificateDB then consider a
default value of 2048. Currently set to 0, which is may cause segfault to
ssl_crtd daemon.
Bug 4102: sslbump cert contains only a dot character in key usage extension
The patch for bug 3966 was slightly incorrect. As a result the Key Usage
field for SSL-bump mimic'ed certificates could end up containing only a
dot (.) character.
Amos Jeffries [Thu, 16 Oct 2014 19:22:37 +0000 (12:22 -0700)]
Bug 3803: ident leaks memory on failure
Begin the process of conversion for IdentStateData to an AsyncJob.
* convert the object from CBDATA struct to a class with
CBDATA_CLASS2() API.
* Bug 3803 is caused by a lack of proper cleanup and consistent exit
actions terminating the job. Take the core logic changes from the
tested bug patch and;
1) define a swanSong() method to cleanup the memory allocated
2) define a deleteThis() method to emulate AsyncJob::deleteThis()
* Locate all code paths leveraging conn->close() to trigger cleanup
via the connection close handler and convert to explicit deleteThis()
with excuse. Including a few which were not but need to in order to
terminate the job correctly as fixed in bug 3803 patch.
The actions performed are nearly identical to the original code. The
differences are that many code paths now omit an AsyncCall step going
via the Comm close handler, and that all paths terminating the IDENT
lookup now go through swanSong() cleanup.
Further cleanup converting to a full AsyncJob is not included, since
there is an explicit hash of running IdentStateData object pointers
being used in the old code.
Amos Jeffries [Thu, 16 Oct 2014 18:39:55 +0000 (11:39 -0700)]
CBDATA: log memory leak situations when --enable-debug-cbdata
CBDATA objects are supposed to be explicitly locked and unlocked by all
users. The nominal 'owner' of the data is also supposed to mark it as
invalid when unlocking its reference.
If a CBDATA object reaches 0 locks and is still valid, it therefore
follows that either the locking or invalidate has not been properly
implemented.
Now that we are migrating to CbcPointer usage instead of explicit
lock/unlock macro calls we have started encountering these situations.
Any object reporting a 'leak' must be investigated;
a) perhapse RefCount is better?
b) using CbcPointer consistently and invalidating correctly.
Amos Jeffries [Thu, 9 Oct 2014 11:53:28 +0000 (04:53 -0700)]
Bug 4088: memory leak in external_acl_type helper with cache=0 or ttl=0
ExternalACLEntry / external_acl_entry objects have been abusing the
CBDATA API for reference counting and since 3.4 this has resulted in
hidden memory leaks as object accounting shows all locks released but
the memory is not freed by any 'owner'.
* convert to using RefCount<> API.
* move ExternalACLEntry pre-define to acl/forward.h
* add ExternalACLEntryPointer in acl/forward.h
* convert LookupDone() method to using explicit typed pointer
* convert from CBDATA_CLASS to MEMPROXY_CLASS memory management.
* convert almost all raw ExternalACLEntry* to Pointer
- remaining usage is in the cache hash pointers. Use an explicit 'cachd'
lock/unlock until this hash is updated to std:: structure types.
Alex Rousskov [Tue, 26 Aug 2014 16:22:01 +0000 (09:22 -0700)]
Use v3 for fake certificate if we add _any_ certificate extension.
We used to force v3 version only when adding the subjectAltName extension.
That broke sites that did not have subjectAltName but used other mimicked x509
extensions, when accessed through Firefox 31 (at least):
https://bugzilla.mozilla.org/show_bug.cgi?id=1045973
Fix %USER_CA_CERT_* and %CA_CERT_ external_acl formating codes
* The attribute part of the %USER_CA_CERT_xx and %CA_CERT_xx formating codes
is not parsed correctly, make these formating codes useless.
* The %USER_CA_CERT_xx documented wrongly
Joe Crayne [Sun, 13 Jul 2014 03:15:01 +0000 (21:15 -0600)]
Bug 3966: Add KeyEncipherment when ssl-bump substitues RSA for EC.
Libnss3, which is used by Firefox to verify the certificate chain, has
different requirements for RSA keys than it does for EC keys. In particular,
RSA keys with the keyUsage extension, must set the KeyEncipherment flag.
This fix was brought to you by the Samizdat project.
http://samizdat.childrenofmay.org
Non https connectiona on SSL-bump enabled port may stuck
This is can be seen on skype when try to connect to server using an
SSL-bump enabled squid port. Squid try to bump the connection, waiting
for ever
the ssl protocol header, and skype client waits for ever an answer from
the
server.
This patch sets the timeout to Config.Timeout.request (request_timeout)
Alex Rousskov [Sat, 21 Jun 2014 04:22:39 +0000 (22:22 -0600)]
Do not leak implicit ACLs during reconfigure.
Many ACLs implicitly created by Squid when grouping multiple ACLs were
not
destroyed because those implicit ACLs where not covered by the global
ACL
registry used for ACL destruction.
See also: r13210 which did not go far enough because it incorrectly
assumed
that all InnerNode() children are aclRegister()ed and, hence, will be
centrally freed.
Also, do not use cbdataFree() on non-POD Acl::Tree objects that have
destructors.
Amos Jeffries [Sat, 21 Jun 2014 04:19:58 +0000 (22:19 -0600)]
Portability: use 64-bit for X-Cache-Age header
While the value is expected to be well within 32-bit range some OS
(OpenBSD 5.5 at least) use 64-bit time_t. Use the larger type size for
calculations which also removes 32-bit wrap errors, and cast for older
systems.
Amos Jeffries [Sat, 21 Jun 2014 04:19:19 +0000 (22:19 -0600)]
Windows: fix various libip build issues
* Missing include ws2tcpip.h for IPv6 definitions
* Alternative IN6_ARE_ADDR_EQUAL definition required
* 'byte' is a reserved / system defined type on Windows,
resolve variable shadowing by renaming to ipbyte.
projects / thirdparty / squid.git / log
