When Squid sends errors to the certificate validation daemon, the daemon cannot
tell which certificate caused which error. This is especially bad because the
validator has to return that same information in the response (the response
format requires the validator to match the error to the certificate).
This patch adjust the validation request format to provide that information
using a set of the following key=value pairs:
error_name_N=the name of the certificate error number N
error_cert_N=the ID of the certificate which caused error_name_N
where N is non-negative integer. N values start from zero and increase
sequentially.
Alex Rousskov [Wed, 5 Jun 2013 15:38:09 +0000 (09:38 -0600)]
Redo r12887 (bug 2066 fix) which introduced several related bugs:
- fatal() if chroot is given
- no chdir to / after chroot if coredump_dir is given
- wrong detection of "none" coredump_dir
- chdir to uninitialized pathbuf if no chroot is given
This emergency fix helps Squid start better but may need more testing/work.
Amos Jeffries [Tue, 4 Jun 2013 05:12:39 +0000 (23:12 -0600)]
Drop Ip::Address(Ip::Address *) constructor entirely
Has been deprecated for a while and appears to no longer be required by
any of the current code. The reference& copy-constructor can easily be
used in its place.
Amos Jeffries [Tue, 4 Jun 2013 04:21:48 +0000 (22:21 -0600)]
Fix NULL-dereference added in rev.12779
With the change of helper responses from Notes to NotePairs the errNote
in NTLM ERR/NA responses was altered to a potentially NULL char*, and
allowed to be printed in debugs() level 4.
This updated NTLM and Negotiate halpers to print the helper response as
"Result: ..." in identical fashion.
Amos Jeffries [Tue, 4 Jun 2013 04:09:31 +0000 (22:09 -0600)]
Bug 2066: squid does not do chdir() after chroot()
The earlier workaround applied only fixed 1 of the 3 places performing
chroot().
This makes chroot and chdir integral parts of setting up Squids running
directory and alters teh chroot() calls to be mainSetCwnd() calls. Which
fixes several potential problems with core dumps from squid -z or -k
executions ending up in unexpected locations, regardless of whether
chroot() and coredump_dir are configured.
Amos Jeffries [Mon, 3 Jun 2013 14:38:26 +0000 (08:38 -0600)]
Fix basic_pam_auth helper detection
The m4_include() for this helpers test script was omitted from rev.12782.
Add that in, and shuffle the PAM-specific configure tests into the helper
required.m4 script.
Amos Jeffries [Mon, 3 Jun 2013 14:05:16 +0000 (08:05 -0600)]
Polish: update Ip::Address to follow Squid coding guidelines
* lower-case initial word for camelCase method names
* _ suffix for private variables.
* upper-case for static methods
* InitAddrInfo() and FreeAddrInfo() are static, do not use as methods
Not all methods are camelCased due to meaning irregularities and there
are other guidelines not being followed which also need to be fixed.
Amos Jeffries [Sun, 2 Jun 2013 14:32:18 +0000 (02:32 +1200)]
Fix incorrect external_acl_type codes
Documentation describes %USER_CA_CERT_* codes for outputing the CA cert
attributes. However the directive parser and internals were all
referencing it as %CA_CERT_*.
This updates the internals to match documentation, and adds an upgrade
notice for any installations using the old token name.
Amos Jeffries [Sun, 2 Jun 2013 11:47:05 +0000 (05:47 -0600)]
Support multicast DNS
Resolve .local domain names using mDNS one-shot queries ahead of regular
recursive DNS qeuries.
* adds the mDNS multicast group IPs as always-present entries in the
nameservers list.
* filters each request. ".local" lookups are permitted to both the mDNS
resolvers and the recursive resolvers, other requests are only
permitted to the regular recursive resolvers.
Amos Jeffries [Sat, 1 Jun 2013 10:01:13 +0000 (04:01 -0600)]
Fix typo in rev.12859
cbdataReference() is a macro with a design which means it requires usage
in the form of an assignment. ie a = cbdataReference(B); , or as a
parameter in equivalent to a variable.
Not using it in either of those manners results in syntax errors about a
',' from any strict compiler, and many older versions of GCC.
Alex Rousskov [Tue, 28 May 2013 16:29:39 +0000 (10:29 -0600)]
Improve ACL handling. Support all-of and any-of ACL types.
This commit encapsulated many significant ACL changes, including:
* Expressiveness: Two new boolean ACLs (all-of and any-of) that allow
admins to group ACLs as needed, to express complex conditions more
naturally, with fewer squid.conf lines. Conditions such as "(a or b) and
(c or d)" are easily expressed now. Explicit groups of ACLs of different
types can now be configured, named, and used in any ACL expression.
* Correctness and performance: When a slow ACL (that has suspended
checks to wait for an async lookup) is ready to resume checking, Squid
resumes checking from that ACL, instead of rechecking all ACLs for the
same action (or the same squid.conf directive) again.
* Internals: Store ACL-related configurations as an expression tree,
streamlining the code and clearing the way for future math-style/natural
ACL conditions support. The usual boolean operators (and, or, and not)
form intermediate nodes while good old configurable ACLs become tree
leaves. The new all-of and any-of ACLs use the boolean operators (and
also become intermediate nodes, of course).
Forcing 64-bit file operations on 64-bit systems exposes
an issue in stdlibc++, which is covered by gcc but not by
icc in non-gcc mode. This patch prevents forcing 64-bit
file operations on systems where long int is 64-bit or more.
It also properly protects a couple of gcc-specific #pragma.
Amos Jeffries [Sat, 25 May 2013 09:02:59 +0000 (03:02 -0600)]
Windows: include 'functional' before defining stdlib API wrapper functions
We currently use of #define to replace functions in the stdlib API with
versions which use the Windows API and error reporting.
Some of these result in precompiler replacement of systen internal
symbols. In this case member functions called bind() in templates of
<functional> API get replaced by Squid::bind() symbol.
This patch investigates the new note ACL type, to match transaction annotation.
Syntax:
acl aclname note name [value ...]
Without values, matches any annotation with a given name. With value(s), matches
any annotation with a given name that also has one of the given values.
Annotation sources include note and adaptation_meta directives as well as helper
and eCAP responses.
Alex Rousskov [Fri, 24 May 2013 15:14:50 +0000 (09:14 -0600)]
Ask for SSL key password when started with -N but without sslpassword_program.
Do not give SSL a password-asking callback if sslpassword_program is not
configured. Without a callback, OpenSSL itself asks for the password (which
works if Squid runs in foreground because of -N).
The fix applies to Ssl::readCertChainAndPrivateKeyFromFiles() context only.
This is not the only place where we read private keys. Some other places are
working correctly, but others may need more work. Also,
Ssl::readCertChainAndPrivateKeyFromFiles() may not really work if
sslpassword_program _is_ configured because "user data" pointer will be nil.
Cleanup: Merge AccessLogEntry 'helperNotes' and 'configNotes' members to 'notes' member
There is not any need to store notes added using Note cfg option and notes added
from helper to separated member. This patch merge them to the same
AccessLogEntry::note member.
Amos Jeffries [Wed, 22 May 2013 06:33:05 +0000 (00:33 -0600)]
Add pt-bz (Belize Portuguee) dialect to translations
In absence of information on whether Belize Portuguese dialect can be
presented with Brazillian or European Portuguese texts or whether they
require a third translation use a symlink entry for pt-bz to pt-br based
on the geographical distance for dialect locale alternatives.
Alex Rousskov [Wed, 22 May 2013 01:04:34 +0000 (19:04 -0600)]
Honor zero-padded logformat codes again.
Trunk r12628 "Turn flags to bool" set Token::zero to false instead of true,
resulting in extra spaces occasionally logged in front of such popular and
default fields as %03tu, screwing up log parsers.
Amos Jeffries [Tue, 21 May 2013 05:39:18 +0000 (23:39 -0600)]
Detect and use -march=natuve when possible
Clang++ 3.2 fails to detect some CPUs correctly and requires the
additional checks enabled by this option to build working executables.
This option supported by GCC 4.3 and later enables additional CPU
detection and enables CPU-specific optimizations. In the interests of
better performance this patch enables it whenever is it available and
possible to use (cross-compilers cannot use it).
Ming Fu [Tue, 21 May 2013 05:17:23 +0000 (23:17 -0600)]
Bug 1991: kqueue causes SSL to hang
Compare the code in normal select and epoll v.s. kqueue. The select use a 0
wait time to get out of select wait in order to handle a list of read_pendings.
However, epoll add the read_pending to read and write event monitor. At a first
look, this seems strange as why read pending has anything to do with write. It
became obvious when the write ready event is triggered. During a write ready
event, if read_pending is on, the read callback is called before the write
callback. As the write buffer is unlikely to be full for an extended period, a
write callback is guaranteed in the immediate future for the read_pending
socket by waiting on write.
The patch follows that same logic as epoll and applies it on kqueue.
The code initializing default values and determining helper detection
was significantly different from other helpers detection logics. Combined
with rev.12786 logic changes resulted in the detection always failing.
Update the --enable-log-deemon-heleprs option to work identically to the
--enable-external-acl-helpers option.
Silamael [Fri, 17 May 2013 08:36:45 +0000 (02:36 -0600)]
Add missing piece omitted from rev.9677.
rev.9677 created forward_max_tries direcive but omitted one of the
checks. This adds that check and allows forward_max_tries to be set
to values greater than 10.
Bug 3744: squid terminated: FATAL: Bungled (null) line 3: sslproxy_cert_sign signTrusted all
This bug is a Makefile dependencies problem.
- The cf_gen includes the cf_gen_defines.cci so this file should included in
cf_gen dependencies.
- Currently the cf_gen_defines.cci exist in cf_gen.$(OBJEXT) dependencies but
does not have any effect because the obj file never build and used.
- Also the cf_gen_defines.cci file depends on autoconf.h so this file should
added to to cf_gen_defines.cc dependencies.
All of the sources has the autoconf.h file in their dependencies.
But the cf_gen_defines.cci is auto-generated and does not exist when the
dependencies computed.
Change the libraries order in LIBS variable inside SQUID_CHECK_OPENSSL_GETCERTIFICATE_WORKS check.
Looks that play a role in some cases (when openssl provided only as static
library in my tests).
Bug 3759: OpenSSL compilation error on stock Fedora17, RHEL, CentOS 6 systems
OpenSSL-1.0.x has changes in TXT_DB interface over the earlier openSSL releases.
Also looks that the IMPLEMENT_LHASH_* macros are not correctly implemented and
causes compile failures.
Some of the linux distributions to overcome the above problems trying to patch
openSSL SDK. For squid this is means that the current checks based on openSSL
version can not work.
This patch try to detect at configure time:
- if the TXT_DB uses the new implementation investigated in openSSL-1.0.x
releases
- If the IMPLEMENT_LHASH_* openSSL macros are correctly implemented.
Then uses the autoconf defines to implement the correct workarounds for used
openSSL SDK.
This patch try to avoid using the SSL_get_certificate function. While configures
squid run tests:
- to examine if the workaround code can be used
- to detect buggy SSL_get_certificate
Inside Ssl::verifySslCertificate try to use workarround code and if this is not
possible uses the SSL_get_certificate if it is not buggy, else hit an assertion
Amos Jeffries [Tue, 14 May 2013 15:31:15 +0000 (09:31 -0600)]
Log an ERROR instead of halting on unknown cache_dir types
Squid-3 can run fine without any configured cache_dir. This assists with
upgrade from older Squid-2 where COSS or NULL cache types may be present.
It also assists with backward compatibility for any future cache types
which may be added in future.
Alex Rousskov [Mon, 13 May 2013 22:48:23 +0000 (16:48 -0600)]
Major ACL handling update, including the following changes:
* Expressiveness: Two new Boolean ACLs (all-of and any-of) that allow admins
to group ACLs as needed, to express complex conditions more naturally, with
fewer squid.conf lines. Conditions such as "(a or b) and (c or d)" are
easily expressed now. Explicit groups of ACLs of different types can now be
configured, named, and used in any ACL expression.
* Correctness and performance: When a slow ACL (that has suspended checks to
wait for an async lookup) is ready to resume checking, resume checking from
that ACL, instead of rechecking all ACLs for the same action (or the same
squid.conf directive) again.
* Internals: Store ACL-related configurations as an expression tree, streamlining
the code and clearing the way for future math-style/natural ACL conditions
support.
Amos Jeffries [Mon, 13 May 2013 16:21:23 +0000 (10:21 -0600)]
Remove origin_tries limiter on forwarding
This limit seems to have been set to prevent large amount of looping when
DIRECT attempts fail under the old model of constant DNS lookups and
retries.
However it is hard-coded and has no configuration knob visible. Under
the curent model of all destinations being enumerated once and tried
sequentially this protection would seem to be no longer necessary and
somewhat harmful as it will be preventing retries reaching destinations
with more than 2 unreachable IPs (think 3 IPv6 and an IPv4 on IPv4-only
network).
Alex Rousskov [Mon, 13 May 2013 14:07:55 +0000 (08:07 -0600)]
Fixed leaking configurable SSL error details.
Trunk r11496 "Configurable SSL error details messages" correctly disabled
collection of HTTP statistics for non-HTTP header fields, such as configurable
SSL error details. However, it also incorrectly disabled deletion of those
non-HTTP header fields.
Configurable SSL error details are only created during [re]configuration time,
so the leak went unnoticed since 2011-06-17, but the same bug caused a major
runtime annotation leak later (r12413) until the new annotation code was
redesigned to avoid using HttpHeader (r12779).
Steve Hill [Mon, 13 May 2013 03:57:03 +0000 (21:57 -0600)]
Add spoof_client_ip access control
... to control whether TPROXY requests have their source IP address
spoofed by Squid. The ACL defaults to allow (i.e. the current
behaviour), but using an ACL that results in a deny result will
disable spoofing for that request.
Example config to disable spoofing for all requests:
spoof_client_ip deny all
Also, polish the TPROXY related flags:
1. The flags.spoofClientIp flag was a general "this is a TPROXY
request" flag, which was a bit confusing given the name of the
flag. So the flags.spoofClientIp flag now only indicates
whether we want to spoof the source IP or not.
2. The flags.interceptTproxy is added to flag whether the request
was received on a "tproxy" port or not.
Nathan Hoad [Sat, 11 May 2013 20:59:44 +0000 (14:59 -0600)]
Bug 3389: Auto-reconnect for tcp access_log.
Major changes:
1. Squid reconnects to TCP logger as needed. Squid keeps trying to connect
forever, using a hard-coded 0.5 second delay between attempts.
2. Squid buffers log records while there is no connectivity. The buffering
limit is configurable.
3. On buffer overflows, Squid worker either dies or starts dropping log
records. The choice is configurable.
4. The tcp logging module honors buffered_logs setting. Old code was flushing
each record.
5. Squid reports changes in logging state to cache.log. Except for every 100th
consecutive connection failures, routine connection retries are not reported
at level 1, to reduce noise level.
6. A new access_log configuration format/style has been added. It allows us to
easily add named options such as buffer-size or on-error. The same format can
be used to add module-specific options in the future, but doing so would
require changes to the high-level logging code. All old configuration
formats/styles are still supported.
7. squid.conf buffered_log option documentation now reflects reality. It used
to talk about cache.log but I do not think Squid uses that option for
cache.log maintenance.
Known minor side-effects of these changes:
i) All access_log logs can now be configured to bypass errors because the old
"fatal" flag is now configurable via log-specific on-error option in
squid.conf. The default is still "die". I have not checked whether modules
other than TCP logger honor that flag.
ii) All access_log logs now use 8*MAX_URL (64KB) instead of a 4*MAX_URL (32KB)
or smaller buffer size by default. The ICAP logger was using 2*MAX_URL buffer
size. The TCP logger was using 64KB buffer size before so no change for TCP. I
decided that it is better to raise the default buffering level for some logs
rather than decrease it for other logs, but it is not clear what the best
default is. The buffer size is now configurable via buffer-size so admins can
control it on individual log basis.
iii) Some access_log configuration styles overlap. To resolve ambiguities,
Squid may need to assume that the first logging ACL name (if any) does not
contain '=' and is not equal to an existing logformat name. It is possible to
use 'all' as the first ACL name if these heuristics cause problems.
TODO: We have attempted to solve more TCP logging problems, but it turns out
that correct solutions would require fixing higher-level logging code, not
specific to TCP logger or Bug 3389 scope. Those unsolved problems include:
A. During reconfiguration, all logs are closed and reopened, even if there
have been no changes to their configuration that necessitate such a drastic
action (or no changes at all!). For TCP logger, this means that the old
connection is used to flush remaining buffered records (if any), and the new
connection is used to log new records, possibly at the same time. Nathan wrote
clever code that keeps logging going using the same job/connection. However,
we had to yank that code out because it clashed with higher-level logging
state in subtle ways.
B. During shutdown, all connections are put in the closing state before logs
are told to flush remaining records. For TCP logger, this means that the
remaining buffered records (if any) are lost. The correct fix may require
rearranging shutdown sequence AND letting EventLoop run during shutdown (among
other things).
C. When logger connectivity is lost, Squid does not notice the problem until
the second TCP socket write (or later). This results in lost records. This is
due to TCP-level buffering. I suspect the only cure for this is adding
logger-to-Squid "I got your records" feedback, which requires changes in the
logging protocol (currently there is no logger-to-Squid communication at all).
Alex Rousskov [Sat, 11 May 2013 15:23:59 +0000 (09:23 -0600)]
Avoid !closing assertions when helpers call comm_read [during reconfigure].
While helper reading code does check for COMM_ERR_CLOSING, it is not sufficient
because helperReturnBuffer() called by the reading code may notice the helper
shutdown flag (set earlier by reconfigure) and start closing the connection
underneath the reading code feet.
projects / thirdparty / squid.git / log
