Amos Jeffries [Tue, 19 Nov 2013 22:45:02 +0000 (15:45 -0700)]
Make HTTP header parser obey relaxed_header_parser
Some unimportant warnings were not obeying the directives silent/loud
setting values.
This abstracts the condition to simplify code and make all the
non-critical warnings depend on the directive for their display level.
Alex Rousskov [Sun, 10 Nov 2013 23:01:35 +0000 (16:01 -0700)]
Replace blocking sleep(3) and close UDS socket on failures.
The two addressed XXX were not causing any known serious bugs on their own,
but the blocking sleep was ugly and possibly in the way of further
kid registration fixes/improvements.
Squid fails parsing error-details.txt template when one or more listed
OpenSSL errors are not supported on running platform.
This patch add a hardcoded list of OpenSSL errors wich can be optional.
Coordinator should not send SNMP client FD to strands when broadcasting SNMP
requests. Strands do not need the descriptor and were forgetting to close it,
causing one FD leak on every SNMP query in SMP mode.
Enhance Ipc::TypedMsgHdr to be able to tell whether the message has a FD.
Alex Rousskov [Fri, 1 Nov 2013 01:25:17 +0000 (19:25 -0600)]
Avoid "hot idle": A series of rapid select() calls with zero timeout.
Squid uses "infinite" precision when it comes to deciding whether the next
timed event is ready but uses millisecond (1e-3) precision when deciding how
long to wait before the next event will be ready. This inconsistency results
in the EventScheduler engine telling the main loop that it has 0 milliseconds
to poll pending I/O, but when asked again (after the I/O is quickly polled),
the EventScheduler engine often does not schedule the promised event and tells
the main loop to poll for another 0 milliseconds again. This cycling may
happen many times in a row (until enough time is wasted for the next event to
become ready using higher precision).
The fixed code adds a minimum 1ms delay for not-yet-ready events. It also
places both decisions into one method (EventScheduler::timeRemaining), and
tries to polish/document decision logic (which is more complex than it may
seem) because the code has to avoid both inconsistent decisions and hot idle
loops while maintaining the traditional "no event is fired before it is due"
guarantee.
TODO: Idle Squid still runs hotter than it should because the maximum waiting
time is artificially capped outside the event queue to EVENT_LOOP_TIMEOUT=1s.
This causes at most one extra loop iteration per second.
Amos Jeffries [Wed, 23 Oct 2013 05:11:22 +0000 (23:11 -0600)]
Add cache_miss_revalidate
Port of 2.7 ignore_ims_on_miss directive by Henrik Nordstrom.
This on/off switch enables Squid to convert conditional requests from
clients to non-conditional fetches that can fill the cache faster under
cold-start conditions.
Alex Rousskov [Wed, 23 Oct 2013 05:10:17 +0000 (23:10 -0600)]
Bug 3480: StoreEntry::kickProducer() segfaults in store_client::copy()
context
Short-term fix: Lock StoreEntry object so that it is not freed by
storeClientCopy2() callbacks. Also lock StoreEntry in storeUnregister()
context because an aborting entry may be deleted there unless it is
double-locked.
See bug 3480 comment #27 for detailed call stack analysis. Additional
cases
include rejected copied HIT due to Var mismatch and hits blocked by
reply_from_cache directive (under development; see bug 3937).
Long-term, we need to make store copying asynchronous and revise
StoreEntry
locking approach.
Amos Jeffries [Sun, 13 Oct 2013 13:41:55 +0000 (07:41 -0600)]
Fix CBDATA_CLASS2 macro definition
CBDATA_UNKNOWN was being used in place of a void no-op statement.
This was incorrect and useless. Now that the value definition is fixed
it is being picked up by the stricter compilers.
Replace the trinary conditional with an if-statement.
Amos Jeffries [Thu, 3 Oct 2013 11:12:31 +0000 (05:12 -0600)]
Add --disable-arch-native build option
The -march-native option is required by Clang 3.2 on some CPU
and produces better optimization in binaries by GCC.
But building inside a virtual machine environment has been found to
cause random Illegal Instruction errors due to mis-detection of CPU.
This option is added to permit selective disabling of teh preferred
default (which is to use the compiler flag when available).
Amos Jeffries [Thu, 3 Oct 2013 11:10:43 +0000 (05:10 -0600)]
Fix various header build issues in librfcnb
Highlighted when the #include sorting was applied to lib/rfcnb/ files
several of the headers were using order-dependent definitions and were
not wrapped with sfety macros.
Amos Jeffries [Thu, 3 Oct 2013 11:02:34 +0000 (05:02 -0600)]
Portability: cleanup includes for sys/time.h and sys/resource.h
These two include files are order-dependent on several operating systems
and as such are provided by Squid libcompat headers. There is no need for
code to include them specially and several HAVE_SYS_TIME_H macros were
missing which breaks the build on systems where the file is missing.
Markus Moeller [Thu, 3 Oct 2013 10:47:40 +0000 (04:47 -0600)]
negotiate_kerberos_auth: upgrade to present group= keys
Use the new kv-pair protocol extensions for emitting group details
directly at the time of authentication. group=X key-value pair is
used to emit the Kerberos SIDS values presented by MS Active
Directory environments.
Also, update the Hiemdal library support for recent library versions.
Also, polish the Kerberos and LDAP Group related code to avoid memory
leaks and improve C++ coding style.
Amos Jeffries [Thu, 3 Oct 2013 10:38:31 +0000 (04:38 -0600)]
Use IPv6 localhost nameserver on DNS configuration errors
When DNS configuration fails to locate a set of nameservers "localhost"
is used as the default but has previously only been setting 127.0.0.1
IPv4 address for localhost. This enables the failover to work as
designed
on IPv6-only networks.
Alex Rousskov [Sat, 28 Sep 2013 15:38:33 +0000 (09:38 -0600)]
Regression Bug 3891: squid.conf parser errors in 3.4.0.1
Alternative implementation for configuration_includes_quoted_values and
quoted string support in squid.conf that avoids the extensions attempted
in 3.4.0.1 leading to parse errors from overly strict parsing of all
directives parameters.
Documentation updated by Amos Jeffries, Treehouse Networks Ltd.
This feature has turned out to be more buggy than previously thought.
After a very log series of discussions the upgrade has been re-thought
and will not include this code in 3.4 series.
Windows: locate CMSG definitions in Winsock2.h when present
Windows defines the CMSG macros in Winsock2.h sometimes. This allows the
definitions there to be used when present and avoids compiler errors
about double definitions.
Bug 3849: Duplicate certificate sent when using https_port
The certificate file given with the "cert=" option it may contain a list of
certificates to be chained to the SSL client, for example intermediate
certificates.
The bug caused because in the certificates chain we are storing also the
certificate of the port. This is works well for SSL-bump because squid
generates a certificate which uses the port certificate as CA certificate.
But in the case of https_port without bumping the port certificate is sent
twice, one as SSL server certificate and one as chained certificate.
This patch try to chain port certificate only when the sslbump is used.
The port name from http_port/https_port was not being propigated to
adapted reqeusts after ICAP/eCAP. Which makes the myportname ACL and
logging of portname not work on adapted requests.
Windows: fix collision between POSIX wrappers and DiskIO class methods
The POSIX function shutdown() is #define'd on Windows. This causes
compiler issues with the DiskIO method of same name. Rename the method
to gracefulShutdown() instead.
Also, Bug 3189 was incomplete and missed a change to DiskThreads
Windows initialization method.
Alex Rousskov [Mon, 9 Sep 2013 01:31:30 +0000 (19:31 -0600)]
Close idle client connections associated with closed idle pinned connections.
Squid was not monitoring idle persistent connections pinned to servers. Squid
would discover that the pinned server connection is closed only after
receiving a new request on the idle client connection and trying to write that
request to the server. In such cases, Squid propagates the pinned connection
closure to the client (as it should).
Chrome and, to a lesser extent, Firefox handle such races by opening a new
connection and resending the failed [idempotent] request transparently to the
user. However, IE usually displays an error page to the user.
While some pconn races cannot be avoided, without monitoring idle pconns,
Squid virtually guaranteed such a race in environments where origin server
idle connection timeout is smaller than client/Squid timeouts and users
are revisiting pages in the window between those two timeouts.
Squid now monitors idle pinned connections similar to idle connections in the
pconn pool and closes the corresponding idle client connection to keep the two
sides in sync (to the extent possible).
It is theoretically possible that this change will break servers that send
whitespace on an idle persistent connection or perhaps send some SSL keepalive
traffic. No such cases are known to exist though.
Jan Sievers [Mon, 9 Sep 2013 01:25:12 +0000 (19:25 -0600)]
Regression Bug 3077: off-by-one error in Digest header decoding
This is a regression in the 3.2 and later version of the original bug
fix patch.
The effect is that all attributes are treated as quoted-string values
with unescaping performed. This ends with credentials wrongly being
rejected as invalid.
Amos Jeffries [Thu, 8 Aug 2013 06:08:02 +0000 (00:08 -0600)]
Fix configure parsing IP/domain directives
Squid would semi-silently accept invalid IP address or hostname in some
directives which required them (eg wccp_router) and use the magic
IP_ANYADDR value. This change makes configure halt with a FATAL error
instead.
Amos Jeffries [Thu, 8 Aug 2013 06:04:47 +0000 (00:04 -0600)]
Add dns_multicast_local to control mDNS operation
Enable admin control over whether mDNS operates or not. Set the default
to OFF due to .arpa reverse-DNS requests causing a rise in traffic from
this feature even on networks without mDNS responders setup.
Also, polish cachemgr idns report to show for queued queries which
resolver type and query the entry is involving. mDNS can cause a queue
to exist as some lookups timeout on the mDNS resolvers.
Amos Jeffries [Thu, 8 Aug 2013 06:03:04 +0000 (00:03 -0600)]
Fix debugs when refresh_pattern is missing the regex pattern
This is a very rare situation and may not be completely required. But it
did hinder debugging of refresh_pattern when the regex field token was
broken by parser updates.
Handle infinite certificate validation loops caused by OpenSSL bug #3090.
If OpenSSL is stuck in a validation loop, Squid breaks the loop and
triggers a
new custom SQUID_X509_V_ERR_INFINITE_VALIDATION SSL validation error.
That
error cannot be bypassed using sslproxy_cert_error because to break the
loop
Squid has to tell OpenSSL that the certificate is invalid, which
terminates
the SSL connection.
Validation loops exceeding SQUID_CERT_VALIDATION_ITERATION_MAX
iterations
are deemed infinite. That macro is defined to be 16384, but that default
can
be overwritten using CPPFLAGS.
Set configuration_includes_quoted_values to OFF by default in 3.4
Early testers have shown that bot hthe parameters() and $-macro detection
is not working at intended in some common situations. ie wheneverr the
config contains regex with '()' brackets or quoted $ anchor.
Leaving 3.HEAD default as ON for now to encourage a fix.
Invalid casting seems to confuse the ABI generator and results in
illegal instruction faults when the unit tests is run.
The class API is already const-correct so there is no need for the cast
to occur, and it should not be done on a non-pointer type anyway.
Also, fixes a missing "struct" type identifier found along the way.
This patch :
- adds support for quoted values in the entire squid.conf
- warn about or prohibit values that can no longer be interpreted as
either quoted strings or simple tokens
- support parameters("/path/to/file.name") syntax to load external
configuration files
- Checks if macros allowed in "double quoted" values.
- replaces the strtok() calls with calls to the new ConfigParser::NextToken()
- modify strtokFile to use new ConfigParser::NextToken()
- Add the new configuration_includes_quoted_values configuration option, to
control the squid parser behaviour. If set to on Squid will recognize each
"quoted string" after a configuration directive as a single parameter
This patch add new logformat codes to log TOS/DSCP values and netfilter marks
for client and server connections. If multiple outgoing connections were used,
the last used connection value logged.
The values printed in hexadecimal form.
The logformat codes are:
%>tos Client connection tos mark set by Squid
%<tos Server connection tos mark set by Squid
%>nfmark Client connection netfilter mark set by Squid
%<nfmark Server connection netfilter mark set by Squid
This patch also modify qos related code to set Comm::Connection::nfmark and
Comm::Connection::tos members in Ip::Qos::setSockNfmark and Ip::Qos::setSockTos
methods. The Comm::Connection members are now set only if the tos and nfmark
set successfuly.
This patch sends an If-None-Match request, when we need to re-validate
if a cached object which has a strong ETag is still valid.
This is also done in the cases an HTTP client request contains HTTP
headers prohibiting a from-cache response (i.e., a "reload" request).
The use of If-None-Match request in this context violates RFC 2616 and
requires using reload-into-ims option within refresh_pattern squid.conf
directive.
The exact definition of a "reload request" and the adjustment/removal of
"reload" headers is the same as currently used for reload-into-ims
option support. This patch is not modifying that code/logic, just adding
an If-None-Match header in addition to the IMS header that Squid already
adds.
Fix external ACL user:pass detail logging after adaptation
When a request is successfully adapted, the external ACL username and
password are now inherited with this patch. This means the
LFT_USER_NAME log token can display the username from an external ACL
if available, for adapted requests.
The HttpRequest will inherit the password for good measure as well -
while none too useful, it seems strange to inherit the username but
not the password.
We can do better than just producing errors about invalid port details
and treatign it as port-0.
We can instead undo the port separation and pass it through as part of
the host name to be verified with the default port number properly
assumed.
Protect against buffer overrun in DNS query generation
see SQUID-2013:2.
This bug has been present as long as the internal DNS component however
most code reaching this point is passing through URL validation first.
With Squid-3.2 Host header verification using DNS directly we may have
problems.
projects / thirdparty / squid.git / log
