Alex Rousskov [Fri, 16 Jul 2010 22:37:42 +0000 (16:37 -0600)]
Added debugging scripts that work with detailed cache.log
scripts/find-alive.pl: pinpoint objects that are still alive, to find leaks
scripts/trace-job.pl: find cache.log lines that correspond to a given job
scripts/trace-master.pl: trace jobs related to a single master transaction
The scripts require maintenance as the logging format changes, but
they often simplify debugging by extracting relevant information from
tons of poorly structured cache.log data.
Alex Rousskov [Tue, 13 Jul 2010 16:43:00 +0000 (10:43 -0600)]
Prevent memory leaks when cloning Range requests.
HttpRequest::range field was set to a new HttpHdrRange object twice:
once in HttpRequest::clone() and once in HttpRequest::hdrCacheInit()
called from clone().
Polished HttpReply::clone() to make sure HttpReply::hdrCacheInit()
does not use uninitialized HttpReply::sline field and to prevent
benign double-initialization of HttpReply::keep_alive.
Alex Rousskov [Wed, 7 Jul 2010 03:17:47 +0000 (21:17 -0600)]
Moved KidIdentifier to globals to make pinger happy.
Pinger and possibly other optional externals require KidIdentifier via
debugs(). Instead of making KidIdentifier global, we could add a "plugin" API
to add program-dependent stuff to debugs() and friends, but we should not add
that kind of complexity unless really necessary. We could also link pinger
with libipc.la but that will probably cause more problems with IPC
dependencies.
Alex Rousskov [Wed, 7 Jul 2010 00:45:34 +0000 (18:45 -0600)]
SMP support, part 1: Essential non-caching functionality.
Added workers squid.conf option to specify how many main Squid
processes to fork and maintain. Zero means old no-daemon mode.
One means the old non-SMP mode.
Added support for process_name and process_number macros and
if-statement conditionals in squid.conf. Search for .pre changes for
documented details. These features allow the admin to configure each
worker process differently if needed.
Support multiple workers listening on the same HTTP[S] port (port
sharing). This allows multiple workers to split the load without any
special rules.
Support or prohibit port sharing for WCCP, DNS, ICP, HTCP, SNMP, and
Ident protocols, depending on protocol-specific restrictions. Sharing is
implemented by registering listening socket descriptors with the
Coordinator process and obtaining them from the Coordinator as needed.
Here are protocol-specific notes:
WCCP: Restricted to the Coordinator process due to how WCCP works.
Workers do not need access to the WCCP code.
DNS: Done by each worker with no sharing. Fixed source ports not
supported unless each worker is given its own outgoing address
because we do not want to match outgoing queries and incoming
responses across processes.
SNMP: Workers share incoming and outgoing sockets.
ICP and HTCP _clients_: Cannot be supported in SMP environment
unless each process has its own address (i.e., unique IP address
and/or unique [ICP] port) because we do not want to match outgoing
queries and incoming responses across processes.
ICP and HTCP _servers_: share listening sockets.
Ident clients do not need to share sockets because they use
unique ports.
Support management signals (squid -k ...) in SMP mode, acting as a
single Squid instance.
Refork dying workers, similar to how we reforked dying process in
non-SMP daemon mode.
Alex Rousskov [Wed, 7 Jul 2010 00:22:37 +0000 (18:22 -0600)]
Do not stop testheaders.sh on the first error, to be compatible with
"make -k check".
If testheaders.sh stops on the first error, it becomes very difficult to
test your changes with "make check" when somebody else broke some other
code or when the checks reach src/cf_parser.h and fail.
TODO: stop on the first error unless running under "make -k check".
Bundle a slightly hacked version of libltdl/ltdl.h which does not use <>
for inclusion of its dependency files. This makes it portable to OS with
older libtool versions.
Amos Jeffries [Wed, 30 Jun 2010 12:32:50 +0000 (00:32 +1200)]
NTLM helpers cleanup pt 3: migrate libsmbval into libntlmauth
Library changes:
* ntlmauth.* files moved to libntlmauth/
* helpers/ntlm_auth/smb_lm/smbval moved to libntlmauth/
* No behaviour changes. Since I can't test the deeper logics.
Just enough to make the code built with portable types available in Squid
* API shuffled slightly to use less .h and to remove all external uses of
private *-priv.h definitions.
Library now provides three NTLM backend API:
libntlmauth/ntlmauth.h - NTLM packet handling
libntlmauth/smb.h - SMB LM credential validation
libntlmauth/rfcnb.h - RFCNB (NetBIOS) domain server communications
Helper Changes:
* NTLM helpers tweaked slightly to build with the adjusted libntlmauth API
and ntlm_smb_lm_auth helper to build as C++
* automake logics updated to obey --disable-auth and --disable-auth-ntlm
NOTE: There will be extra code safety and testing benefits gained by
converting libntlmauth to C++ as well. But that requries someone who
can test the code behaviour during the upgrade. For now this wil do.
Amos Jeffries [Sun, 27 Jun 2010 08:41:35 +0000 (02:41 -0600)]
Correct deny_info tokens again.
url_presentable was not just enabling URL-encoding.
It restricts the set of tokens to those which produce data usable in
deny_info URLs.
Renamed to building_deny_info_url for clarity of use and added back in the
restrictions on adding protocol dumps and non-existent page names into the
URL.
Robert Collins [Tue, 15 Jun 2010 08:26:43 +0000 (20:26 +1200)]
Bug 2950: HTTP responses with no Date, Last-modified or expired can
now be cached (given an appropriate refresh_pattern). Previously we
were not caching them in case of an infinite loop in cache farms:
however Squid adds Date: headers now which remove that concern.
Amos Jeffries [Tue, 15 Jun 2010 07:18:24 +0000 (01:18 -0600)]
Maintenance: snapshot detect package name and version better.
The old patterns are no longer unique. Were setting the snapshot package
and version variables wrong.
Detect and use the formal PACKAGE_FOO=FOO settings instead now.
Alex Rousskov [Mon, 14 Jun 2010 21:22:01 +0000 (15:22 -0600)]
Support ICP and HTCP _servers_ sharing listening sockets.
Without a shared cache, the servers will report many false misses.
ICP and HTCP _clients_ cannot be supported in SMP environment unless each
process has its own address (i.e., unique IP address and/or unique [ICP] port)
because we cannot match outgoing queries and incoming responses across
processes. If ICP/HTCP support in SMP is needed, the easiest change would
be to use random source ports for sending ICP/HTCP queries. This may also
improve security by avoiding well-known source ports for UDP queries.
Author: Tsantilas Christos <chtsanti@users.sourceforge.net> , Alex Rousskov <rousskov@measurement-factory.com>
Support ICAP 206 Partial Content extension
The ICAP 206 Partial Content extension allows the ICAP agents to optionally
combine adapted and original HTTP message content.
For more information about ICAP Partial Content extension look at the
draft from the ICAP forum:
projects / thirdparty / squid.git / log
