Amos Jeffries [Tue, 10 Aug 2010 02:51:49 +0000 (20:51 -0600)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
HTTP/1.1 Compliance: Improved HTTP Range header field validation.
1) Improved HttpHdrRangeSpec::parseInit() to parse syntactically valid
range specs:
* Suffix ranges with 0 length (i.e. -0) are syntactically valid.
* Check that last-byte-pos is greater than or equal to first-byte-pos.
After the change, HttpHdrRangeSpec::parseInit() successfully parses suffix
ranges with 0 length. They were rejected before. RFC 2616 section 14.35.1 says
such range specs are syntactically valid but unsatisfiable. Thus, we should
ignore the range spec itself, but not the whole range header. These range
specs will be rejected later, during canonization.
2) In HttpHdrRangeSpec::parseInit(), ignore the whole range header if one of
range specs is syntactically invalid (i.e. range spec parsing fails).
Co-Advisor test case: test_clause/rfc2616/invalidRange
Amos Jeffries [Sun, 1 Aug 2010 12:54:47 +0000 (06:54 -0600)]
Basic split-stack functionality
Enable split-stack detection by default.
There is now enough split-stack support to enable accepting IPv6 client
connections on systems with separate IPv4/IPv6 stacks. Also some limited
server connection capabilities (see tcp_outgoing_addr config hacks).
SNMP, ICP, HTCP listeners and outbound connections currently default to
IPv4-only on these systems to retain backward-compatibility.
But may be explicity configured to IPv6-only. There is no support as yet
for dual-protocol behaviour in the sockets of these three protocols.
Amos Jeffries [Sun, 1 Aug 2010 12:53:19 +0000 (06:53 -0600)]
Hack: define top_build_prefix for old autotools
This hack was removed during the libtool 2.2 upgrade.
The issue shows up as bundle builds failing to link libltdl/libltdlc.la
when they should in fact be linking ../libltdl/libltdlc.la in src/Makefile
It is caused by the macros of libtool 2.2 assuming the presence of
top_build_prefix but since autoconf 2.62 that was replaced with
ac_top_build_prefix and is no longer automatically defined in Makefile's.
Some distros also seem to have back-ported the removal of top_build_prefix
into their old autoconf causing macro version tests to fail.
httpHdrCcParseInit() ignored all unknown Cache-Control directives
except for the first one because the (type != CC_OTHER) check
applied to the debugging statement only.
Co-Advisor test case: test_case/rfc2616/endHdr-fwd-set-Cache-Control-toSrv
Bug 2994: pt 1: Open *_port directives correctly in IPv4-only mode.
Was opening snmp_port, icp_port, htcp_port under the v4-mapping assumption.
This forces the ports both listening and outgoing to IPv4-only unless
v4-mapping is actually available in the system.
Bug fix: 32bit integer used in HttpStateData to store the bytes received from next hop
A simple integer used to store the bytes received from the next hop
(http server of proxy), which my cause problems when receives huge
http objects on 32bit systems.
Bug fix: The bytes sent/received to/from the ICAP server may not logged correctly on 32bit systems
A simple long int (doint/outint) used to log bytes sent/received to/from
the ICAP server, ICAP requests with very large http objects will not
logged correctly. Use int64_t (dooff/outoff) instead
Replace most USE_IPV6 with run-time support probing
This unifies the code built for IPv4-only, dual-stack and split-stack.
* --disable-ipv6 option remains, however it now prevents the run-time probe
* Probing previously done in ./configure at build time is now merged and
performed run-time on every startup. IPv6 is enabled or disabled based on
the underlying OS support for sockets and setsockopt operations required.
* Parsing and other operations which can be performed without specific IPv6
connectivity are enabled.
* Some DNS logic alterations have had to be made to merge the split-stack
DNS and leverage it for IPv4-only mode. Otherwise the logics are unchanged
from previous dual-stack builds which have been well tested.
Client side must stop reading when switching to a tunnel mode. The old code
called low-level commSetSelect to stop reading, but that left Comm tables in
an inconsistent state, with the client side reader callback still scheduled.
Squid would assert when the tunnel called comm_read with its own callback.
The bug is unrelated to half-closed connections despite halfClosedReader
mentioned in the assertion text. The assertion means "no more than one active
reader per FD".
Alex Rousskov [Fri, 16 Jul 2010 22:37:42 +0000 (16:37 -0600)]
Added debugging scripts that work with detailed cache.log
scripts/find-alive.pl: pinpoint objects that are still alive, to find leaks
scripts/trace-job.pl: find cache.log lines that correspond to a given job
scripts/trace-master.pl: trace jobs related to a single master transaction
The scripts require maintenance as the logging format changes, but
they often simplify debugging by extracting relevant information from
tons of poorly structured cache.log data.
Alex Rousskov [Tue, 13 Jul 2010 16:43:00 +0000 (10:43 -0600)]
Prevent memory leaks when cloning Range requests.
HttpRequest::range field was set to a new HttpHdrRange object twice:
once in HttpRequest::clone() and once in HttpRequest::hdrCacheInit()
called from clone().
Polished HttpReply::clone() to make sure HttpReply::hdrCacheInit()
does not use uninitialized HttpReply::sline field and to prevent
benign double-initialization of HttpReply::keep_alive.
Alex Rousskov [Wed, 7 Jul 2010 03:17:47 +0000 (21:17 -0600)]
Moved KidIdentifier to globals to make pinger happy.
Pinger and possibly other optional externals require KidIdentifier via
debugs(). Instead of making KidIdentifier global, we could add a "plugin" API
to add program-dependent stuff to debugs() and friends, but we should not add
that kind of complexity unless really necessary. We could also link pinger
with libipc.la but that will probably cause more problems with IPC
dependencies.
Alex Rousskov [Wed, 7 Jul 2010 00:45:34 +0000 (18:45 -0600)]
SMP support, part 1: Essential non-caching functionality.
Added workers squid.conf option to specify how many main Squid
processes to fork and maintain. Zero means old no-daemon mode.
One means the old non-SMP mode.
Added support for process_name and process_number macros and
if-statement conditionals in squid.conf. Search for .pre changes for
documented details. These features allow the admin to configure each
worker process differently if needed.
Support multiple workers listening on the same HTTP[S] port (port
sharing). This allows multiple workers to split the load without any
special rules.
Support or prohibit port sharing for WCCP, DNS, ICP, HTCP, SNMP, and
Ident protocols, depending on protocol-specific restrictions. Sharing is
implemented by registering listening socket descriptors with the
Coordinator process and obtaining them from the Coordinator as needed.
Here are protocol-specific notes:
WCCP: Restricted to the Coordinator process due to how WCCP works.
Workers do not need access to the WCCP code.
DNS: Done by each worker with no sharing. Fixed source ports not
supported unless each worker is given its own outgoing address
because we do not want to match outgoing queries and incoming
responses across processes.
SNMP: Workers share incoming and outgoing sockets.
ICP and HTCP _clients_: Cannot be supported in SMP environment
unless each process has its own address (i.e., unique IP address
and/or unique [ICP] port) because we do not want to match outgoing
queries and incoming responses across processes.
ICP and HTCP _servers_: share listening sockets.
Ident clients do not need to share sockets because they use
unique ports.
Support management signals (squid -k ...) in SMP mode, acting as a
single Squid instance.
Refork dying workers, similar to how we reforked dying process in
non-SMP daemon mode.
Alex Rousskov [Wed, 7 Jul 2010 00:22:37 +0000 (18:22 -0600)]
Do not stop testheaders.sh on the first error, to be compatible with
"make -k check".
If testheaders.sh stops on the first error, it becomes very difficult to
test your changes with "make check" when somebody else broke some other
code or when the checks reach src/cf_parser.h and fail.
TODO: stop on the first error unless running under "make -k check".
projects / thirdparty / squid.git / log
