The --enable-truncate and USE_TRUNCATE code has been removed.
As discussed in Bug 1371, the truncate feature is dangerous on
async disk I/O storage schemes and more troublesome than any
possible performance benefits.
Fixed SEGV during reconfigure due to NULL pointer bug in eventDelete()
Since the cancel method may now delete multiple events (when
arg is NULL) it no longer returns after a deletion and
we have a potential NULL pointer problem. If we just
deleted the last event in the list then *E is now equal
to NULL. We need to break here or else we'll get a NULL
pointer dereference in the last clause of the for loop.
A fix for the !flags.write_draining assertion. Due to the way diskd
works, drainWriteQueue() could get called twice in the stack. If so,
just return when the write_draining flag is set.
Comment out some debugging messages that are printed before cache_log
is initialized because we see them on stderr for commands like
squid -k reconfigure.
Fixed disk file leak. UFS-based disk files were not always closed
because the storeClose() may be called when an I/O request is pending.
With this fix, UFSStoreState::writeCompleted now calls tryClosing()
if the try_closing flag is set.
The DiskdFile::close function calls the DiskdIOStrategy::send function and
consequently the DiskdIOStrategy::SEND function, with shm_offset value of -1.
If the msgsnd in SEND fails, the SharedMemory::put function is called with
offset=-1, triggering an abort.
Patch by Christos Tsantilas. Reviewed by Guido Serassio and Duane Wessels.
This change should fix bug #1837: Segfault on configuration error
When quitting on a fatal error, such as a configuration error, Squid may need
to write clean state/log files. Squid uses comm_ routines to do so. Thus, we
must initialize comm_ before such fatal errors are discovered.
Perhaps a better fix would be to avoid writing clean state/log files until
the old ones become dirty?
This change should fix bug #1898: "assertion failed: tunnel.cc:372"
The assertion fails in SslStateData::Connection::dataSent method which is
called by SslStateData::writeClientDone and SslStateData::writeServerDone
methods.
Both write*Done methods did not check for the comm_err_t flag.
When the SSL side closes the connection after a successful but partial write,
write*Done is called with the len argument equal to the number of written
bytes, which is positive but less than the size of the given data for write,
and the dataSent() assertion "amount == (size_t)len" fails.
Server transactions keep refcounted pointer to FwdState that launched them.
Server transactions use the pointer to call FwdState upon completion or
abnormal termination. If nobody else points to the FwdState object after the
transaction is done, the object gets destroyed, even if there are other
servers to be tried.
To avoid FwdState destruction when selecting the next server, FwdState must
keep a refcounted pointer to self. That pointer should be created before the
server-selection loop and cleared after the loop. Currently, we create the
pointer when FwdState is created (which is earlier than needed) and clear it
in various places (which may be late and is probably buggy).
cache.log was being created as root, then would later fail to open
with 'Permission denied'. We were calling leave_suid() after parsing
the config, but before calling configDoConfigure().
We must call configDoConfigure() before leave_suid() because
configDoConfigure() is where we turn username strings into
uid values.
Since I changed the semantics of store_dirs_rebuilding yesterday (to match
how it works in Squid-2 again), I forgot that storeRebuildComplete()
probably wouldn't do any work as long as store_dirs_rebuilding was greater
than zero. Now it needs to do its work when store_dirs_rebuilding == 1.
- Bug #1356: Close open DNS sockets when doing idnsShutdown. Leaving them
open may result in an idnsVCClosed callback being called when the name
servers array have been freed by idnsFreeNameservers.
(based on Christos Tsantilas and Guido Serassio work)
store_dirs_rebuilding is initialized to _1_ as a hack so that
storeDirWriteCleanLogs() doesn't try to do anything unless _all_
cache_dirs have been read. For example, without this hack, Squid
will try to write clean log files if -kparse fails (becasue it
calls fatal()).
The diskd helper passes errno in responses. With this patch, errno
is now set in the main squid process after processing a diskd
response.
Initialize unused fields of diomsg in messages from diskd to squid.
Previously, the diskd helper did not initialize the unused fields
of diomsg. This was annoying when looking at debugging output
and trying to match requests with responses. This patch initializes
the unused fields.
- ICAP-unrelated improvements from the squid3-icap branch on SF
(see further below for ICAP-specific improvements):
- Replaced BodyReader with BodyPipe. BodyReader was a
collection of function pointers augmented with body size
calculation logic. BodyReader was used to deliver request
body (of a known size) from the client side to the server
side. Reference counting was used to communicate abort
conditions to the other side (it did not work well because
decreasing the reference count does not have any side-effects
if the count remains positive). Direct calls between sides
sometimes resulted in a call-me-when-I-am-calling-you "loops"
and related bugs.
BodyPipe is used to deliver request or response body (possibly
of unknown size) from the body producer to the body consumer.
A producer can be the client side (for virgin requests), the
server side (for virgin replies), or the ICAP side (for
adapted messages). A consumer can be the client side (for
adapted responses, including responses in a request
satisfaction mode), the server side (for adapted requests),
and the ICAP side (for virgin requests and responses).
BodyPipe uses asynchronous calls for communication between
sides to avoid call-me-when-I-am-calling-you "loops".
BodyPipe has methods to communicate normal termination and
abort conditions to the other side. The use of those methods
is mandatory. Reference counting is used only as a garbage
collection mechanism.
BodyPipe is used to read request bodies, including requests
for which there is no consumer and the connection is in a
'closing' state. BodyPipe can auto-consume body so that a
'closing' connection does not have to rely on the body
consumer presence when eating up remaining body data.
If auto-consumption is turned on and the pipe starts
consuming before a real consumer is attached to the pipe, the
setConsumerIfNotLate call fails, and the real consumer has to
handle the failure.
The new BodyPipe approach should make support for HTTP/1.1
chunked requests easier. Only a few places in the pipe-related
code assume that the request size is known.
- Removed ClientBody as unused, replaced by BodyReader, then
BodyPipe.
- Moved HttpRequest::body_reader to HttpMsg::body_pipe so that
all HTTP message bodies can be communicated via pipes. This
is needed for the server side to supply response bodies to
ICAP and for the ICAP side to supply adapted message bodies
to others.
- When cleaning HttpRequest or HttpReply, reset body_pipe to
NULL instead of asserting that it is already NULL. BodyPipes
are owned and maintained by other objects and HttpMsg is used
only as a mechanism to pass the pipe pointer from the body
producer to the consumer. To maintain guarantees similar to
the old code, the BodyPipe destructor asserts that both the
producer and the consumer are gone when the pipe is
destructed.
- When appending body data, do not append more than the known
body size. This fixes the following assertion when POSTing
from IE in my tests: assertion failed: client_side.cc:3205:
"bodySizeLeft() > 0".
I suspect IE or some Javascripts running on IE were appending
extra CRLF to a POST, exposing the bug, and triggering the
above assertion.
- WARNING: Ftp-specific BodyPipe changes are untested, but the
old code probably did not work well with ICAP either. More
testing is needed.
- Moved more common server-side code from http.* and ftp.* into
Server.*. Most ICAP-related code is in the Server class now.
The code move to the Server class and migration to BodyPipe
exposed several FTP/HTTP inconsistencies and bugs. I marked
those I could not fix with XXXs.
- Distinguish the end of communication with the origin server
from the end of communication with ICAP. Clean them up
separately when possible. Terminate when both are completed
(or aborted).
- Polished persistentConnStatus() to avoid calling
statusIfComplete() until really necessary (and appropriate).
This makes debugging easier to understand for some.
- Use auto-consumption feature to consume data from closing
connections for which there is no real body consumer.
- Use BodyPipe for maintaining the "closing" state of a
connection instead of in.abortedSize. This change "removes" a
few memory leaks and an assertion, but does need more work,
especially when the regular BodyPipe consumer leaves early
and does not consume the request body.
- The client stream code sometimes marks the "closing"
connection as STREAM_UNPLANNED_COMPLETE, leading to a
double-close. I do not yet understand why. There is now code
to ignore multiple attempts to enter the "closing" state.
- ICAP improvements from the squid3-icap branch on SF, including:
- Added icap_service_failure_limit squid.conf option. The limit
specifies the number of failures that Squid tolerates when
establishing a new TCP connection with an ICAP service. If
the number of failures exceeds the limit, the ICAP service is
not used for new ICAP requests until it is time to refresh
its OPTIONS. The per-service failure counter is reset to zero
each time Squid fetches new service OPTIONS.
A negative value disables the limit.
The limit used to be hardcoded to 10.
(based on the patch by Axel Westerhold)
- Added icap_service_revival_delay squid.conf option. The
delay specifies the number of seconds to wait after an ICAP
OPTIONS request failure before requesting the options again.
The failed ICAP service is considered "down" until fresh
OPTIONS are fetched.
The actual delay cannot be smaller than the [still] hardcoded
minimum delay of 60 seconds.
(based on the patch by Axel Westerhold)
- Added icap_client_username_header and
icap_client_username_encode squid.conf options to control how
the authenticated client username should be sent to the ICAP
service. (based on the patch by Axel Westerhold)
- Handle REQMOD transaction failures where we cannot proceed
with the normal request flow.
- Use ICAPInitiator API to send "success" or "abort" messages
to ICAP transaction initiator. Store virgin and adapted
metadata as public fields (if the newly added ICAPInOut type)
that the initiator can access when receiving our "successful
adaptation" message. This keeps messages simple.
- Using ICAPInitiator API and a "universal" BodyPipe API makes
it possible to exchange bodies directly with client- or
server-side code without ICAPClient* translators, which are
now gone along with the ICAPInitXaction function in
ICAPClient.
- Added ICAPInitiator interface that classes initiating ICAP
transactions must now support. The API currently has just two
methods: one for receiving adapted message headers
(indicating a successful ICAP transaction, at least to the
point of fetching adapted headers) and one for receiving a
notification of an aborted ICAP transaction.
Most ICAP initiators (or their close relatives) will also need
to implement BodyConsumer and/or BodyProducer APIs to exchange
virgin and/or adapted HTTP message bodies with the initiated
ICAP transaction. However, that activity is not addressed in
this API. New AsyncCall API is used to declare the callback
wrappers.
- Use BodyPipe instead of MsgPipe for receiving virgin and
sending adapted message bodies. BodyPipe is not much
different from MsgPipeBody, but it is better to use a
"universal" class that the rest of Squid code now uses. One
complication is that BodyPipes are currently not created for
messages with zero-size bodies. The code had to be changed to
not assume that a zero-size body comes with a pipe.
- Deleted MsgPipe and related classes. Message pipes had two
purposes: coordinate HTTP message adaptation (start, get the
adapted headers, abort) and exchange HTTP message bodies. The
latter is now done via BodyPipe API. The former can be
implemented directly in ICAPModXact.
Deleted ICAPClient* and related classes as (my) design
failure.
The original idea behind message pipes and ICAPClient* classes
was to isolate ICAP code from the Squid core. The core code
was supposed to use ICAPClient* classes for all ICAP-related
needs, and ICAPClient* classes were supposed to translate core
needs into "ICAP needs" and use message pipes to communicate
with asynchronously running ICAP transactions. The latter part
worked fine, but the former did not.
The core code still did a lot of ICAP-specific work on its
own. This could be because ICAP processing affects the flow so
much or because the core code had not been refactored enough
to minimize ICAP interactions. Whatever the reason, we ended
up with a lot of complex code/logic coordinating the core code
and ICAPClient* classes. While ICAPClient* classes were
"translating", they could not hide the key actions or events
(such as message body exchange or transaction aborts) from the
core. The core code still had to support those actions or
handle those events. Thus, every major action or event was
handled twice: once in the core side code and once in a
ICAPClient* class.
Removing ICAPClient* "translation" step simplified the code
and possibly improved performance. As for the "ICAP
separation" goal, the current exposure to the ICAPModXact
class can be hidden by a generic "Message Adaptation
Transaction" class if we need to support more adaptation
protocols. The core code should not be affected much by such a
change.
- ClientHttpRequest: Support the new ICAPInitiator API and talk
to ICAPModXact directly instead of using ICAPClient* classes,
which are now gone.
- ConnStateData: Use BodyPipe for delivering virgin request
bodies to the server or ICAP side. Implement the BodyProducer
interface. ClientHttpRequest: Use BodyPipe instead of
BodyReader when receiving request bodies (from client side or
ICAP). Implement the BodyConsumer interface. See the first
BodyPipe CVS log message for the rationale.
- Use BodyPipe for delivering virgin reply bodies to ICAP and
receiving adapted reply bodies from ICAP. Implement the
BodyProducer interface.
Use BodyPipe instead of BodyReader when receiving request
bodies (from client side or ICAP). Implement the BodyConsumer
interface.
- Replaced never-failing doIcap() with startIcap() that fails
if we cannot select an ICAP service or the selected service
is not usable. Rearranged
ClientRequestContext::icapAclCheckDone() to bypass ICAP
errors when possible. Now, ClientRequestContext::startIcap()
is very similar to Server::startIcap(). Same for
icapAclCheckDone(). Made
ClientHttpRequest::handleIcapFailure() public because
ClientRequestContext::icapAclCheckDone() calls it.
- Polished TTL handling to make sure we use the default TTL
when the ICAP server did not provide an explicit value or if
we failed to communicate with the server. The latter case may
not have been handled correctly before.
- The minimum options update gap (currently hard-coded) must be
smaller than the default options TTL. Otherwise, we get stale
options and down ICAP services around the update time because
we cannot update soon enough.
- Support asynchronous transaction start. This allows for a
better handling of startup errors (or at least makes them
similar to other transaction errors).
- Call a swanSong() method upon expected transaction
termination (including aborts). This allows for proper and
prompt [partial] transaction cleanup, without waiting for the
destructor to be called. The destruction may be delayed by
refcounting if we have other transaction users waiting for
some transaction notifications.
- Do not reuse a connection if we are still reading or writing
(even if no actual I/O is scheduled). The old code would
reuse such connections, and read/write leftovers from aborted
transactions from/to the ICAP server.
- Do not send last-chunk in ICAP Preview with a null-body. It is
possible that the old code would send the last-chunk under
some Preview conditions with null-body, but I am not sure.
- Fixed HttpStateData memory leak visible when no RESPMOD
services are enabled. ICAPAccessCheck constructor was
cbdata-locking HttpStateData, but was not releasing the lock
when there was no matching service class, leading to an
HttpStateData leak. Furthermore, ICAPAccessCheck would then
call HttpStateData back without validating the cbdata
pointer, probably calling wrong or invalid HttpStateData.
- Fixed "is it too late to bypass?" conditions in
ClientHttpRequest::handleIcapFailure(). We should be able to
bypass more often now. However, handleIcapFailure() still has
the old bug: it does not check whether the service is
optional. The current fix implies that now Squid may bypass
essential services more often.
- Call storeEntry()->complete() when ending request
satisfaction. Without this call, we may keep the connection
open, which does not work with responses that mark the end of
their body by closing a connection. (Christos Tsantilas)
- Fixed ieof condition detection. Squid was sending last-chunk
without ieof bit and was sending two last chunks when doing
preview (Tsantilas Christos).
- When ICAP server wants the entire virgin body and sends 100
Continue after Preview, do not stop backing up virgin body
data for echoing if we promised to support 204 No Content
responses outside of Preview. If we allow 204s, 100 Continue
may be followed by a 204 No Content and we will need the
entire virgin body to echo back.
- Rewrote MemBufClaim into a VirginBodyAct class to simplify
and clarify code in hope to minimize the number of bugs like
the one mentioned above. MemBufClaim was protecting an area
of virgin body pipe content and, as a side effect, was
providing the transaction with the content pointer for the
write or send actions.
Now VirginBodyAct just maintains the activity offset and the
transaction code uses that to consume virgin body correctly.
The size of the area is no longer maintained because it was
usually unknown or unused; and when it was known and used
(i.e., Preview), it could be taken from the preview state
object anyway. Renamed and documented VirginBodyAct-related
methods to clarify the intent.
- When sending last-chunk in Preview, send ieof extension if we
wrote the entire body. The old code would not send ieof if we
wrote as many bytes as promised in the Preview header, even
if we promised to write everything. This would mislead
compliant ICAP servers that do not look at the Content-Length
header and reply with 100 Continue, expecting more body data.
- Do not reset Preview size to zero when expecting a virgin
body of unknown size. A Squid user reported that this change
works.
- Polished debugging: Instead of using pointers, use unique ICAP
transaction IDs. This helps with isolating a transaction in a
large log, where pointers may be reused many times. Print
connection descriptor like most of the core code does. Other
minor improvements.
- Replaced BodyReader with BodyPipe. BodyReader was a
collection of function pointers augmented with body size
calculation logic. BodyReader was used to deliver request
body (of a known size) from the client side to the server
side. Reference counting was used to communicate abort
conditions to the other side (it did not work well because
decreasing the reference count does not have any side-effects
if the count remains positive). Direct calls between sides
sometimes resulted in a call-me-when-I-am-calling-you "loops"
and related bugs.
BodyPipe is used to deliver request or response body (possibly
of unknown size) from the body producer to the body consumer.
A producer can be the client side (for virgin requests), the
server side (for virgin replies), or the ICAP side (for
adapted messages). A consumer can be the client side (for
adapted responses, including responses in a request
satisfaction mode), the server side (for adapted requests),
and the ICAP side (for virgin requests and responses).
BodyPipe uses asynchronous calls for communication between
sides to avoid call-me-when-I-am-calling-you "loops".
BodyPipe has methods to communicate normal termination and
abort conditions to the other side. The use of those methods
is mandatory. Reference counting is used only as a garbage
collection mechanism.
BodyPipe is used to read request bodies, including requests
for which there is no consumer and the connection is in a
'closing' state. BodyPipe can auto-consume body so that a
'closing' connection does not have to rely on the body
consumer presence when eating up remaining body data.
If auto-consumption is turned on and the pipe starts
consuming before a real consumer is attached to the pipe, the
setConsumerIfNotLate call fails, and the real consumer has to
handle the failure.
The new BodyPipe approach should make support for HTTP/1.1
chunked requests easier. Only a few places in the pipe-related
code assume that the request size is known.
- Removed ClientBody as unused, replaced by BodyReader, then
BodyPipe.
- Added basic support for asynchronous calls. The calls are
implemented using Squid events, at least for now. The
provided functions and macros provide simple debugging at the
call scheduling and execution times. Macros are used to log
the location of the call and the call handler. For a simple
usage sketch, please see AsyncCall.cc.
It may be possible to customize call scheduling and debugging
and in a class context, but I have not tested that theory yet.
The code was inspired by ICAP/MsgPipe implementation, that is
being deleted for reasons unrelated to asynchronous calling.
- Support printing refcounted pointers to ostreams. When
printing, report the reference count after the raw pointer.
This makes debugs() statements simpler and their output more
informative. Null pointers are printed as a "NULL" string.
Author: Amos Jeffries <squid3@treenet.co.nz>
IPV6 DNS and address formats RFCs
rfc2874.txt
DNS Extensions to Support IPv6 Address Aggregation and Renumbering
Documents IPv6 reverse DNS lookups.
fc3226.txt
DNSSEC and IPv6 A6 aware server/resolver message size requirements
Documents resolver requirements for IPv6 DNS structures and handling
of advanced IPv6 lookups of A6 and DNAME records.
rfc3513.txt
Internet Protocol Version 6 (IPv6) Addressing Architecture
Documents handling requirements for IP addresses under IPv6
and also new special case addresses defined by IANA.
rfc3596.txt
DNS Extensions to Support IP Version 6
Documents AAAA record details for basic IPv6 DNS resolvers.
hno [Sun, 25 Feb 2007 18:32:27 +0000 (18:32 +0000)]
Kill the psuedo-support for wais. Was never really implemented.
This change removes the wais_relay_host/port directives. WAIS support was
never really implemented but relied on forwarding the requests to some other
proxy/gateway using HTTP. Now this can just as well be done with cache_peer
so there is no point in having this duplicated and unmaintained forwarding
code around.
serassio [Sun, 28 Jan 2007 22:37:46 +0000 (22:37 +0000)]
Bug #1865: deny_info redirection with authentication related acls
this patch modifies deny_info to not redirect when authentication
is required. Any redirect deny_info lines is ignored if the request
was not authenticated.
hno [Wed, 3 Jan 2007 19:57:47 +0000 (19:57 +0000)]
Cleanup of AuthUserRequest->lastReply to keep ACL_AUTHENTICATED state
short-circuiting acl processing a bit when there is multiple auth acls
involved, and preparing for future auth related work.
hno [Wed, 3 Jan 2007 19:39:48 +0000 (19:39 +0000)]
Bug #1792: max_user_ip not working with ntlm_auth
the ntlm/negotiate scheme reqrite was a bit confused about the FINISHED
vs DONE states, skipping a little of the processing required for the
user and ip caches.
serassio [Sun, 24 Dec 2006 20:52:40 +0000 (20:52 +0000)]
Bug #1584: Unable to register with multiple WCCP2 routers
Disable PMTU-discovery on the WCCP2 UDP socket, making Linux and possibly
others send unique IP ID fields in the UDP traffic. Hopefully this will
convince Cisco IOS that the WCCP exchanges is not duplicate traffic.
serassio [Sun, 24 Dec 2006 20:43:08 +0000 (20:43 +0000)]
Bug #1840: Disable digest and netdb queries to multicast peers
Sending HTTP requests to the multicast group does not make much sense. Force
digest and netdb exchanged to be disabled on the multicast group. Such
exchanges only makes sense on the individual peers.
hno [Thu, 14 Dec 2006 08:36:01 +0000 (08:36 +0000)]
wccp2_router config fixes. Should not need to specify the port.
in fact the port is currently ignored completely, but the config
type (sockaddr_in_list) did not understand host specifications alone
only port or host:port).
Now sockaddr_in_list supports port, host:port and host.
Fully numeric host names without domain is not supported.
* change the timekeeping to use a fixed-size stack of times rather than
directly referencing the time counter structs
* start/stop the previous level timers during profiling stop/start
(someone could probably turned the UNACCOUNTED timer into the "level 0 timer"
taking out the special cases..)
* Add in some more profiling points which'll surface in later commits.
projects / thirdparty / squid.git / log
