hno [Thu, 14 Dec 2006 08:36:01 +0000 (08:36 +0000)]
wccp2_router config fixes. Should not need to specify the port.
in fact the port is currently ignored completely, but the config
type (sockaddr_in_list) did not understand host specifications alone
only port or host:port).
Now sockaddr_in_list supports port, host:port and host.
Fully numeric host names without domain is not supported.
* change the timekeeping to use a fixed-size stack of times rather than
directly referencing the time counter structs
* start/stop the previous level timers during profiling stop/start
(someone could probably turned the UNACCOUNTED timer into the "level 0 timer"
taking out the special cases..)
* Add in some more profiling points which'll surface in later commits.
wessels [Wed, 1 Nov 2006 06:30:55 +0000 (06:30 +0000)]
- Many ICAP fixes from Alex Rousskov accumulated on the
sourceforge squid3-icap branch since 2006/10, including:
- Polished ICAP service selection code and implemented bypass of
optional services. The code implements icap_class
configuration directive which is currently used as a "set of
interchangeable ICAP services". Squid2 and current squid.conf
may imply otherwise.
- Support Transfer-* ICAP OPTIONS response header. If Squid
knows that a service does not want the URL, Squid will not use
the service, even if it is an essential service with
bypass=0. Note that we may make this decision before we know
what the service wants. Eventually, ACLs should initiate and
wait for the OPTIONS transaction for yet-unprobed services.
- When ICAP transactions fail to connect to the service many
times, the service is suspended until the next OPTIONS
update. The limit is currently hard-coded to 10. Suspended
service is a down service and will be skipped by the ACL
service selection algorithm.
- Rewrote the code updating ICAP service options. We no longer
mark the service being updated as "down". Only presence of
valid and fresh options is important. We also try to update
the options before they expire to avoid any service downtime
or use of stale options.
- Report interesting changes in the ICAP service state, some
with debugging level one to alert the cache administrator.
- When cloning a request during an ICAP 204 "No Content" REQMOD
response, preserve the client address so that the rest of the
code has access to it. This change appears to fix Squid Bug
#1712.
- After ICAP 100 Continue, expect new ICAP headers instead of
HTTP headers. Reset ICAP message object to be ready to parse
ICAP headers again. (Tsantilas Christos
<chtsanti@users.sourceforge.net>)
- The ieof HTTP chunk-extension was written after chunk-data
instead of being written after the chunk-size. (Tsantilas
Christos <chtsanti@users.sourceforge.net>)
- Merged common code from the ICAPClientReqmodPrecache and
ICAPClientReqmodPrecache classes into the newly added
ICAPClientVector class. The specific vectors do not have a
common owner (yet?) because ServerStateData and
ClientHttpRequest do not have a common base class. Thus,
ICAPClientVector has to rely on its kids to communicate with
their owners. However, at least 50% of the logic was common
and has been moved. Eventually, we may want to create a
simple ICAPOwner API that ServerStateData and
ClientHttpRequest can implement and ICAPClientVector can rely
on. This will make the code simpler and more efficient. The
big merge was motivated by a couple of bugs that were found
in one vector class but that did not exist or behaved
differently in the other vector, mostly likely due to natural
diversion of used-to-be identical code.
- Rewrote communication between a server-side ICAPClient*mod*
vector and its owner. When a server-side ICAPClient*mod*
vector was notifying its owner of more adapted data, the
owner could delete the vector (by calling icap->ownerAbort)
if the store entry was not willing to accept the data. The
same deletion could happen when a vector was notifying the
owner of a successful termination. In all those cases, the
vector did not expect to be deleted and could continue to do
something, causing segmentation faults. Now, when more data
is available, the vector calls its owner and checks the
return value of the call. If it is false, the vector knows it
has been deleted and quits. When vector terminates, it calls
its owner and trusts the owner to always delete the vector.
The "check return value and quit" design is not perfect, but
we are paying the price for isolating the vectors from their
owners while using direct calls between them (instead of
MsgPipe or a similar less efficient indirect approach we use
elsewhere).
- Renamed doIcap to startIcap and moved more common code there.
Changed its return type to bool. We now handle three cases
when ICAP ACLs call back: 1) No service was selected
(because there was no applicable service or because all
applicable services were broken and optional). We proceed as
if ICAP was not configured. 2) The selected essential
service is broken. This is a fatal transaction error and we
return an "ICAP protocol error" HTTP error response. We could
proceed with the ICAP stuff, but it saves a lot of cycles to
abort early. 3) The selected service is not broken. We
proceed with the ICAP stuff. The old code did not detect
case #2, even though there was code to handle that case (with
dangerous XXX suggestions that are now gone). The code
should probably be polished further to move common ftp/http
logic from icapAclCheckDone()s into ServerStateData.
- Make sure there is an accept callback when we are accepting.
If there is no callback and we accept, we will silently leak
the accepted FD. When we are running out of FDs, there is
often no accept callback. The old code, when running out of
FDs, would create lots of "orphaned" or "forgotten" FDs that
will eventually get into a CLOSED_WAIT state and remain there
until Squid quits. The new code does not call accept() if
there is no accept callback and does not register the accept
FD for reading if the AcceptLimiter is deferring, because
when the AcceptLimiter kicks in, it will register the accept
FD for reading. There are most likely other places/cases
where accept FD should not be registered for reading.
- When an exception is caught, mark the ICAP connection as
non-reusable so that it is not recycled while a write is
pending but simply closed instead. Our write callback will
still be called, unfortunately, because there is no way to
clear the callback without invalidating its data (i.e., the
transaction pointer). This change prevents pconn.cc:253:
"!comm_has_incomplete_write(fd)" assertion from firing when
things go wrong (e.g., the ICAP server cannot be contacted to
retrieve OPTIONS). Not all exceptions caught by the ICAP
xaction should lead to the ICAP connection termination, but
it is very difficult if not impossible to reliably detect
exceptional conditions when it is safe to reuse the ICAP
connection, and it is probably not worth it anyway.
- Added Tsantilas Christos <chtsanti@users.sourceforge.net>
to CONTRIBUTORS for fixing ICAP bugs.
wessels [Tue, 24 Oct 2006 10:48:10 +0000 (10:48 +0000)]
Alex reports getting coredumps (with high debugging) at process
exit because debug_log got closed, but was not set to NULL. Seems
like we should not call fclose() before exit so that destructors
can write debugging if necessary. The file will be closed anyway
when the process truly exits.
wessels [Thu, 19 Oct 2006 07:39:40 +0000 (07:39 +0000)]
bugfix: In clientProcessRequest(), the call to connNoteUseOfBuffer() was
moved from the beginning to the end of the function. This broke request
body processing because "conn->in.notYetUsed" was wrong at the time
of BodyReader creation. As a workaround we now subtract http->req_sz
from conn->in.notYetUsed when telling the BodyReader how much
data there is on the socket.
wessels [Fri, 13 Oct 2006 02:46:42 +0000 (02:46 +0000)]
Removing port 563 from the default SSL_ports and Safe_ports ACLs
under the assumption that this port (for secure NNTP) is very
rarely used through Squid, and that allowing it by default increases
the chance that it can be abused for generic tunneling.
adrian [Mon, 2 Oct 2006 07:34:18 +0000 (07:34 +0000)]
Fix/Add request-line parser debugging; fix bug with pipelined connection parsing
The unparsed request buffer was being relocated before the headers were being
parsed. That was fine - the request parsing used to happen with a copy of the
buffer - but now a copy isn't being made. The buffer relocation needed to be
changed to happen after the request was parsed.
adrian [Wed, 27 Sep 2006 19:17:52 +0000 (19:17 +0000)]
Replace the client-side request line parser
* remove the 'inbuf' copy during parsing;
* .. but create a 'url' temporary copy for now since the *URL() routines
in client_side.cc expect a mutable zero-terminated string;
I looked into it and it won't take much work to rework those routines
to take an immutable buffer.
adrian [Tue, 26 Sep 2006 21:19:22 +0000 (21:19 +0000)]
Various changes to break out the request parsing state and remove extra malloc/copies
* Create a HttpParser structure to encapsulate the current parsing state
* Add some functions to return the current request/header length with some checks
(which are expensive and should be inlined later once the code is proven to
work.)
* Use the connection buffer rather than a temporary buffer to parse the request
headers, saving a malloc and copy (the 'prefix' buffer)
ICAP cleanup of some cbdataReference misunderstanding.
cbdataReference should only be used to create additional callback references
to a cbdata object. The owner of the object is the owner and has automatic
rights to the object, and should not call cbdataReference.
adrian [Wed, 20 Sep 2006 06:59:26 +0000 (06:59 +0000)]
Bypass the expensive check to the allocator getStats() when obtaining inuse data
A lot of busy code calls getStats() to get the inuse values when calculating
queue length (diskthreads) and store memory use for object memory expiry.
This was destroying performance.
The changes implement a "shortcut" to obtain the inuse values for both the
malloc and mempool allocator. getStats() is still expensive for mempools
(as it does GC before updating all the statistics to get accurate information)
but code has been modified to not use getStats() where all it wanted was
inuse info.
adrian [Tue, 19 Sep 2006 13:56:57 +0000 (13:56 +0000)]
First part of a comm layer rework
* Remove new-style CommWrite, CommFill
* Add in a low-overhead C replacement for comm_read and comm_write
* Remove the difference between comm_write and comm_old_write; everything
is back to using the same API for now
* create a simple C-based read/write IO callback mechanism, used by
comm_read and comm_write
Whats left to do:
* convert accept/connect to using the new C-based read/write IO callback
mechanism (and remove the requirement to allocate/free transient
structures)
* rework the deferred read manager code somewhat
* then, remove the new-style CommRead code and use comm_read + whatever
the deferred read manager has become
* remove whats not being used
projects / thirdparty / squid.git / log
