Amos Jeffries [Thu, 20 Mar 2008 11:30:19 +0000 (23:30 +1200)]
3.1 Cleanups pt 1: Add testheaders.sh script
This script automatically tests all .h files in any directory it is passed
(relative to curent make directory) using the compiler options it is also passed.
It should be integrated in to the unit-testing in every Makefile.am so that
it provides dependancy tree testing of all .h without any additional work by
developers, maintainers, or testers.
- puts testheaders.sh script to use in many Makefile.am
(tried for all subdirs tested, but some may have escaped)
- fixes many include problems where .h were not self-sufficient.
They required .cc or other .h files using them to include additional .h
Now any given .h will include all objects it requires for compiling.
(linking is still not guaranteed by this alone)
- References SQUIDHOSTNAMELEN properly as defined by RFC 2181.
Now also split into a proper rfc2181.h for use.
(obsoleting many dependencies on squid.h all by itself)
- Migrates dlink_* (squid custom double-linked-list code) into new object.
(releasing many dependencies on squid.h, structs.h, and protos.h)
- Migrates the SquidConfig* types into SquidConfig.cc, now built into libsquid.la
(releasing many dependencies on structs.h, and globals.*)
- removes squid.h includes from many unit-tests
(releasing them from link-dependencies, .o links not yet cleaned up)
Henrik Nordstrom [Thu, 20 Mar 2008 05:15:04 +0000 (06:15 +0100)]
At the sprint the issue about our current 10ms main loop timeout came
up, and it was suggested the problem most likely have been fixed in
HEAD. And even if it hasn't been fixed it's something which should be
fixed rather than plastered over by spinning around on a short timeout
when there is no activity..
This backs out the change to revert the change..
Alex commit reversing the change said:
revno: 8332
committer: rousskov
branch nick: HEAD
timestamp: Tue 2007-07-24 01:55:21 +0000
message:
Reversed bug #2011 fix because it may slow down ICAP, BodyPipe, and other code
using zero-delay events to implement "asynchronous" calls.
The code should probably be rewritten (a) to avoid any waiting/blocking when
there are ready events and (b) to allow waiting longer when there are no ready
events.
Author: Robert Collins <robertc@robertcollins.net>
This is most of a changeover of scripts for squid 3 trunk to use bzr;
the missing bit appears to need a bzr 1.1 (to do 'rdiff' basically) or
thereabouts, I'll look into that in a bit. I'm not sure that the
unconverted cvs calls will actually trigger with our current setup or
not.
amosjeffries [Wed, 27 Feb 2008 04:49:32 +0000 (04:49 +0000)]
Merge auto-docs branch. pt 2
- Updates build config (dyn,html) for portable dynamic and static docs
- Merges all code-comment changes
- Updates some function parameter types to reduce silent conversions
- Fixes some typos.
- Migrates some TODO items to the automatic TODO list.
rousskov [Wed, 27 Feb 2008 01:43:30 +0000 (01:43 +0000)]
Bug #2224 fix: reentrant debugging crashes Squid
Reentrant debugging occurs when something being written into the debugging
stream produces its own debugging. For example, a field accessor method may
make cbdata-validation calls, which would produce debugging. Logging such
field would crash Squid if sufficiently high debugging level is enabled.
With this change, the Debug methods detect reentrant calls and mostly ignore
them, allowing the caller to append debugging information to the existing
debug stream. A short debugging label is added before and after the reentrant
debugging message to reduce confusion that overlapping debugging statements
may cause during log analysis.
rousskov [Tue, 26 Feb 2008 07:24:00 +0000 (07:24 +0000)]
Added a CPPUNIT assertion to test whether a failed CPPUNIT test case properly
destructs local objects on exit.
If a previous test case fails for any reason, the new assertion will fail
instead of store_dir.cc aborting. This may save us from tracking non-existent
StoreController initialization bugs.
rousskov [Tue, 26 Feb 2008 07:22:07 +0000 (07:22 +0000)]
Bug #2240 fix: AsyncCall-related unit tests fail
Removed EventDispatcher references, disabling the following test cases:
testRunOnce, testRegisterDispatcher, testRegisterEngine,
testEngineTimeout, testStopOnIdle.
Eventually, we will add AsyncCallQueue test cases, but EventLoop testing
does not have to wait for that.
rousskov [Tue, 26 Feb 2008 07:21:13 +0000 (07:21 +0000)]
Bug #2240 fix: AsyncCall-related unit tests fail
Removed EventDispatcher references, deleting testDispatch test case.
The test cases now use the AsyncCall queue to force call firing
without a real main loop.
Eventually, we will add AsyncCallQueue test cases, but event testing
does not have to wait for that.
rousskov [Tue, 26 Feb 2008 07:19:29 +0000 (07:19 +0000)]
Bug #2240 fix: AsyncCall-related unit tests fail
Removed libsquid.la from SWAP_TEST_LDADD: Store-related test cases use comm
stubs. They must not include libsquid which provides the true comm code.
Added AsyncCall* sources to selected test cases: Every test case that needs
event.cc also needs AsyncCallQueue and AsyncCall. Placed all three in
$TEST_CALL_SOURCES variable and used as needed. This prevents test case
compilation failures due to undefined call-related symbols.
I suspect that there are more files in the same "calls" group, but it would be
a lot easier to identify them once we clean src/ up and group files into
subdirs and libraries.
rousskov [Tue, 26 Feb 2008 07:15:48 +0000 (07:15 +0000)]
Protos.h said to remove HttpRequestMethod.h #include when no method_t is used
by prototypes. Removed the header and adjusted other source files that relied
on that header being in protos.h.
serassio [Wed, 20 Feb 2008 03:15:09 +0000 (03:15 +0000)]
Author: Klaubert Herr <klaubert@gmail.com>
Patch to strip kerberos realm from username
This patch add a new option to squid_ldap_group to strip kerberos realm from
username received from squid.
This is useful when you make kerberos authentication on squid, and try to
authorize the user using ldap in MS Active Directory, quering for
sAMAccountName.
rousskov [Tue, 19 Feb 2008 04:54:07 +0000 (04:54 +0000)]
Bug 2224 fix: reentrant debugging crashes Squid
This patch does not fix the true/core problem, but it changes some pieces of
code that expose that problem. There are probably other pieces not masked
by this patch.
rousskov [Sun, 17 Feb 2008 00:41:55 +0000 (00:41 +0000)]
When METHOD_ENUM_END and METHOD_OTHER were reordered, the rest of the
code was not reviewed and an infinite loop in
clientReplyContext::purgeAllCached was created.
Polished the ++ operator and added a TODO to replace it eventually.
rousskov [Sat, 16 Feb 2008 00:26:00 +0000 (00:26 +0000)]
Bug 2225 fix: method CONNECT acl wrongly applied to method GET
The method comparison operator was broken for all methods other than "other".
I did not check whether we need to do case-insensitive compare of method
strings when the method is "other". We have to if Squid does not convert all
unknown methods to uppercase.
rousskov [Wed, 13 Feb 2008 07:01:47 +0000 (07:01 +0000)]
Merging async-call branch changes to HEAD:
Removed class fdc_t. After CommCallbackData removal, fdc_t was
only used to check that a FD is "active" and to check that a FD
is half_closed. fd_table flags.open seems to be identical to
the "active" state flag so we use that now, via newly added
isOpen() static function. AbortChecker already maintains
half_closed status.
Synced ipc code to call fd_open instead of fdc_open.
rousskov [Wed, 13 Feb 2008 07:00:57 +0000 (07:00 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
Below you will find log messages from the async-call branch that are
relevant to the file(s) being committed.
Made comm_read and comm_write calls to use the new CommCalls.
Converted the ConnStateData related code in client_side.cc to
use CommCalls.
BodyPipe now uses the new job calls interface. Many changes in
BodyPipe's related code.
rousskov [Wed, 13 Feb 2008 06:57:47 +0000 (06:57 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
Below you will find log messages from the async-call branch that are
relevant to the file(s) being committed.
Removed CommDispatcher as unused.
Use async calls for firing ready events. Deleted
EventDispatcher as unused.
Converted CompletionDispatcher-based SignalDispatcher into
AsyncEngine-based SignalEngine to get rid of CompletionDispatchers.
rousskov [Wed, 13 Feb 2008 06:55:26 +0000 (06:55 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
Below you will find log messages from the async-call branch that are
relevant to the file(s) being committed.
Convert the comm_* calls to use CommCalls.
Use the AsyncJob::deleteThis method as "delete this"
replacement instead of the previously commited block "if
(inCall) musStop(...) else delete this"
ICAPInitiate::sendAnswer dialers take care of message locking
now.
rousskov [Wed, 13 Feb 2008 06:51:37 +0000 (06:51 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
Below you will find log messages from the async-call branch that are
relevant to the file(s) being committed.
Added initial implelentation of AsyncCall-based wrappers for
comm callbacks. The comm layer no longer calls callbacks from
the select loop. Instead, the select loop schedules async
calls. Scheduled calls are then called from the main loop (like
all other async calls), after the select loop completes.
Removed accept loop. We cannot loop because async calls do not
get fired during the loop and, hence, cannot register new
callbacks for new ready FDs.
The loop is implicit now. When the next callback is registered,
we check whether the last accept(2) call was successful or
OPTIMISTIC_IO is defined and call acceptNext() again if yes.
AcceptNext() may schedule another async call (using the
being-submitted callback) if the socket was still ready. Since
callbacks are fired until there are no callabacks left, we
still have an accept loop.
Removed CommDispatcher as unused.
Removed unused IOFCB, IOWCB, and CWCB.
Removed class fdc_t. After CommCallbackData removal, fdc_t was
only used to check that a FD is "active" and to check that a FD
is half_closed. fd_table flags.open seems to be identical to
the "active" state flag so we use that now, via newly added
isOpen() static function. AbortChecker already maintains
half_closed status.
The accept-specific functionality is still implemented by
AcceptFD class. Removed fdc_t used to marshall accept-ralted
calls to AcceptFD anyway. fdc_table now stores AcceptFDs
directly. I did not rename the table to ease merging with other
code, but added a TODO for that.
Removed calls to comm_iocallbackpending(). They were added to
"Speed up processing of queued events significantly, to not
cause large delays when under low load" but I do not see how
having pending callbacks can be relevant because all pending
callbacks are (should be) executed before FDs are probed.
Removed unused nullCallback() methods.
Removed CommCallbackData and related code. It looks like it
remained after one of the big comm rewrites, just to handle
accept errors. We can now schedule an async call to notify of
those errors and do not seem to need CommCallbackData at all.
Removed commfd_completed_events: a list of completed (but not
yet fired) callbacks. We simply schedule the async call now
instead of maintaining our own list of callbacks to call. This
change allows us to eliminate the CommDispatcher class (which
was the motivation behind these changes), but I have not done
that yet.
For comm_io_callback_t, being active seems to be the same as
having a callback. Replaced active data member with a method
that checks for the callback presence.
Relaxed comm_read_cancel() preconditions so that the callers do
not have to check all the assertions inside before calling that
function.
rousskov [Wed, 13 Feb 2008 06:49:44 +0000 (06:49 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
Below you will find log messages from the async-call branch that are
relevant to the file(s) being committed.
Do not use CompletionDispatcher.
Async calls are not longer using time-based events with zero
delay. They have a dedicated AsyncCallQueue.
Moved duplicated code into dispatchCalls().
EventLoop will need further internal polishing to polish, simplify and
possibly speed up the code.
rousskov [Wed, 13 Feb 2008 06:44:35 +0000 (06:44 +0000)]
Merging async-call branch changes to HEAD:
BodyPipe now uses the new job calls interface:
- Instead of scheduling BodyPipe calls we are scheduling BodyProducer
and BodyCosnumer calls
- BodyProducer/BodyConsumer::noteXXXXXX(BodyPipe &bp) methods converted
to Body*::noteXXXXXX(BodyPipe::Pointer) to allow them used with new
async calls interface.
- BodyPipe::tell* methods and related AsyncCallWrappers removed.
- Implement the CallNoteBodyProducer and CallNoteBodyConsumer JobCall based
classes. These classes used too schedule calls for BodyConsumer and
BodyProducer respectivelly. They are like the normal JobCalls but
aditionally checks if the BodyConsumer and BodyProducer is still part
of the pipe.
- Implement the NullBodyConsumer class as a BodyConsumer child which just
reads the data from pipe and discard them. This class used to replace
old AutoConsume mechanism.
- Removing the ICAPXaction_{Enter,Exit} calls from ICAPModXact::note* methods
Their logic implemented inside the new async calls code.
Remove the old "late call avoidance" code from BodyPipe as no longer needed.
rousskov [Wed, 13 Feb 2008 06:40:02 +0000 (06:40 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
The code is still more complex than I want it to be. However, these are
low-level interfaces that most developers will not have to deal with and
the complexity will be significantly reduced if we get rid of old style
function-pointer based callbacks, which is probably desirable for several
reasons. Other simplifications will be found in time as well.
TODO: src/ICAP/AsyncJob needs to be moved to src/. It is not moved yet to
preserve commit logs.
rousskov [Wed, 13 Feb 2008 06:33:48 +0000 (06:33 +0000)]
Merging async-call branch changes to HEAD:
Synced after changing delayed read callback storage model from
(function pointer, opaque parameter) pair to AsyncCall.
Internally, comm module uses AsyncCalls to store callback
information now.
rousskov [Wed, 13 Feb 2008 06:29:25 +0000 (06:29 +0000)]
Merging async-call branch changes to HEAD:
ConnStateData used a RefCount pointer (the openReference) to keep itself alive
which is not compatible with the cbdata used by CommCalls. This change makes
the ConnStateData class non RefCountable and uses cbdata instead.
rousskov [Wed, 13 Feb 2008 06:26:36 +0000 (06:26 +0000)]
Merging async-call branch changes to HEAD:
Added toCbdata() method to deal with multiple inheritence in cbdata-protected
objects. The pointers to the object used by others may not point to the part
of the object with the cbdata magic.
rousskov [Wed, 13 Feb 2008 06:17:50 +0000 (06:17 +0000)]
Merging async-call branch changes to HEAD:
ConnStateData used a RefCount pointer (the openReference) to keep itself alive
which is not compatible with the cbdata used by CommCalls. This change makes
the ConnStateData class non RefCountable and uses cbdata instead.
rousskov [Wed, 13 Feb 2008 06:12:45 +0000 (06:12 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
Most of the changes in the ICAP/ICAP* files are related to removing handler
wrappers and using CommCalls. AsyncJob is now a [virtual] base class for
ICAPInitiator. All ICAP initiators and ICAP transactions are now "jobs".
rousskov [Wed, 13 Feb 2008 06:07:52 +0000 (06:07 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
Below you will find log messages from the async-call branch that are
relevant to the file(s) being committed.
Made comm_read and comm_write calls to use the new CommCalls.
Converted the ConnStateData related code in client_side.cc to
use CommCalls.
BodyPipe now uses the new job calls interface. Many changes in
BodyPipe's related code.
rousskov [Wed, 13 Feb 2008 06:02:13 +0000 (06:02 +0000)]
Merging async-call branch changes to HEAD:
Removed calls to comm_iocallbackpending(). They were added to
"Speed up processing of queued events significantly, to not
cause large delays when under low load" but I do not see how
having pending callbacks can be relevant because all pending
callbacks are (should be) executed before FDs are probed.
rousskov [Wed, 13 Feb 2008 05:58:29 +0000 (05:58 +0000)]
Merging async-call branch changes to HEAD:
Async-call work replaces event-based asynchronous calls with
stand-alone implementation. The common async call API allows Squid
core do call, debug, and troubleshoot all callback handlers in a
uniform way.
An async "job" API is introduced to manage independent logical threads
or work such as protocol transaction handlers on client, server, and
ICAP sides. These jobs should communicate with each other using async
calls to minimize dependencies and avoid reentrant callback loops.
These changes will eventually improve overall code quality, debugging
quality, and Squid robustness.
Below you will find log messages from the async-call branch that are
relevant to the file(s) being committed.
Added initial implelentation of AsyncCall-based wrappers for
comm callbacks. The comm layer no longer calls callbacks from
the select loop. Instead, the select loop schedules async
calls. Scheduled calls are then called from the main loop (like
all other async calls), after the select loop completes.
Removed accept loop. We cannot loop because async calls do not
get fired during the loop and, hence, cannot register new
callbacks for new ready FDs.
The loop is implicit now. When the next callback is registered,
we check whether the last accept(2) call was successful or
OPTIMISTIC_IO is defined and call acceptNext() again if yes.
AcceptNext() may schedule another async call (using the
being-submitted callback) if the socket was still ready. Since
callbacks are fired until there are no callabacks left, we
still have an accept loop.
Removed CommDispatcher as unused.
Removed unused IOFCB, IOWCB, and CWCB.
Removed class fdc_t. After CommCallbackData removal, fdc_t was
only used to check that a FD is "active" and to check that a FD
is half_closed. fd_table flags.open seems to be identical to
the "active" state flag so we use that now, via newly added
isOpen() static function. AbortChecker already maintains
half_closed status.
The accept-specific functionality is still implemented by
AcceptFD class. Removed fdc_t used to marshall accept-ralted
calls to AcceptFD anyway. fdc_table now stores AcceptFDs
directly. I did not rename the table to ease merging with other
code, but added a TODO for that.
Removed calls to comm_iocallbackpending(). They were added to
"Speed up processing of queued events significantly, to not
cause large delays when under low load" but I do not see how
having pending callbacks can be relevant because all pending
callbacks are (should be) executed before FDs are probed.
Removed unused nullCallback() methods.
Removed CommCallbackData and related code. It looks like it
remained after one of the big comm rewrites, just to handle
accept errors. We can now schedule an async call to notify of
those errors and do not seem to need CommCallbackData at all.
Removed commfd_completed_events: a list of completed (but not
yet fired) callbacks. We simply schedule the async call now
instead of maintaining our own list of callbacks to call. This
change allows us to eliminate the CommDispatcher class (which
was the motivation behind these changes), but I have not done
that yet.
For comm_io_callback_t, being active seems to be the same as
having a callback. Replaced active data member with a method
that checks for the callback presence.
Relaxed comm_read_cancel() preconditions so that the callers do
not have to check all the assertions inside before calling that
function.
The CommCall code is still more complex than I want it to be.
However, these are low-level interfaces that most developers
will not have to deal with and the complexity will be
significantly reduced if we get rid of old style
function-pointer based callbacks, which is probably desirable
for several reasons.
projects / thirdparty / squid.git / log
