Fixes for the "%err_code and %err_detail logformat codes" patch
The newly added file src/err_detail_type.h must listed in squid_SOURCES
variable in Makefile.am to be included in archives generated using "make dist"
For the same reason the files scripts/{calc-must-ids.pl,calc-must-ids.sh}
must listed in EXTRA_DIST variable.
Author: Alex Rousskov <rousskov@measurement-factory.com>, Christos Tsantilas <chtsanti@users.sourceforge.net>
Added %err_code and %err_detail logformat codes to record details about transaction failures
For example, when Squid responds with 500 Internal Server Error, it is often
useful to know what went wrong.
This patch :
- log a detail string instead of numbers for common errors
- log a label with the detail code number for generic errors (system errors,
or exceptions)
- adds more details about errors, especially those detected with exceptions:
We record a hash of the filename and the source code line number of the
first caught exception.
- adds two scripts which can help the developers to find the exact position
of the caught exception:
1) The calc-must-ids.pl take as argument one or more files and compute for
each Must expression in the given files its id;
2) The calc-must-ids.sh can be used to find the exact position of a Must
expression from its id. Example usage:
# ./scripts/calc-must-ids.sh 0xB79EF14
./src/adaptation/ecap/MessageRep.cc:356: 0xB79EF14 Must(false);
Amos Jeffries [Sun, 17 Oct 2010 03:26:02 +0000 (21:26 -0600)]
Bug 2785: DNS needs to set EDNS options advertising Squid capabilities
... allowing Squid to advertise a larger UDP reply size than 512 bytes.
Internally Squid has a buffer allocated on demand so there is no
practicable limit on individual packets. Network topology and external
software places stricter boundaries on what works and what does not.
Squid does not parse the additional section of replies so for now the
full auto-negotiation EDNS allows is not used. Instead a configuration
option is provided for admin to configure a desirable packet size in
bytes. EDNS defaults to "none" (disabled) until tested in a wider
environment.
Testing so far has brought to light problems with EDNS adverts on A and
IPv4-PTR queries. So support is limited to AAAA and IPv6-PTR queries only.
EDNS compliant resolvers have the option of caching the info between
requests for a short while so this will hopefully leak over to improve
IPv4 responses as well.
Amos Jeffries [Sat, 16 Oct 2010 23:19:46 +0000 (17:19 -0600)]
basic_msnt_auth helper and NTLM/SMBLIB/RFCNB library polish.
Samba smblib/rfcnb code:
* Import the latest copy which I could find a download link to.
This source is from 1997 so I suspect there is something even newer
we should be using. Time was tight is the only excuse for using
these sources. Our originals were from 1995 and 1996 depending on the
helper using it, with a mix of patches.
* These two directories are in ours sources as lib/smblib and lib/rfcnb.
Each has its own convenience library. Kept separate with original
filenames to simplify future upgrades or removal.
* Samba sources have been diffed and compared function by function
against the copies previously in our sources. Functionality extensions
we use have been grafted back on top of the new(er) Samba sources.
- this was mostly around passing extra Unicode, DC hints and pre-crypted
passwords to the login checks.
- some files from libntlmauth have yet to be compared in fine detail,
that will be completed today before merge.
- some basic API function and struct definitions had to be moved to the
API headers to prevent needing to include the *-priv.h private
definitions externally to the library.
* the Samba API headers have been wrapped with #ifndef safety wrappers
* compile errors and include changes required to compile have been
made (code stays C)
* duplicate code in helpers/basic_auth/MSNT/* and libntlmauth/* is
removed.
* abuse of the smblib-priv.h and rfcnb-priv.h headers and all local
re-definitions has been erased from our code. Replaced by includes
of the library API headers: rfcnb/rfcnb.h smblib/smblib.h
libntlmauth:
* smblib/rfcnb bits erased
* moved to lib/ntlmauth in its much reduced form
* built as a convenience library instead of full library
Amos Jeffries [Fri, 15 Oct 2010 10:24:57 +0000 (04:24 -0600)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Bug 3002 pt2: store initialization (-z) does not work with SMP configs.
In SMP mode, when -z command line option is specified, start kids and allow
them to create their cache_dirs instead of trying to create macro-dependent
cache_dirs in the Master process.
In SMP mode, quit if cache_dir option is found before the workers option.
Amos Jeffries [Tue, 12 Oct 2010 11:57:10 +0000 (05:57 -0600)]
Use DISTCHECK_CONFIGURE_FLAGS to make recursive distcheck work again.
DISTCHECK_CONFIGURE_FLAGS is a magic automake macro for the
distcheck target recursive tests between scripted runs.
we use it to perform the same duty between our nested scripts.
We used it earlier but lost it through a bit of forgetfulness. Documented
now to prevent that happening again.
Amos Jeffries [Sat, 9 Oct 2010 11:20:12 +0000 (00:20 +1300)]
Polish FTP login error handing
Reverts a regression added recently that blocked the challenge events.
Fixes another potential nul-pointer dereference bug.
* 421/426 server overload equate to HTTP overload. But do special such that
the credentials are asked of the browser on retries.
* 43x and 53x FTP status are all credentials failures of various types.
Other failures are not credential related.
This leaves the other non-credential errors as general failures.
Amos Jeffries [Thu, 7 Oct 2010 07:53:45 +0000 (20:53 +1300)]
Author: Alex Rousskov <rousskov@measurement-factory.com>
Author: Amos Jeffries <squid3@treenet.co.nz>
API for subscribing AsyncCall handlers to event producers
This API allows AsyncCall handlers to be subscribed for receiving multiple
event callbacks from producer/factory classes.
Intended use-cases include the main port listeners which are started once.
Run for a long time. And over their lifetime need to generate multiple
calls to any one of several handlers without having specific type details
hard-coded about the calls they are spawning.
ie they cannot use "new X(y,z)" because they are not aware of y and z.
Nor can one AsyncCall be scheduled and fired multiple times.
Other use-cases already sighted are UDP readers and event timers.
Amos Jeffries [Tue, 5 Oct 2010 11:43:27 +0000 (05:43 -0600)]
Unlink RefCount smart-pointer constness from its data.
Reason for having this in the first place is unknown. Several of the dev
agree there seems to be no reason to keep it and many to remove:
* The data RefCount points to is always dynamic
* The data pointed to is deleted with the last reference
* Its desirable to pass pointers around as const knowing that they will
not be changed to point at another object, but still manipulate the
data object itself.
Alex Rousskov [Mon, 4 Oct 2010 15:52:31 +0000 (09:52 -0600)]
HTTP Compliance: Send Age:0 header even if it may break IE5.
Removed a hack that prevented zero-valued Age header from being sent to HTTP
clients. The hack was added around January 2000 and is related to the
following patch:
http://devel.squid-cache.org/hno/patches/squid-2.2.STABLE5.age.patch
The above patch sites IE5 as the possible culprit that hangs when processing
zero Age responses.
If we get complaints about broken browsers not being able to handle
zero-valued Age headers, we can make Age:0 header inclusion conditional
on something like SUPPORT_IE5 #define or even on the User-Agent value.
Alex Rousskov [Mon, 4 Oct 2010 15:44:32 +0000 (09:44 -0600)]
HTTP Compliance: do not cache replies to requests with CC/no-store.
Per RFC 2616, do not store any part of response to requests with a no-store
Cache-Control directive.
We may still _serve_ no-store requests from the cache because RFC 2616 does not
seem to prohibit that. This may change if HTTPbis or developers decide to
prohibit no-store hits.
Co-Advisor test case:
test_case/rfc2616/ccReqDirMsg-no-store-basic
Alex Rousskov [Mon, 4 Oct 2010 15:40:04 +0000 (09:40 -0600)]
HTTP Compliance: reply with 504 (Gateway Timeout) if required validation fails.
RFC 2616 says that we MUST reply with 504 (Gateway Timeout) if validation
fails and cached reply has proxy-revalidate, must-revalidate or s-maxage
Cache-Control directive.
FwdState::makeConnectingError() method is added to set error status depending
on whether the request was a validation request.
Co-Advisor test cases:
test_case/rfc2616/noSrv-hit-must-reval-s-maxage-resp
test_case/rfc2616/noSrv-hit-must-reval-proxy-revalidate-resp
test_case/rfc2616/noSrv-hit-must-reval-must-revalidate-resp
Alex Rousskov [Mon, 4 Oct 2010 15:32:20 +0000 (09:32 -0600)]
HTTP Compliance: support requests with Cache-Control: min-fresh.
Added min-fresh directive support for Cache-Control header. The directive is
handled in refreshCheck() by incrementing age and check_time by min-fresh
value.
Co-Advisor test case:
test_case/rfc2616/ccReqDirMsg-min-fresh-obey
Alex Rousskov [Mon, 4 Oct 2010 15:16:18 +0000 (09:16 -0600)]
HTTP Compliance: add appropriate Warnings if serving a stale hit.
Per RFC 2616, we MUST add "110 Response is stale" Warning if serving a
stale reply for any reason, including configured overrides. We MUST add
"111 Revalidation failed" Warning if serving a stale reply because an
attempt to revalidate the response failed, due to an inability to reach
the server.
The patch adds a new stale_if_hit request flag, which is set in
refreshCheckHTTP() when entry freshness is calculated. refreshCheckHTTP()
is now called in offline mode, to set stale_if_hit properly. We check for
the offline mode before returning from refreshCheckHTTP() to preserve the
original logic.
refreshCheckHTTP() is no longer called for internal requests, to avoid
setting of stale_if_hit flag. It did not do anything important for
internal requests anyway.
Co-Advisor test cases:
test_case/rfc2616/noSrv-hit-stale-max-age-req
test_case/rfc2616/ccReqDirMsg-max-stale-warning
Alex Rousskov [Mon, 4 Oct 2010 14:52:38 +0000 (08:52 -0600)]
Use InstanceId for async job and calls identification.
Side-effect: removes inconsistent prefixes for job debugging: ecapxN, icapxN,
asyncN are now all jobN, simplifying searching and processing debugging logs.
Alex Rousskov [Mon, 4 Oct 2010 14:37:32 +0000 (08:37 -0600)]
Accept ICAP OPTIONS responses with unknown body types.
Warn about the unknown OPTIONS body type but ignore it instead of rejecting
the entire OPTIONS response. Note that ICAP does not standardize OPTIONS
body types, and Squid does not recognize any body type.
ICAP servers are supposed to negotiate the use of OPTIONS bodies but
negotiation mechanism is not standardized and some do not negotiate at all.
Alex Rousskov [Tue, 28 Sep 2010 15:20:36 +0000 (09:20 -0600)]
HTTP Compliance: Reply with an error if required validation fails.
RFC 2616 says that proxy MUST not use stale entries that have s-maxage,
proxy-revalidate, or must-revalidate cache-directive.
Add new fail_on_validation_err request flag to store result from
refreshCheck(). It is needed to avoid refreshLimits() recalculation in
clientReplyContext::handleIMSReply().
Split LOG_TCP_REFRESH_FAIL into LOG_TCP_REFRESH_FAIL_OLD (stale reply sent)
and LOG_TCP_REFRESH_FAIL_ERR (error forwarded). However, both are still logged
as TCP_REFRESH_FAIL for backward-compatibility with external scripts and such.
We may decide to start logging more detailed codes later.
Co-Advisor test cases:
test_case/rfc2616/noSrv-hit-must-reval-s-maxage-resp
test_case/rfc2616/noSrv-hit-must-reval-proxy-revalidate-resp
test_case/rfc2616/noSrv-hit-must-reval-must-revalidate-resp
Alex Rousskov [Mon, 27 Sep 2010 18:42:05 +0000 (12:42 -0600)]
Cleanup: Remove old_rep2 from clientReplyContext::handleIMSReply().
StoreEntry::getReply() returns pointer to HttpReply that belongs to MemObject.
It does not create a new object. Hence, outer old_rep, inner old old_rep, and
inner new old_rep2 in handleIMSReply() are equal.
will have an effect for kids 1 through 4 only and will place them on even
cores starting with core #1.
If there are conflicts for a given process, the latest option wins and a
warning is printed. If the number of specified processes do not match the
number of specified cores, Squid quits with an error. Multiple
cpu_affinity_map options are merged.
Squid builds on systems without Linux CPU affinity calls, but setting affinity
only works if there are sched_getaffinity(2) and sched_setaffinity(2)
available. If there is no OS support but cpu_affinity options are configured,
Squid quits with an error. If there is OS support but calls fail, Squid prints
an error but does not quit.
Alex Rousskov [Wed, 22 Sep 2010 23:09:58 +0000 (17:09 -0600)]
HTTP Compliance: Make date parser stricter to better handle malformed Expires.
Check that there is no zone or zone is "GMT" in parse_date_elements().
Make sure there is no junk at the end of date in parse_date().
This will affect Date, IMS, and other date-carrying header fields recognized
by Squid but should not cause any messages to be rejected. Squid would just
ignore the malformed headers as if they are not there.
Co-Advisor test case:
test_case/rfc2616/invalidExpiresMakesStale-rfc1123x
Henrik Nordstrom [Mon, 20 Sep 2010 19:27:24 +0000 (21:27 +0200)]
Bug 3056 - comm.cc "!fd_table[fd].closing()" assertion from helperServerFree when a helper crashes while processing requests
reshuffle helperServerFree so it first unregisters the failed helper
and starts new ones if needed before it calls the callbacks on any
pending requests. If not those ends up resheduling the request on
this same crashed and partially shut down helper.
Alex Rousskov [Mon, 20 Sep 2010 19:00:02 +0000 (13:00 -0600)]
Author: Stefan Fritsch <sf@sfritsch.de>
Bug #3058 fix: errorSend and ICY leak MemBuf object.
HttpReply::pack() returns a MemBuf pointer that must be deleted. Fixed leaking
errorSend() function in errorpage.cc and ICY parsing hack in
HttpStateData::processReplyHeader().
Alex Rousskov [Mon, 20 Sep 2010 17:47:46 +0000 (11:47 -0600)]
Bug #3048 fix: "commio_has_callback(fd, IOCB_READ, ccb)" assertion;
may also be applicable to a similar IOCB_WITE assertion.
When we start closing a descriptor, we call commio_finish_callback() to remove
I/O callbacks. If this is not done from commHandleRead or commHandleWrite,
then select(2) structures may still have our descriptor registration and will
call Comm back to read or write before the descriptor is closed for good. This
will trigger a commio_has_callback() assertion.
Bug 3053: cache version 1 LFS support detection broken
Alter the upgrade detection to only be used if needed, and to load the
correct on-disk sizes for the fields being converted.
Lock the sfileno type down to 32-bits.
This also adds two new upgrade parsers:
Fix 64-bit builds previously using sfileno as full 64-bit int
Migrates swap.state from 32-bit to 64-bit time_t (old squid 2.4/2.5)
64-bit->32-bit system time_t migration remains broken.
Per-file meta data is not altered by these changes, so problems may
remain there.
projects / thirdparty / squid.git / log
