via http://$visible_hostname/ requests intercepted by the internal
server feature. Also https:// if SSL/TLS is available on the receiving
port.
In order to safely identify the manager reports the path prefix
/squid-internal-mgr/ is added. The old cache_oject:// scheme format
paths follow that identifier prefix.
To retrieve pages the proxy visible_hostname, management port (first
forward-proxy port), and the path prefix must all be present in the URL.
The "manager" ACL is altered to url_regex in order to match the new
protocol+path URL syntax.
Unlike the cache_object:// scheme, http[s]:// do not accept password
as part of the URL. If one is needed it must be sent via the HTTP
Authorization: Basic authentication header.
NP: use of this per-action cachemgr_passwd is not secure and should be
avoided. Stronger security can be gained via http_access with regular
proxy_auth and other ACLs.
- Inside IdleConnList::removeAt method the last element of the
IdleConnList::theList_ array initialized with random memory
- Inside IdleConnList::removeAt method if the IdleConnList::parent_ is NULL
(ICAP connections pools) the size_ of the array is not decreased after
element removed
- Inside IdleConnList::closeN method, it removes always all elements from the
list except the first one
When FwdServer::_peer is set, HttpStateData constructor creates a new special
HttpRequest, overwriting the request pointer set in the parent (ServerStateData)
constructor to fwd->request.
This special HttpRequest sets the proper urlpath (which maybe different from
the original HttpRequest), the host (HttpRequest::SetHost/GetHost) to be the
peer hostname and inherits flags, protocol, method. Also sets the
HttpRequest::flags.proxying.
Probably this is originaly done to handle only the differences in urlpath and
the host. But this is has as result to have two HttpRequests object in
HttpStateData, but their difference is not clear.
This patch removes the HttpStateData::orig_request member and uses only the
HttpStateData::request member
Bugs fixed with this patch:
- Debugs() and error pages sometimes display the cache_peer hostname as the URL requested domain name when going to an origin. Regardless of what the virtual
host name actually is.
- The request_header_access configuration parameter does not work when
sending requests to parent proxies.
- Squid may cache replies to requests with no-store in headers when uses a
parent cache.
- parent caches which have been configured as "sibling" for specific domains
using the neighbor_type_domain parameter are not counted.
Alex Rousskov [Thu, 23 Jun 2011 21:03:57 +0000 (15:03 -0600)]
Removed pointless call to maybeReadVirginBody() for aborted entries.
No runtime changes expected.
Setting flags.do_next_read to zero and then calling maybeReadVirginBody() is
pointless because maybeReadVirginBody() does nothing but wasting cycles when
the flag is zero. Also, even if we do manage to read the virgin response body
somehow, we cannot write it to the aborted store entry anyway.
This patch
- Adds SQUID_X509_V_ERR_DOMAIN_MISMATCH to TheSslErrorArray and in
errors/templates/error-details.txt template file to enable bypass and
details reporting for that error.
- Fixes the ssl error details handling code, to report the first honored error
in a certificate, not the first certification validation error (which could
have been bypassed).
- Adjust ssl_verify_cb() to copy ctx->error to error_no in the beginning and
never use ctx->error after that.
Amos Jeffries [Fri, 17 Jun 2011 16:32:04 +0000 (04:32 +1200)]
Upgrade comm layer Connection handling
The premise underlying this large patch is that instead of copying and
re-copying and re-lookups for the FD related data we can take the
ConnectionDetail class which is generated to store a few bits of IP
information about newly accept()'d connections and make it persist across
the whole of Squid.
It has been renamed from ConnectionDetails to Comm::Connection and has
absorbed a few FD data fields from other classes long the code paths.
Its scope is to hold an FD (or potential FD) plus meta data.
Comm::Connection are valid before, during and after the period when their
particular FD is open. The meta data may be used beforehand to setup the
FD (in the case of peer selection or other TcpAcceptor usage), and it may
remain in use after FD closure until logging or all linked job and call
objects have detected the closure and terminated. A global function
Comm::IsConnOpen() may be used on the pointer objects to detect whether
they point at an active connection.
Most of the patch is simple parameter changes to functions and methods to
pass a "cont Comm::ConnectionPointer &" instead of an "int FD". Along with
class FD fields being converted to these object pointers.
In order to support this alteration there have been behavioral changes to:
The socket accept() Job
Comm::TcpAcceptor altered to spawn Comm::Connection objects and to
operate with one controlling their active/closed state.
FTP data channel handling Calls.
efficiency improvements making use of Comm::Connection as a feedback
channel between TcpAcceptor and FtpStateData to cancel the listening
Job. Most of the underlying logic change is already in trunk to use
the Subscription API. This just streamlines and fixes some race bugs.
Peer selection
updated to spawn a set of Comm::Connection objects. To do this it is
updated to determine *all* peers including DIRECT ones. Doing the DNS
lookup instead of leaving it to comm_connect() on the other side of
FwdState. It also absorbs the outgoing address selection from FwdState
and can now specify details of local+remote ends of an outgoing TCP link.
Forwarding
updated to handle the new outputs from peer selection and to open sequentially.
pconn handling
updated to use destination IP/port and hold a Comm::Connection instead
of domain name indexing an FD. This allows us to maintain idle pools
and re-use FD more efficiently with virtual-hosted servers. Along
with maintaining certainty that the pconn selected actually goes to
the exact destination IP:port needed by forwarding.
comm layer outgoing connections
now have a control job Comm::ConnOpener to do this. Due to the peer
selection and forwarding changes this is a much simpler operation.
HTTP / CONNECT tunnel / gopher / whois / FTP
updated to receive a server and client Comm::Connection object from
forwarding. To operate on those until they close or are finished with.
SNMP / ICP / HTCP / DNS port listeners
updated to work with Comm::Connection holding their listening socket
meta data. This is a side-effect of the ICP and Comm read/write/timeout
changes.
This project adds support for a translatable and customisable error detail file
(errors/templates/error_details.txt). The file is stored like we store error
page templates today. Inside the file, an HTTP-like format used that can be
later extended to other error details (and beyond):
name: value
details: "value"
descr: "value"
or
name: value
details: "multi
line
value"
descr: "value with a \"quoted string\" inside"
The code supports future translations, just like Squid already support error
page translations.
This is a Measurement Factory project
----
Some Technical details:
- The errorpage code which is related to loading and parsing error templates
moved to TemplateFile class. This class is used as base class for
ErrorPageFile class which used to load error page templates.
- The HttpHeader parser used to parse error details
- The error details for various languages cached to memory
- The ErrorDetailsList used to store a list of error details for a
language/locale
- The ErrorDetailsManager is a class used to load and manage multiple error
details list (ErrorDetailsList objects) for many languages. It also
implements a simple cache.
Amos Jeffries [Fri, 17 Jun 2011 02:31:45 +0000 (14:31 +1200)]
Fix squidclient -V option and allow non-HTTP protocols to be tested
The "-" case is for old style HTTP (called 0.9) where there is no version
string. The "-V 0.9" is for testing servers with broken version number
tag "HTTP/0.9". Do not mix these up!
This also adds the ability to send non-HTTP version tags for testing.
ie "-V ICAP/1.0" or "-V ICY/1.0"
Amos Jeffries [Fri, 17 Jun 2011 02:14:01 +0000 (14:14 +1200)]
Upgrade ICAP persistent connection handling
ICAP services use a "service" model of pconn different from the
"TCP destination" model which PconnPool objects are designed for.
This patch alters Adaptation::Icap::ServiceRep to use the simpler
IdleConnList object for pconn storage. IdleConnList stores a
"set of idle connections" more compatible with the ICAP model.
In order to implement ICAP max-connections feature the closeN()
operation is added to IdleConnList.
The result is removal of the complex hash and management operations on
push/pop of the idle conn set. The only expected behaviour change is
more frequent re-use of idle connections on services with multiple IP
addresses. Speed gains are minimal, but positive.
Fixed bypass of SSL certificate validation errors.
The bypass code was calling ACLChecklist::fastCheck() multiple times
if multiple certificate errors were found. That method should not be
called multiple times because it changes the internal ACLChecklist
state, producing wrong answers for repeated calls.
This patch fixes the ACLChecklist::fastCheck() method so it can be called
multiple times. Each fastCheck() call results in an independent access
list check.
author: Alex Rousskov <rousskov@measurement-factory.com>, Christos Tsantilas <christos@chtsanti.net>
Bug 3153 fix: Prevent ICAP RESPMOD transactions getting stuck with the adapted body.
Part 1.
Server is expected to receive adapted response headers and then consume the
adapted response body, if any. If the server receives the headers and then
aborts, it must notify the ICAP side that nobody will consume the body.
Otherwise, the ICAP transaction will fill the BodyPipe buffer and get stuck
waiting for the consumer to free some space.
Part 2:
This fix still leaves one potential race condition unhandled: The ICAP
Initiatee disappears right after sending the adapted headers to the Server
(because there is nothing else for that initiatee to do). After the
noteAdaptationAnswer() call is scheduled by ICAP and before it is received by
the Server job, there is no usable link between Server and ICAP. There is no
way for the Server to notify the ICAP transaction that the Server job is
aborting during that time (and there is no Server job at all after it aborts,
naturally).
The solutions is to develop a custom AsyncCall which will call the
expectNoConsumption() on the message pipe if the call cannot be dialed (i.e.,
the message cannot be delivered to Server).
Amos Jeffries [Mon, 13 Jun 2011 12:25:12 +0000 (06:25 -0600)]
Auth lookup state cbdata upgrade
The authenticators utilize a "statedata" structure to store and pass
the callback and Auth::UserRequest an auth lookup is about.
This patch converts the structure from a CBDATA_GLOBAL_TYPE struct to a
CBDATA_CLASS2 and adds a parameterized constructor for it.
The result is that all the code using it no longer has to explicitly
manage fields assignments and cbdata referencing. Simply new the object
when submitting to the helper system and delete once its handler has
been called.
Amos Jeffries [Sun, 5 Jun 2011 15:43:46 +0000 (03:43 +1200)]
Request-Line log format codes for split client and server view
Since URL-rewrite, adaptation and a number of other things can alter the
request as it goes through Squid it is useful to be able to log both the
original incoming and the final outgoing versions of these details.
This adds logformat codes to extend %rm %ru %rp and %rv with < and >
modifiers for in and outbound display.
The old tag versions are kept for now. There is still work to do cleaning
up the log data object which will clarify if the various data fields
which used to be passed to logging match either of the original or final
request data and can be obsoleted. It is already clear that the logged
path matches neither and contains some normalized version.
Amos Jeffries [Thu, 2 Jun 2011 10:39:53 +0000 (04:39 -0600)]
Translation: auto-generate the lists of available translation files
Use automake includes built during source maintenance and stored in VCS.
Avoiding earlier problems encountered with the approaches attempting to
bootstrap the translation from within Makefile.
This also generates relevant lists for the manual pages. Preparing to
translate those automatically as well.
Currently the httpHeaderParseQuotedString does not support multiline quoted
string fields. According the rfc2616 multiline quoted string fields should
supported:
quoted-string = ( <"> *(qdtext | quoted-pair ) <"> )
qdtext = <any TEXT except <">>
TEXT = <any OCTET except CTLs,
but including LWS>
LWS = [CRLF] 1*( SP | HT )
This patch replaces the "[\r]\n " or "[\r]\n\t" with a single space.
projects / thirdparty / squid.git / log
