Squid 3.1.0.10 release notes

-Squid 3.1.0.10 release notes +Squid 3.1.0.11 release notes Squid Developers @@ -13,7 +13,7 @@ for Applied Network Research and members of the Web Caching community. Notice

-The Squid Team are pleased to announce the release of Squid-3.1.0.10 for testing. +The Squid Team are pleased to announce the release of Squid-3.1.0.11 for testing. This new release is available for download from or the . @@ -45,6 +45,7 @@ The most important of these new features are: Quality of Service (QoS) Flow support SSL Bump (for HTTPS Filtering and Adaptation) eCAP Adaptation Module support + ICAP Bypass and Retry enhancements Most user-facing changes are reflected in squid.conf (see below). @@ -251,6 +252,65 @@ While decrypted, the traffic can be inspected using ICAP.

Details in +ICAP Bypass and Retry enhancements + +

Details in + +

ICAP is now extended with full bypass and dynamic chain routing to handle multiple + adaptation services. + +ICAP Adaptation Service Sets and Chains + +

An adaptation service set contains similar, interchangeable services. No more + than one service is successfully applied. If one service is down or fails, + Squid can use another service. Think "hot standby" or "spare" ICAP servers. + +

Sets may seem similar to the existing "service bypass" feature, but they allow + the failed adaptation to be retried and succeed if a replacement service is + available. The services in a set may be all optional or all essential, + depending on whether ignoring the entire set is acceptable. The mixture of + optional and essential services in a set is supported, but yields results that + may be difficult for a human to anticipate or interpret. Squid warns when it + detects such a mixture. + +

When performing adaptations with a set, failures at a service (optional or + essential, does not matter) are retried with a different service if possible. + If there are no more replacement services left to try, the failure is treated + depending on whether the last service tried was optional or essential: Squid + either tries to ignore the failure and proceed or terminates the master + transaction. + +

An adaptation chain is a list of different services applied one after another, + forming an adaptation pipeline. Services in a chain may be optional or + essential. When performing adaptations, failures at an optional service are + ignored as if the service did not exist in the chain. + +

Request satisfaction terminates the adaptation chain. + +

When forming a set or chain for a given transaction, optional down services are ignored as if they did not exist. + +

ICAP and eCAP services can be mixed and matched in an adaptation set or chain. + +Dynamically form adaptation chains based on the ICAP X-Next-Services header. + +

If an ICAP service with the routing=1 option in squid.conf returns an ICAP + X-Next-Services response header during a successful REQMOD or RESPMOD + transaction, Squid abandons the original adaptation plan and forms a new + adaptation chain consisting of services identified in the X-Next-Services + header value (using a comma-separated list of adaptation service names from + squid.conf). The dynamically created chain is destroyed once the new plan is + completed or replaced. + +

This feature is useful when a custom adaptation service knows which other + services are applicable to the message being adapted. + +

Limit adaptation iterations to adaptation_service_iteration_limit to protect + Squid from infinite adaptation loops caused by ICAP services constantly + including themselves in the dynamic adaptation chain they request. When the + limit is exceeded, the master transaction fails. The default limit of 16 + should be large enough to not require an explicit configuration in most + environments yet may be small enough to limit side-effects of loops. + Windows support

This Squid version can run on Windows as a system service using the Cygwin emulation environment, @@ -481,26 +541,123 @@ This section gives a thorough account of those changes in three categories: It is currently not possible to apply more than one adaptation service at the same vectoring point to the same HTTP transaction. + - See also: icap_service and ecap_service + adaptation_masterx_shared_names + + For each master transaction (i.e., the HTTP request and response + sequence, including all related ICAP and eCAP exchanges), Squid + maintains a table of metadata. The table entries are (name, value) + pairs shared among eCAP and ICAP exchanges. The table is destroyed + with the master transaction. + + This option specifies the table entry names that Squid must accept + from and forward to the adaptation transactions. + + An ICAP REQMOD or RESPMOD transaction may set an entry in the + shared table by returning an ICAP header field with a name + specified in adaptation_masterx_shared_names. Squid will store + and forward that ICAP header field to subsequent ICAP + transactions within the same master transaction scope. + + Only one shared entry name is supported at this time. - adaptation_service_set + adaptation_service_chain - Defines a named adaptation service set. The set is populated in - the order of adaptation_service_set directives in this file. - When adaptation ACLs are processed, the first and only the first - applicable adaptation service from the set will be used. Thus, - the set should group similar, redundant services, rather than a - chain of complementary services. + Configures a list of complementary services that will be applied + one-by-one, forming an adaptation chain or pipeline. This is useful + when Squid must perform different adaptations on the same message. - If you have a single adaptation service, you do not need to - define a set containing it because adaptation_access accepts - service names. + adaptation_service_chain chain_name service_name1 svc_name2 ... - Example: - adaptation_service_set svcBlocker urlFilterPrimary urlFilterBackup - adaptation service_set svcLogger loggerLocal loggerRemote + The named services are used in the chain declaration order. The first + applicable adaptation service from the chain is used first. The next + applicable service is applied to the successful adaptation results of + the previous service in the chain. + + When adaptation starts, broken services are ignored as if they were + not a part of the chain. A broken service is a down optional service. + + Request satisfaction terminates the adaptation chain because Squid + does not currently allow declaration of RESPMOD services at the + "reqmod_precache" vectoring point (see icap_service or ecap_service). + + The services in a chain must be attached to the same vectoring point + (e.g., pre-cache) and use the same adaptation method (e.g., REQMOD). + + A chain may contain a mix of optional and essential services. If an + essential adaptation fails (or the failure cannot be bypassed for + other reasons), the master transaction fails. Otherwise, the failure + is bypassed as if the failed adaptation service was not in the chain. + + + adaptation_service_iteration_limit + + Limits the number of iterations allowed when applying adaptation + services to a message. If your longest adaptation set or chain + may have more than 16 services, increase the limit beyond its + default value of 16. If detecting infinite iteration loops sooner + is critical, make the iteration limit match the actual number + of services in your longest adaptation set or chain. + + Infinite adaptation loops are most likely with routing services. + + + adaptation_service_set + + Configures an ordered set of similar, redundant services. This is + useful when hot standby or backup adaptation servers are available. + + adaptation_service_set set_name service_name1 service_name2 ... + + The named services are used in the set declaration order. The first + applicable adaptation service from the set is used first. The next + applicable service is tried if and only if the transaction with the + previous service fails and the message waiting to be adapted is still + intact. + + When adaptation starts, broken services are ignored as if they were + not a part of the set. A broken service is a down optional service. + + The services in a set must be attached to the same vectoring point + (e.g., pre-cache) and use the same adaptation method (e.g., REQMOD). + + If all services in a set are optional then adaptation failures are + bypassable. If all services in the set are essential, then a + transaction failure with one service may still be retried using + another service from the set, but when all services fail, the master + transaction fails as well. + + A set may contain a mix of optional and essential services, but that + is likely to lead to surprising results because broken services become + ignored (see above), making previously bypassable failures fatal. + Technically, it is the bypassability of the last failed service that + matters. + + + chunked_request_body_max_size +

New option to enable handing of broken HTTP/1.1 clients sending chunk requests. + + A broken or confused HTTP/1.1 client may send a chunked HTTP + request to Squid. Squid does not have full support for that + feature yet. To cope with such requests, Squid buffers the + entire request and then dechunks request body to create a + plain HTTP/1.0 request with a known content length. The plain + request is then used by the rest of Squid code as usual. + + The option value specifies the maximum size of the buffer used + to hold the request before the conversion. If the chunked + request size exceeds the specified limit, the conversion + fails, and the client receives an "unsupported request" error, + as if dechunking was disabled. + + Dechunking is enabled by default. To disable conversion of + chunked requests, set the maximum to zero. + + Request dechunking feature and this option in particular are a + temporary hack. When chunking requests and responses are fully + supported, there will be no need to buffer a chunked request. delay_pool_uses_indirect_client @@ -668,6 +825,105 @@ This section gives a thorough account of those changes in three categories:

Controls how many different forward paths Squid will try before giving up. Default: 10 + icap_log +

New option to write ICAP log files record ICAP transaction summaries, one line per + transaction. Similar to access.log. + + The icap_log option format is: + icap_log [ [acl acl ...]] + icap_log none [acl acl ...]] + + Please see access_log option documentation for details. The two + kinds of logs share the overall configuration approach and many + features. + + ICAP processing of a single HTTP message or transaction may + require multiple ICAP transactions. In such cases, multiple + ICAP transaction log lines will correspond to a single access + log line. + + ICAP log uses logformat codes that make sense for an ICAP + transaction. Header-related codes are applied to the HTTP header + embedded in an ICAP server response, with the following caveats: + For REQMOD, there is no HTTP response header unless the ICAP + server performed request satisfaction. For RESPMOD, the HTTP + request header is the header sent to the ICAP server. For + OPTIONS, there are no HTTP headers. + + The following format codes are also available for ICAP logs: + + icap::st Bytes sent to the ICAP server (TCP payload + only; i.e., what Squid writes to the socket). + + icap::h ICAP request header(s). Similar to >h. + + icap::a %icap::to/%03icap::Hs %icap:: + + icap_retry +

New option to determine which retriable ICAP transactions are + retried. + + Transactions that received a complete ICAP response + and did not have to consume or produce HTTP bodies to receive + that response are usually retriable. + + icap_retry allow|deny [!]aclname ... + + Squid automatically retries some ICAP I/O timeouts and errors + due to persistent connection race conditions. + + + icap_retry_limit + + Limits the number of retries allowed. When set to zero (default), + no retries are allowed. + + Communication errors due to persistent connection race + conditions are unavoidable, automatically retried, and do not + count against this limit. + + include

New option to import entire secondary configuration files into squid.conf. @@ -690,6 +946,12 @@ This section gives a thorough account of those changes in three categories: loadable_modules @DEFAULT_PREFIX@/lib/MinimalAdapter.so + log_icap aclname [aclname ...] + + This options allows you to control which requests get logged + to icap.log. See the icap_log directive for ICAP log details. + + log_uses_indirect_client

Whether to use any result found by follow_x_forwarded_for in access.log. Default: ON @@ -751,8 +1013,6 @@ NOCOMMENT_START terminate the transaction. Bypassing validation errors is dangerous because an error usually implies that the server cannot be trusted and the connection may be insecure. - - See also: sslproxy_flags and DONT_VERIFY_PEER. qos_flows local-hit= sibling-hit= parent-hit= @@ -981,9 +1241,98 @@ NOCOMMENT_START https_port intercept sslbump connection-auth[=on|off]

New port options. see http_port. + icap_service bypass=on|off|1|0 routing=on|off|1|0 +

New options 'bypass=' and 'routing='. + + bypass=on|off|1|0 + If set to 'on' or '1', the ICAP service is treated as + optional. If the service cannot be reached or malfunctions, + Squid will try to ignore any errors and process the message as + if the service was not enabled. No all ICAP errors can be + bypassed. If set to 0, the ICAP service is treated as + essential and all ICAP errors will result in an error page + returned to the HTTP client. + + Bypass is off by default: services are treated as essential. + + routing=on|off|1|0 + If set to 'on' or '1', the ICAP service is allowed to + dynamically change the current message adaptation plan by + returning a chain of services to be used next. The services + are specified using the X-Next-Services ICAP response header + value, formatted as a comma-separated list of service names. + Each named service should be configured in squid.conf and + should have the same method and vectoring point as the current + ICAP transaction. Services violating these rules are ignored. + An empty X-Next-Services value results in an empty plan which + ends the current adaptation. + + Routing is not allowed by default: the ICAP X-Next-Services + response header is ignored. + + logfile_rotate

No longer controls cache.log rotation. Use debug_options rotate=N instead. + logformat +

New log format tag sets %icap::* %adapt::* for adaptation information. + %Hs tag deprecated and replaced by request/reply specific >Hs and <Hs + HTTP request/reply format tags may now be optionally prefixed with http::. + Old forms will be deprecated in some as yet undecided future release. + + dt Total time spent making DNS lookups (milliseconds) + + [http::]>Hs HTTP status code sent to the client + [http::]sh Received HTTP request headers size + [http::] + maximum_object_size_in_memory

Default size limit increased to 512KB.

Squid 3.1.0.10 release notes

Squid 3.1.0.11 release notes

Squid Developers

3. Windows support

1. Notice

2.9 ICAP Bypass and Retry enhancements +

ICAP Adaptation Service Sets and Chains

Dynamically form adaptation chains based on the ICAP X-Next-Services header.

3. Windows support