<UL>
<LI><A NAME="toc2.1">2.1</A> <A HREF="#ss2.1">Helper protocol extensions</A>
<LI><A NAME="toc2.2">2.2</A> <A HREF="#ss2.2">SSL Server Certificate Validator</A>
-<LI><A NAME="toc2.3">2.3</A> <A HREF="#ss2.3">TPROXY Support for OpenBSD 5.1+ and FreeBSD 9+</A>
+<LI><A NAME="toc2.3">2.3</A> <A HREF="#ss2.3">Store-ID</A>
+<LI><A NAME="toc2.4">2.4</A> <A HREF="#ss2.4">TPROXY Support for OpenBSD 5.1+ and FreeBSD 9+</A>
+<LI><A NAME="toc2.5">2.5</A> <A HREF="#ss2.5">Transaction Annotations</A>
</UL>
<P>
<H2><A NAME="toc3">3.</A> <A HREF="#s3">Changes to squid.conf since Squid-3.3</A></H2>
<UL>
<LI>Helper protocol extensions</LI>
<LI>SSL Server Certificate Validator</LI>
+<LI>Store-ID</LI>
<LI>TPROXY Support for OpenBSD 5.1+ and FreeBSD 9+</LI>
+<LI>Transaction Annotations</LI>
</UL>
</P>
<P>Most user-facing changes are reflected in squid.conf (see below).</P>
<EM>ssl_crtd</EM> related options. </P>
-<H2><A NAME="ss2.3">2.3</A> <A HREF="#toc2.3">TPROXY Support for OpenBSD 5.1+ and FreeBSD 9+</A>
+<H2><A NAME="ss2.3">2.3</A> <A HREF="#toc2.3">Store-ID</A>
+</H2>
+
+<P>Details at
+<A HREF="http://wiki.squid-cache.org/Features/StoreID">http://wiki.squid-cache.org/Features/StoreID</A>.</P>
+
+<P>This feature is a redesigned equivalent to the Squid-2.7 feature known as StoreURL-rewrite.</P>
+
+<P><EM>Notice</EM> that this is not a direct portage of the Squid-2.7 feature so behaviour
+differences do exist. Although the new feature works in similar enough ways that the old
+helper scripts used for Squid-2.7 are expected to work in this and later versions of Squid.</P>
+
+<P>Squid traditionally uses the requested URL as an index key ID to locate objects in cache.
+It is not the only key possible and the Store-ID feature exposes an API for external
+helpers to provide Squid with an alternative key name for any URL.</P>
+
+<P>When any client request is received which requires a cache lookup the URL is passed to
+a helper specified with the <EM>store_id_program</EM> directive to check for an alternative
+Store ID. This allows the helper to identify URLs which refer to duplicate resources and
+de-duplicate the cache content. <EM>store_id_access</EM> is provided to allow ACL-based
+tuning of which traffic gets sent to the helper and reduce overheads.</P>
+
+<P>One subtle and noteworthy difference between Squid-2 and Squid-3 which is highlighted by
+this feature is that <EM>refresh_pattern</EM> applies its regex argument against the Store
+ID key and not the transaction URL. So using the Store-ID feature to alter the value
+affects which <EM>refresh_pattern</EM> directive will be matched.</P>
+
+<P>Store-ID helpers bundled with Squid can be built with the --enable-storeid-rewrite-helpers
+options which is added in this version. Currently there is a <EM>file</EM> helper
+provided.</P>
+
+
+<H2><A NAME="ss2.4">2.4</A> <A HREF="#toc2.4">TPROXY Support for OpenBSD 5.1+ and FreeBSD 9+</A>
</H2>
<P>Details at
These systems require <EM>--with-nat-devpf</EM> to enable /dev/pf support when using PF firewall.</P>
+<H2><A NAME="ss2.5">2.5</A> <A HREF="#toc2.5">Transaction Annotations</A>
+</H2>
+
+<P>Previously the only annotation methods available were ICAP/eCAP HTTP header insertions
+or external ACL <EM>tag=</EM> result code. Each of which had only limited possibilities
+for use and little or no correlation.</P>
+
+<P>It is now possible to add annotations to a client transaction from several sources:
+<UL>
+<LI> Directly from squid.conf using the <EM>note</EM> directive with
+ACL-based selection of which annotation is linked to any
+particular transaction.
+</LI>
+<LI> By configured helper processes returning a key=value pair.
+The key name becomes the annotation name.</LI>
+</UL>
+</P>
+
+<P>Annotations on the transaction can be passed to ICAP services or eCAP modules using the
+<EM>adaptation_meta</EM> directive to send them as headers.
+They can also be logged using the <EM>%note</EM> log format code in custom logs. With
+the new helper response syntax changes this means all helper response key=value details
+such as URL-rewrite or store-id changes, external ACL tag etc. are now able to be logged.</P>
+
+<P>Annotations which are already assigned to a transaction can be checked using an ACL test
+of the new <EM>note</EM> ACL type. This can match a particular note by name and value,
+of for any notes with a given name.</P>
+
+<P>NOTE: not all helper interfaces are yet enabled to convert key=value into annotations
+and the external ACL interface does not yet send annotations to the helper.</P>
+
+
<H2><A NAME="s3">3.</A> <A HREF="#toc3">Changes to squid.conf since Squid-3.3</A></H2>
<P>There have been changes to Squid's configuration file since Squid-3.3.</P>
<DT><B>sslcrtvalidator_program</B><DD>
<P>Specifies the location of a SSL server certificate validator helper.</P>
+<DT><B>store_id_access</B><DD>
+<P>Whether the URL for a given request is passed to the Store-ID helper process.
+Used to improve StoreID performance by quickly eliminating helper delays using ACL tests.</P>
+<P>Ported equivalent to <EM>storeurl_access</EM> from 2.7</P>
+
+<DT><B>store_id_bypass</B><DD>
+<P>Whether the StoreID helper may be bypassed when overloaded.</P>
+
+<DT><B>store_id_children</B><DD>
+<P>Controls the number of StoreID helper processes.</P>
+<P>Options <EM>startup=N</EM>, <EM>idle=N</EM>, <EM>concurrency=N</EM>
+<UL>
+<LI>startup=N allow finer tuning of how many helpers are started initially.</LI>
+<LI>idle=N allow fine tuning of how many helper to retain as buffer against sudden traffic loads.</LI>
+<LI>concurrency=N was previously called url_rewrite_concurrency as a distinct directive.</LI>
+</UL>
+</P>
+
+<DT><B>storeurl_rewrite_program</B><DD>
+<P>A helper program to provide cache storage internal key ID value for a request.</P>
+<P>Ported equivalent to <EM>storeurl_rewrite_program</EM> from 2.7</P>
+
</DL>
</P>
<P>
<DL>
+<DT><B>access_log</B><DD>
+<P>Configuration syntax extended to support name=value options.
+<EM>New Syntax:</EM> access_log module:place [option ...] [acl ...]</P>
+<P>New option <EM>logformat=</EM> to specify the logging format name.</P>
+<P>New option <EM>buffer-size=</EM> to specify how large the log buffer
+for this log is to be when <EM>buffered_logs</EM> is enabled.</P>
+<P>New option <EM>on-error=</EM> to specify what handling is to be done
+if the logging module encounters a non-recoverable error writing logs.
+With the value <EM>die</EM> (the default) Squid halts operation.
+With the value <EM>drop</EM> Squid drops log lines and continue running.</P>
+
<DT><B>acl</B><DD>
<P>New test type <EM>server_cert_fingerprint</EM> to match against
server SSL certificate fingerprint.</P>
+<P>New test type <EM>note</EM> to match against transaction annotations
+by name and value, or just by name.</P>
+<P>New test type <EM>any-of</EM> to match if any one of a set of named ACLs.</P>
+<P>New test type <EM>all-of</EM> to match against all of a set of named ACLs.</P>
<DT><B>auth_param</B><DD>
<P>New result code <EM>BH</EM> to signal helper internal errors
<A HREF="http://wiki.squid-cache.org/Features/AddonHelpers">http://wiki.squid-cache.org/Features/AddonHelpers</A>.</P>
<DT><B>http_port</B><DD>
+<P>Support IPv6 for <EM>intercept</EM> mode. Requires ip6tables support on Linux,
+PF support on OpenBSD and IPFW support on FreeBSD. Squid will no longer complain
+about misconfiguration if IPv6 support is missing, we now rely on the firewall
+tools reporting misconfiguration when the NAT rules are created.</P>
<P>Support <EM>tproxy</EM> mode traffic on BSD systems with BINDANY support
(OpenBSD 5+, FreeBSD 9+ so far).</P>
<P>Changed build options behind <EM>intercept</EM> traffic mode handling on BSD.
<P>New format code <EM>%note</EM> to log a transaction annotation linked to the
transaction by ICAP, eCAP, a helper, or the <EM>note</EM> squid.conf directive.</P>
+<DT><B>pipeline_prefetch</B><DD>
+<P>Updated to take a numeric count of prefetched pipeline requests instead of ON/OFF.</P>
+
+<DT><B>refresh_pattern</B><DD>
+<P><EM>NOTE:</EM> the regular expression pattern operates on the cache Store-ID value.
+Which by default is identical to the requested URL, but may differ for some
+objects if the Store-ID feature is in use.</P>
+
<DT><B>unlinkd_program</B><DD>
<P>New helper response format utilizing result codes <EM>OK</EM> and <EM>BH</EM>,
to signal helper lookup results. Also, key-value response values to return
<P>
<DL>
-<P><EM>There are no removed squid.conf tags in Squid-3.4.</EM></P>
-
<DT><B>storeurl_access</B><DD>
-<P>Not yet ported from 2.7</P>
+<P>Replaced by <EM>store_id_access</EM>.</P>
<DT><B>storeurl_rewrite_children</B><DD>
-<P>Not yet ported from 2.7</P>
+<P>Replaced by <EM>store_id_children</EM>.</P>
<DT><B>storeurl_rewrite_concurrency</B><DD>
-<P>Not yet ported from 2.7</P>
+<P>Replaced by <EM>store_id_children</EM> with <EM>concurrency=N</EM> option.</P>
<DT><B>storeurl_rewrite_program</B><DD>
-<P>Not yet ported from 2.7</P>
+<P>Replaced by <EM>store_id_program</EM>.</P>
</DL>
</P>
<P>
<DL>
+<DT><B>--enable-storeid-rewrite-helpers</B><DD>
+<P>New option to control which Store-ID helpers are built. As with other
+helper options use --disable-* to prevent any helpers building and
+omit to get all helper auto-detected.</P>
+<P>Currenly only a helper using <EM>file</EM> for backend is provided.</P>
+
<DT><B>--with-nat-pf</B><DD>
<P>New option to alter the behaviour of <EM>http_port ... intercept</EM> option
in squid.conf.</P>
<H2><A NAME="s5">5.</A> <A HREF="#toc5">Regressions since Squid-2.7</A></H2>
-<P>Some squid.conf and ./configure options which were available in Squid-2.7 are not yet available in Squid-3.3</P>
+<P>Some squid.conf options which were available in Squid-2.7 are not yet available in Squid-3.4</P>
<P>If you need something to do then porting one of these from Squid-2 to Squid-3 is most welcome.</P>
helpers to provide Squid with an alternative key name for any URL.
<p>When any client request is received which requires a cache lookup the URL is passed to
- a helper specified with the <em>store_id_rewrite_program</em> directive to check for
- an alternative Store ID. This allows the helper to identify URLs which refer to duplicate
- resources and de-duplicate the cache content. <em>store_id_access</em> is provided to
- allow ACL-based tuning of which traffic gets sent to the helper and reduce overheads.
+ a helper specified with the <em>store_id_program</em> directive to check for an alternative
+ Store ID. This allows the helper to identify URLs which refer to duplicate resources and
+ de-duplicate the cache content. <em>store_id_access</em> is provided to allow ACL-based
+ tuning of which traffic gets sent to the helper and reduce overheads.
<p>One subtle and noteworthy difference between Squid-2 and Squid-3 which is highlighted by
this feature is that <em>refresh_pattern</em> applies its regex argument against the Store
which do not yet support the getsockname() API.
These systems require <em>--with-nat-devpf</em> to enable /dev/pf support when using PF firewall.
+
<sect1>Transaction Annotations
<p>Previously the only annotation methods available were ICAP/eCAP HTTP header insertions
or external ACL <em>tag=</em> result code. Each of which had only limited possibilities
<tag>sslcrtvalidator_program</tag>
<p>Specifies the location of a SSL server certificate validator helper.
+ <tag>store_id_access</tag>
+ <p>Whether the URL for a given request is passed to the Store-ID helper process.
+ Used to improve StoreID performance by quickly eliminating helper delays using ACL tests.
+ <p>Ported equivalent to <em>storeurl_access</em> from 2.7
+
+ <tag>store_id_bypass</tag>
+ <p>Whether the StoreID helper may be bypassed when overloaded.
+
+ <tag>store_id_children</tag>
+ <p>Controls the number of StoreID helper processes.
+ <p>Options <em>startup=N</em>, <em>idle=N</em>, <em>concurrency=N</em>
+ <itemize>
+ <item>startup=N allow finer tuning of how many helpers are started initially.
+ <item>idle=N allow fine tuning of how many helper to retain as buffer against sudden traffic loads.
+ <item>concurrency=N was previously called url_rewrite_concurrency as a distinct directive.
+ </itemize>
+
+ <tag>storeurl_rewrite_program</tag>
+ <p>A helper program to provide cache storage internal key ID value for a request.
+ <p>Ported equivalent to <em>storeurl_rewrite_program</em> from 2.7
+
</descrip>
<sect1>Changes to existing tags<label id="modifiedtags">
<tag>pipeline_prefetch</tag>
<p>Updated to take a numeric count of prefetched pipeline requests instead of ON/OFF.
+ <tag>refresh_pattern</tag>
+ <p><em>NOTE:</em> the regular expression pattern operates on the cache Store-ID value.
+ Which by default is identical to the requested URL, but may differ for some
+ objects if the Store-ID feature is in use.
+
<tag>unlinkd_program</tag>
<p>New helper response format utilizing result codes <em>OK</em> and <em>BH</em>,
to signal helper lookup results. Also, key-value response values to return
<sect1>Removed tags<label id="removedtags">
<p>
<descrip>
- <p><em>There are no removed squid.conf tags in Squid-3.4.</em>
+ <tag>storeurl_access</tag>
+ <p>Replaced by <em>store_id_access</em>.
+
+ <tag>storeurl_rewrite_children</tag>
+ <p>Replaced by <em>store_id_children</em>.
+ <tag>storeurl_rewrite_concurrency</tag>
+ <p>Replaced by <em>store_id_children</em> with <em>concurrency=N</em> option.
+
+ <tag>storeurl_rewrite_program</tag>
+ <p>Replaced by <em>store_id_program</em>.
+
</descrip>
<sect>Regressions since Squid-2.7
-<p>Some squid.conf and ./configure options which were available in Squid-2.7 are not yet available in Squid-3.3
+<p>Some squid.conf options which were available in Squid-2.7 are not yet available in Squid-3.4
<p>If you need something to do then porting one of these from Squid-2 to Squid-3 is most welcome.
<tag>refresh_stale_hit</tag>
<p>Not yet ported from 2.7
- <tag>storeurl_access</tag>
- <p>Not yet ported from 2.7
-
- <tag>storeurl_rewrite_children</tag>
- <p>Not yet ported from 2.7
-
- <tag>storeurl_rewrite_concurrency</tag>
- <p>Not yet ported from 2.7
-
- <tag>storeurl_rewrite_program</tag>
- <p>Not yet ported from 2.7
-
<tag>update_headers</tag>
<p>Not yet ported from 2.7
projects / thirdparty / squid.git / commitdiff
