doc/release-notes/release-3.4.sgml

   1 <!doctype linuxdoc system>
   2 <article>
   3 <title>Squid 3.4.0.0 release notes</title>
   4 <author>Squid Developers</author>
   5
   6 <abstract>
   7 This document contains the release notes for version 3.4 of Squid.
   8 Squid is a WWW Cache application developed by the National Laboratory
   9 for Applied Network Research and members of the Web Caching community.
  10 </abstract>
  11
  12 <toc>
  13
  14 <sect>Notice
  15 <p>
  16 The Squid Team are pleased to announce the release of Squid-3.4.0.0 for testing.
  17
  18 This new release is available for download from <url url="http://www.squid-cache.org/Versions/v3/3.HEAD/"> or the
  19  <url url="http://www.squid-cache.org/Mirrors/http-mirrors.html" name="mirrors">.
  20
  21 While this release is not deemed ready for production use, we believe it is ready for wider testing by the community.
  22
  23 We welcome feedback and bug reports. If you find a bug, please see <url url="http://wiki.squid-cache.org/SquidFaq/BugReporting">
  24  for how to submit a report with a stack trace.
  25
  26 <sect1>Known issues
  27 <p>
  28 Although this release is deemed good enough for use in many setups, please note the existence of
  29 <url url="http://bugs.squid-cache.org/buglist.cgi?query_format=advanced&amp;product=Squid&amp;bug_status=UNCONFIRMED&amp;bug_status=NEW&amp;bug_status=ASSIGNED&amp;bug_status=REOPENED&amp;version=3.4" name="open bugs against Squid-3.4">.
  30
  31
  32 <sect1>Changes since earlier releases of Squid-3.4
  33 <p>
  34 The 3.4 change history can be <url url="http://www.squid-cache.org/Versions/v3/3.HEAD/changesets/" name="viewed here">.
  35
  36 <sect>Major new features since Squid-3.3
  37 <p>Squid 3.4 represents a new feature release above 3.3.
  38
  39 <p>The most important of these new features are:
  40 <itemize>
  41         <item>Helper protocol extensions
  42         <item>SSL Server Certificate Validator
  43         <item>Store-ID
  44         <item>TPROXY Support for OpenBSD 5.1+ and FreeBSD 9+
  45         <item>Transaction Annotations
  46         <item>Multicast DNS
  47 </itemize>
  48
  49 Most user-facing changes are reflected in squid.conf (see below).
  50
  51
  52 <sect1>Helper protocol extensions
  53 <p>Details at <url url="http://wiki.squid-cache.org/Features/AddonHelpers">.
  54
  55 <p>The Squid helper protocol used to communicate with authenticators,
  56    URL-rewriters, Redirectors, and External ACL helpers has been updated
  57    and extended.
  58
  59 <p><em>BH</em> status code is now accepted from all helpers to report
  60    internal error events separate from <em>ERR</em> rejection code.
  61    Permitting Squid to perform recovery operations specific to
  62    helper failure instead of a blanket client rejection.
  63
  64 <p>Arbitrary key-value pairs can be returned from any helper.
  65    Allowing future helpers to be forward- and backward- compatible
  66    with this and future version of Squid.
  67
  68
  69 <sect1>SSL Server Certificate Validator
  70 <p>Details at <url url="http://wiki.squid-cache.org/Features/SslServerCertValidator">.
  71
  72 <p>The helper consulted after the internal OpenSSL validation, regardless of the
  73    validation results. The helper will receive:
  74
  75 <itemize>
  76         <item>the origin server certificate (chain),
  77         <item>the intended domain name, and
  78         <item>a list of OpenSSL validation errors (if any).
  79 </itemize>
  80
  81 <p>If the helper decides to honor an OpenSSL error or report another validation
  82    error(s), the helper will return:
  83
  84 <itemize>
  85         <item>A list of certificates.
  86         <item>A list of items consists the the validation error name (see <em>%err_name</em>
  87         error page macro and <em>%err_details</em> code for <em>logformat</em>), error reason
  88         (<em>%ssl_lib_error macro</em>), and the offending certificate.
  89 </itemize>
  90
  91 <p>The returned information mimics what the internal OpenSSL-based validation code
  92    collects now. Returned errors, if any, are fed to <em>sslproxy_cert_error</em>,
  93    triggering the existing SSL error processing code.
  94
  95 <p>The helper invocation controlled by the <em>sslcrtvalidator_program</em> and
  96    <em>sslcrtvalidator_children</em> configurations options which are similar to the
  97    <em>ssl_crtd</em> related options.
  98
  99
 100 <sect1>Store-ID
 101 <p>Details at <url url="http://wiki.squid-cache.org/Features/StoreID">.
 102
 103 <p>This feature is a redesigned equivalent to the Squid-2.7 feature known as StoreURL-rewrite.
 104
 105 <p><em>Notice</em> that this is not a direct portage of the Squid-2.7 feature so behaviour
 106   differences do exist. Although the new feature works in similar enough ways that the old
 107   helper scripts used for Squid-2.7 are expected to work in this and later versions of Squid.
 108
 109 <p>Squid traditionally uses the requested URL as an index key ID to locate objects in cache.
 110    It is not the only key possible and the Store-ID feature exposes an API for external
 111    helpers to provide Squid with an alternative key name for any URL.
 112
 113 <p>When any client request is received which requires a cache lookup the URL is passed to
 114    a helper specified with the <em>store_id_program</em> directive to check for an alternative
 115    Store ID. This allows the helper to identify URLs which refer to duplicate resources and
 116    de-duplicate the cache content. <em>store_id_access</em> is provided to allow ACL-based
 117    tuning of which traffic gets sent to the helper and reduce overheads.
 118
 119 <p>One subtle and noteworthy difference between Squid-2 and Squid-3 which is highlighted by
 120    this feature is that <em>refresh_pattern</em> applies its regex argument against the Store
 121    ID key and not the transaction URL. So using the Store-ID feature to alter the value
 122    affects which <em>refresh_pattern</em> directive will be matched.
 123
 124 <p>Store-ID helpers bundled with Squid can be built with the --enable-storeid-rewrite-helpers
 125    options which is added in this version. Currently there is a <em>file</em> helper
 126    provided.
 127
 128
 129 <sect1>TPROXY Support for OpenBSD 5.1+ and FreeBSD 9+
 130 <p>Details at <url url="http://wiki.squid-cache.org/ConfigExamples/Intercept/OpenBsdPf">.
 131
 132 <p>The Packet Filter (PF) firewall in OpenBSD 4.4 and later offers traffic interception
 133    using several very simple methods. One of which is the <em>divert-to</em> rule type
 134    which acts as a simple routing diversion instead of performing NAT packet alterations.
 135
 136 <p>The IP Firewall (IPFW) on FreeBSD 9+ contains a port of the Linux Netfilter TPROXY feature.
 137
 138 <p>This version of Squid adds support for these features through the ./configure
 139    options --enable-pf-transparent and --enable-ipfw-transparent when Squid is built on
 140    systems with the required support. No special extras are required to enable
 141    <em>http_port ... tproxy</em> configuration to work.
 142
 143 <p>NOTE: To resolve NAT lookup issues on recent PF firewall versions the code behind
 144    <em>./configure --enable-pf-transparent</em> has been altered and is expected to
 145    break on the version of PF firewall shipped with BSD systems such as NetBSD and FreeBSD
 146    which do not yet support the getsockname() API.
 147    These systems require <em>--with-nat-devpf</em> to enable /dev/pf support when using PF firewall.
 148
 149
 150 <sect1>Transaction Annotations
 151 <p>Previously the only annotation methods available were ICAP/eCAP HTTP header insertions
 152    or external ACL <em>tag=</em> result code. Each of which had only limited possibilities
 153    for use and little or no correlation.
 154
 155 <p>It is now possible to add annotations to a client transaction from several sources:
 156 <itemize>
 157         <item>  Directly from squid.conf using the <em>note</em> directive with
 158                 ACL-based selection of which annotation is linked to any
 159                 particular transaction.
 160
 161         <item>  By configured helper processes returning a key=value pair.
 162                 The key name becomes the annotation name.
 163 </itemize>
 164
 165 <p>Annotations on the transaction can be passed to ICAP services or eCAP modules using the
 166    <em>adaptation_meta</em> directive to send them as headers.
 167    They can also be logged using the <em>%note</em> log format code in custom logs. With
 168    the new helper response syntax changes this means all helper response key=value details
 169    such as URL-rewrite or store-id changes, external ACL tag etc. are now able to be logged.
 170
 171 <p>Annotations which are already assigned to a transaction can be checked using an ACL test
 172    of the new <em>note</em> ACL type. This can match a particular note by name and value,
 173    of for any notes with a given name.
 174
 175 <p>NOTE: not all helper interfaces are yet enabled to convert key=value into annotations
 176          and the external ACL interface does not yet send annotations to the helper.
 177
 178
 179 <sect1>Multicast DNS
 180 <p>The internal DNS component fof Squid now supports multicast DNS (mDNS) resolution in
 181     accordance with RFC 6762.
 182
 183 <p>There is no additional or special configuration required. The multicast DNS group IP
 184    addresses for IPv4 and IPv6 resolving are added to the set of available DNS resolvers
 185    and used automatically for domain names ending in <em>.local</em> before attempting a
 186    secondary resolution on the configured resolvers. Domains without <em>.local</em> are
 187    resolved using only the configured DNS resolvers.
 188
 189 <p>Statistics for multicast DNS resolution can be found on the <em>idns</em> cache manager
 190    report.
 191
 192
 193 <sect>Changes to squid.conf since Squid-3.3
 194 <p>
 195 There have been changes to Squid's configuration file since Squid-3.3.
 196
 197 <p>Squid supports reading configuration option parameters from external
 198    files using the syntax <em>parameters("/path/filename")</em>. For example:
 199 <verb>
 200     acl whitelist dstdomain parameters("/etc/squid/whitelist.txt")
 201 </verb>
 202
 203 <p>There hasve also been changes to individual directives in the config file.
 204
 205 This section gives a thorough account of those changes in three categories:
 206
 207 <itemize>
 208         <item><ref id="newtags" name="New tags">
 209         <item><ref id="modifiedtags" name="Changes to existing tags">
 210         <item><ref id="removedtags" name="Removed tags">
 211 </itemize>
 212 <p>
 213
 214 <sect1>New tags<label id="newtags">
 215 <p>
 216 <descrip>
 217         <tag>configuration_includes_quoted_values</tag>
 218         <p>Whether Squid supports directive parameters with spaces, quotes, and other
 219            special characters. Surround such parameters with "double quotes".
 220
 221         <tag>note</tag>
 222         <p>Use ACLs to annotate a transaction with customized annotations
 223            which can be logged in access.log
 224
 225         <tag>spoof_client_ip</tag>
 226         <p>Access control to determine whether to disable the TPROXY spoofing on upstream traffic.
 227
 228         <tag>sslcrtvalidator_children</tag>
 229         <p>Specifies the settings for how many SSL server certificate
 230            validator helpers are run and when they are started.
 231
 232         <tag>sslcrtvalidator_program</tag>
 233         <p>Specifies the location of a SSL server certificate validator helper.
 234
 235         <tag>store_id_access</tag>
 236         <p>Whether the URL for a given request is passed to the Store-ID helper process.
 237            Used to improve StoreID performance by quickly eliminating helper delays using ACL tests.
 238         <p>Ported equivalent to <em>storeurl_access</em> from 2.7
 239
 240         <tag>store_id_bypass</tag>
 241         <p>Whether the StoreID helper may be bypassed when overloaded.
 242
 243         <tag>store_id_children</tag>
 244         <p>Controls the number of StoreID helper processes.
 245         <p>Options <em>startup=N</em>, <em>idle=N</em>, <em>concurrency=N</em>
 246         <itemize>
 247                 <item>startup=N allow finer tuning of how many helpers are started initially.
 248                 <item>idle=N allow fine tuning of how many helper to retain as buffer against sudden traffic loads.
 249                 <item>concurrency=N was previously called url_rewrite_concurrency as a distinct directive.
 250         </itemize>
 251
 252         <tag>storeurl_rewrite_program</tag>
 253         <p>A helper program to provide cache storage internal key ID value for a request.
 254         <p>Ported equivalent to <em>storeurl_rewrite_program</em> from 2.7
 255
 256 </descrip>
 257
 258 <sect1>Changes to existing tags<label id="modifiedtags">
 259 <p>
 260 <descrip>
 261         <tag>access_log</tag>
 262         <p>Configuration syntax extended to support name=value options.
 263           <em>New Syntax:</em> access_log module:place [option ...] [acl ...]
 264         <p>New option <em>logformat=</em> to specify the logging format name.
 265         <p>New option <em>buffer-size=</em> to specify how large the log buffer
 266            for this log is to be when <em>buffered_logs</em> is enabled.
 267         <p>New option <em>on-error=</em> to specify what handling is to be done
 268            if the logging module encounters a non-recoverable error writing logs.
 269            With the value <em>die</em> (the default) Squid halts operation.
 270            With the value <em>drop</em> Squid drops log lines and continue running.
 271
 272         <tag>acl</tag>
 273         <p>New test type <em>server_cert_fingerprint</em> to match against
 274            server SSL certificate fingerprint.
 275         <p>New test type <em>note</em> to match against transaction annotations
 276            by name and value, or just by name.
 277         <p>New test type <em>any-of</em> to match if any one of a set of named ACLs.
 278         <p>New test type <em>all-of</em> to match against all of a set of named ACLs.
 279
 280         <tag>auth_param</tag>
 281         <p>New result code <em>BH</em> to signal helper internal errors
 282            available in all authentication schemes.
 283         <p>New key <em>message=</em> for error message details in all authentication schemes.
 284         <p>New result code <em>OK</em> and key <em>ha1=</em> in Digest authentication.
 285         <p>New result codes <em>OK</em>, <em>ERR</em> replace result codes <em>AF</em>,
 286            and <em>NA</em> in NTLM and Negotiate authentication.
 287         <p>New key <em>token=</em> for NTLM and Negotiate authentication <em>OK</em> responses.
 288         <p>Details at <url url="http://wiki.squid-cache.org/Features/AddonHelpers">.
 289
 290         <tag>external_acl_type</tag>
 291         <p>Deprecated <em>protocol=3.0</em> option. No longer necessary.
 292         <p>New result code <em>BH</em> to signal helper internal errors
 293         <p>Details at <url url="http://wiki.squid-cache.org/Features/AddonHelpers">.
 294
 295         <tag>http_port</tag>
 296         <p>Support IPv6 for <em>intercept</em> mode. Requires ip6tables support on Linux,
 297            PF support on OpenBSD and IPFW support on FreeBSD. Squid will no longer complain
 298            about misconfiguration if IPv6 support is missing, we now rely on the firewall
 299            tools reporting misconfiguration when the NAT rules are created.
 300         <p>Support <em>tproxy</em> mode traffic on BSD systems with BINDANY support
 301            (OpenBSD 5+, FreeBSD 9+ so far).
 302         <p>Changed build options behind <em>intercept</em> traffic mode handling on BSD.
 303            see <em>--enable-pf-transparent</em> for more details.
 304
 305         <tag>logformat</tag>
 306         <p>New format code <em>%note</em> to log a transaction annotation linked to the
 307            transaction by ICAP, eCAP, a helper, or the <em>note</em> squid.conf directive.
 308         <p>New format code <em>%&gt;qos</em> to log client connection TOS/DSCP value set by Squid.
 309         <p>New format code <em>%&lt;qos</em> to log server connection TOS/DSCP value set by Squid.
 310         <p>New format code <em>%&gt;nfmark</em> to log client connection netfilter mark set by Squid.
 311         <p>New format code <em>%&lt;nfmark</em> to log server connection netfilter mark set by Squid.
 312
 313         <tag>pipeline_prefetch</tag>
 314         <p>Updated to take a numeric count of prefetched pipeline requests instead of ON/OFF.
 315
 316         <tag>refresh_pattern</tag>
 317         <p><em>NOTE:</em> the regular expression pattern operates on the cache Store-ID value.
 318            Which by default is identical to the requested URL, but may differ for some
 319            objects if the Store-ID feature is in use.
 320
 321         <tag>unlinkd_program</tag>
 322         <p>New helper response format utilizing result codes <em>OK</em> and <em>BH</em>,
 323            to signal helper lookup results. Also, key-value response values to return
 324            multiple values to Squid.
 325         <p>Details at <url url="http://wiki.squid-cache.org/Features/AddonHelpers">.
 326
 327         <tag>url_rewrite_program</tag>
 328         <p>New helper response format utilizing result codes <em>OK</em>, <em>ERR</em>,
 329            and <em>BH</em> to signal helper lookup results. Also, key-value response
 330            values to return multiple values to Squid.
 331         <p>Details at <url url="http://wiki.squid-cache.org/Features/AddonHelpers">.
 332
 333 </descrip>
 334
 335 <sect1>Removed tags<label id="removedtags">
 336 <p>
 337 <descrip>
 338         <tag>storeurl_access</tag>
 339         <p>Replaced by <em>store_id_access</em>.
 340
 341         <tag>storeurl_rewrite_children</tag>
 342         <p>Replaced by <em>store_id_children</em>.
 343
 344         <tag>storeurl_rewrite_concurrency</tag>
 345         <p>Replaced by <em>store_id_children</em> with <em>concurrency=N</em> option.
 346
 347         <tag>storeurl_rewrite_program</tag>
 348         <p>Replaced by <em>store_id_program</em>.
 349
 350 </descrip>
 351
 352
 353 <sect>Changes to ./configure options since Squid-3.3
 354 <p>
 355 There have been some changes to Squid's build configuration since Squid-3.3.
 356
 357 This section gives an account of those changes in three categories:
 358
 359 <itemize>
 360         <item><ref id="newoptions" name="New options">
 361         <item><ref id="modifiedoptions" name="Changes to existing options">
 362         <item><ref id="removedoptions" name="Removed options">
 363 </itemize>
 364
 365
 366 <sect1>New options<label id="newoptions">
 367 <p>
 368 <descrip>
 369         <tag>--enable-storeid-rewrite-helpers</tag>
 370         <p>New option to control which Store-ID helpers are built. As with other
 371            helper options use --disable-* to prevent any helpers building and
 372            omit to get all helper auto-detected.
 373         <p>Currenly only a helper using <em>file</em> for backend is provided.
 374
 375         <tag>--with-nat-devpf</tag>
 376         <p>New option to alter the behaviour of <em>http_port ... intercept</em> option
 377            in squid.conf.
 378         <p>When this option is used Squid performs the /dev/pf lookups required to
 379            support PF <em>rdr-to</em> rules. Otherwise Squid will perform perform the
 380            getsockname() API calls to support PF <em>divert-to</em> rules.
 381         <p>NOTE: systems such as NetBSD and FreeBSD which do not yet support
 382            the getsockname() API in recent PF versions require this option.
 383
 384 </descrip>
 385
 386 <sect1>Changes to existing options<label id="modifiedoptions">
 387 <p>
 388 <descrip>
 389         <tag>--enable-pf-transparent</tag>
 390         <p>NAT table support updated to use the getsockname() API provided by the
 391            latest PF versions <em>divert-to</em>. This allows <em>http_port</em>
 392            in squid.conf to support both <em>intercept</em> and <em>tproxy</em> traffic
 393            and to silence NAT lookup failure messages on recent BSD.
 394         <p>NOTE: systems such as NetBSD and FreeBSD which do not yet support
 395            the getsockname() API in recent PF versions require <em>--with-nat-devpf</em>
 396            to re-enable /dev/pf support when using PF firewall.
 397
 398 </descrip>
 399 </p>
 400
 401 <sect1>Removed options<label id="removedoptions">
 402 <p>
 403 <descrip>
 404         <p><em>There are no removed ./configure options in Squid-3.4.</em>
 405
 406 </descrip>
 407
 408
 409 <sect>Regressions since Squid-2.7
 410
 411 <p>Some squid.conf options which were available in Squid-2.7 are not yet available in Squid-3.4
 412
 413 <p>If you need something to do then porting one of these from Squid-2 to Squid-3 is most welcome.
 414
 415 <sect1>Missing squid.conf options available in Squid-2.7
 416 <p>
 417 <descrip>
 418         <tag>broken_vary_encoding</tag>
 419         <p>Not yet ported from 2.6
 420
 421         <tag>cache_dir</tag>
 422         <p><em>COSS</em> storage type is lacking stability fixes from 2.6
 423         <p>COSS <em>overwrite-percent=</em> option not yet ported from 2.6
 424         <p>COSS <em>max-stripe-waste=</em> option not yet ported from 2.6
 425         <p>COSS <em>membufs=</em> option not yet ported from 2.6
 426         <p>COSS <em>maxfullbufs=</em> option not yet ported from 2.6
 427
 428         <tag>cache_peer</tag>
 429         <p><em>idle=</em> not yet ported from 2.7
 430         <p><em>monitorinterval=</em> not yet ported from 2.6
 431         <p><em>monitorsize=</em> not yet ported from 2.6
 432         <p><em>monitortimeout=</em> not yet ported from 2.6
 433         <p><em>monitorurl=</em> not yet ported from 2.6
 434
 435         <tag>cache_vary</tag>
 436         <p>Not yet ported from 2.6
 437
 438         <tag>collapsed_forwarding</tag>
 439         <p>Not yet ported from 2.6
 440
 441         <tag>error_map</tag>
 442         <p>Not yet ported from 2.6
 443
 444         <tag>external_refresh_check</tag>
 445         <p>Not yet ported from 2.7
 446
 447         <tag>ignore_ims_on_miss</tag>
 448         <p>Not yet ported from 2.7
 449
 450         <tag>location_rewrite_access</tag>
 451         <p>Not yet ported from 2.6
 452
 453         <tag>location_rewrite_children</tag>
 454         <p>Not yet ported from 2.6
 455
 456         <tag>location_rewrite_concurrency</tag>
 457         <p>Not yet ported from 2.6
 458
 459         <tag>location_rewrite_program</tag>
 460         <p>Not yet ported from 2.6
 461
 462         <tag>refresh_pattern</tag>
 463         <p><em>stale-while-revalidate=</em> not yet ported from 2.7
 464         <p><em>ignore-stale-while-revalidate=</em> not yet ported from 2.7
 465         <p><em>negative-ttl=</em> not yet ported from 2.7
 466
 467         <tag>refresh_stale_hit</tag>
 468         <p>Not yet ported from 2.7
 469
 470         <tag>update_headers</tag>
 471         <p>Not yet ported from 2.7
 472
 473 </descrip>
 474
 475 </article>