Non-HTTP bypass

Non-HTTP bypass

Intercepting proxies often receive non-HTTP connections. Squid cannot currently
deal with such connections well because it assumes that a given port receives
HTTP, FTP, or HTTPS traffic exclusively. This patch allows Squid to tunnel
unexpected connections instead of terminating them with an error.

In this project, we define an unexpected connection as a connection that
resulted in a Squid error during first request parsing. Which errors trigger
tunneling behavior is configurable by the admin using ACLs.

Here is a configuration sketch:

  # define what Squid errors indicate receiving non-HTTP traffic:
  acl foreignProtocol squid_error ERR_PROTOCOL_UNKNOWN ERR_TOO_BIG

  # define what Squid errors indicate receiving nothing:
  acl serverTalksFirstProtocol squid_error ERR_REQUEST_START_TIMEOUT

  # tunnel everything that does not look like HTTP:
  on_first_request_error tunnel foreignProtocol

  # tunnel if we think the client waits for the server to talk first:
  on_first_request_error tunnel serverTalksFirstProtocol

  # in all other error cases, just send an HTTP "error page" response:
  on_first_request_error respond all

  # Configure how long to wait for the first byte on the incoming
  # connection before raising an ERR_REQUEST_START_TIMEOUT error.
  request_start_timeout 5 seconds

The overall intent of this TCP tunnel is to get Squid out of the communication
loop to the extent possible. Once the decision to tunnel is made, no Squid
errors are going to be sent to the client and tunneled traffic is not going to
be sent to Squid adaptation services or logged to access.log (except for a
single summary line at the end of the transaction). Connection closure at the
server (or client) end of the tunnel is propagated to the other end by closing
the corresponding connection.

This patch also:

 Add "on_first_request_error", a new ACL-driven squid.conf directive that can
be used to establish a blind TCP tunnel which relays all bytes from/to the
intercepted connection to/from the intended destination address. See the sketch
above.
The on_first_request_error directive supports fast ACLs only.

 Add "squid_error", a new ACL type to match transactions that triggered a given
Squid error. Squid error IDs are used to configure one or more errors to match.
This is similar to the existing ssl_error ACL type but works with
Squid-generated errors rather than SSL library errors.

 Add "ERR_PROTOCOL_UNKNOWN", a Squid error triggered for http_port connections
that start with something that lacks even basic HTTP request structure. This
error is triggered by the HTTP request parser, and probably only when/after the
current parsing code detects an error. That is, we do not want to introduce
new error conditions, but we want to treat some of the currently triggered
parsing errors as a "wrong protocol" error, possibly after checking the parsing
state or the input buffer for some clues. There is no known way to reliably
distinguish malformed HTTP requests from non-HTTP traffic so the parser has
to use some imprecise heuristics to make a decision in some cases.
In the future, it would be possible to add code to reliably detect some popular
non-HTTP protocols, but adding such code is outside this project scope.

 Add "request_start_timeout", a new squid.conf directive to trigger a new
Squid ERR_REQUEST_START_TIMEOUT error if no bytes are received from the
client on a newly established http_port connection during the configured
time period. Applies to all http_ports (for now).

No support for tunneling through cache_peers is included. Configurations
that direct outgoing traffic through a peer may break Squid.

This is a Measurement Factory project