From: Alex Rousskov <rousskov@measurement-factory.com>
Date: Mon, 11 Aug 2014 16:09:06 +0000 (-0600)
Subject: Initial native FTP Relay support.
X-Git-Tag: SQUID_3_5_0_1~117
X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=b3cb9958457df8c623387c426359e5bce58c84e4;p=thirdparty%2Fsquid.git

Initial native FTP Relay support.

* Added ftp_port directive telling Squid to relay native FTP commands.
* Active and passive FTP support on the user-facing side;
  require passive connections to come from the control connection src IP.
* IPv6 support (EPSV and, on the user-facing side, EPRT).
* Intelligent adaptation of relayed FTP FEAT responses.
* Relaying of multi-line FTP control responses using various formats.
* Support relaying of FTP MLSD and MLST commands (RFC 3659).
* Several Microsoft FTP server compatibility features.
* ICAP/eCAP support (at individual FTP command/response level).
* Optional "current FTP directory" tracking (cannot be 100% reliable due to
  symbolic links and such, but is helpful in some common use cases).
* FTP origin control connection is pinned to the FTP user connection.
* No caching support -- no reliable Request URIs for that (see above).
* Significant FTP code restructuring on the server-facing side.
* Initial steps towards HTTP code restructuring on the client-facing side.

See merged revisions commit log for details.
---

b3cb9958457df8c623387c426359e5bce58c84e4
diff --cc doc/release-notes/release-3.5.sgml
index 6958bb857e,7821c69901..ed3af7edd5
--- a/doc/release-notes/release-3.5.sgml
+++ b/doc/release-notes/release-3.5.sgml
@@@ -43,6 -43,6 +43,7 @@@ The 3.5 change history can be <url url=
  	<item>Support named services
  	<item>Upgraded squidclient tool
  	<item>Helper support for concurrency channels
++	<item>Native FTP Relay
  </itemize>
  
  Most user-facing changes are reflected in squid.conf (see below).
@@@ -163,6 -163,6 +164,48 @@@
     With these helpers concurrency may now be set to 0 or any higher number as desired.
  
  
++<sect1>Native FTP Relay
++<p>Details at <url url="http://wiki.squid-cache.org/Features/FtpRelay">.
++
++<p>Squid is now capable of accepting native FTP commands and relaying native
++   FTP messages between FTP clients and FTP servers. Native FTP commands
++   accepted at ftp_port are internally converted or wrapped into HTTP-like
++   messages. The same happens to Native FTP responses received from FTP origin
++   servers. Those HTTP-like messages are shoveled through regular access
++   control and adaptation layers between the FTP client and the FTP origin
++   server. This allows Squid to examine, adapt, block, and log FTP exchanges.
++   Squid reuses most HTTP mechanisms when shoveling wrapped FTP messages. For
++   example, http_access and adaptation_access directives are used.
++
++<p>FTP Relay is a new, experimental, complex feature that has seen limited
++   production exposure. Some Squid modules (e.g., caching) do not currently
++   work with native FTP proxying, and many features have not even been tested
++   for compatibility. Test well before deploying!
++
++<p>Native FTP proxying differs substantially from proxying HTTP requests with
++   <em>ftp://</em> URIs because Squid works as an FTP server and receives
++   actual FTP commands (rather than HTTP requests with FTP URLs).
++
++<p>FTP Relay highlights:</p>
++
++<itemize>
++    <item>Added ftp_port directive telling Squid to relay native FTP commands.
++    <item>Active and passive FTP support on the user-facing side; require
++	  passive connections to come from the control connection source IP
++	  address.
++    <item>IPv6 support (EPSV and, on the user-facing side, EPRT).
++    <item>Intelligent adaptation of relayed FTP FEAT responses.
++    <item>Relaying of multi-line FTP control responses using various formats.
++    <item>Support relaying of FTP MLSD and MLST commands (RFC 3659).
++    <item>Several Microsoft FTP server compatibility features.
++    <item>ICAP/eCAP support (at individual FTP command/response level).
++    <item>Optional "current FTP directory" tracking with the assistance of
++	  injected (by Squid) PWD commands (cannot be 100% reliable due to
++	  symbolic links and such, but is helpful in some common use cases).
++    <item>No caching support -- no reliable Request URIs for that (see above).
++</itemize>
++
++
  <sect>Changes to squid.conf since Squid-3.4
  <p>
  There have been changes to Squid's configuration file since Squid-3.4.
@@@ -228,6 -228,6 +271,22 @@@ This section gives a thorough account o
  	</verb>
  	<p>The default value for extras is: "%>a/%>A %un %>rm myip=%la myport=%lp"
  
++	<tag>ftp_port</tag>
++
++	<p>New configuration directive to accept and relay native FTP
++	   commands. Typically used for port 21 traffic.  By default, native
++	   FTP commands are not accepted.
++
++	<tag>ftp_client_idle_timeout</tag>
++
++	<p>This new configuration directive controls how long Squid should
++	   wait for an FTP request on a connection to an ftp_port.  Many FTP
++	   clients do not deal with idle connection closures well,
++	   necessitating a longer default timeout (30 minutes) than
++	   client_idle_pconn_timeout used for incoming HTTP requests (2
++	   minutes). The current default may be changed as we get more
++	   experience with FTP relaying.
++
  </descrip>
  
  <sect1>Changes to existing tags<label id="modifiedtags">
diff --cc src/client_side.cc
index 65344d0a72,325d0f7d46..d8966a220e
--- a/src/client_side.cc
+++ b/src/client_side.cc
@@@ -3660,13 -3635,22 +3635,18 @@@ httpsAccept(const CommAcceptCbParams &p
      ++incoming_sockets_accepted;
  
      // Socket is ready, setup the connection manager to start using it
-     ConnStateData *connState = new ConnStateData(xact);
+     ConnStateData *connState = Https::NewServer(xact);
+     AsyncJob::Start(connState); // usually async-calls postHttpsAccept()
+ }
  
-     if (s->flags.tunnelSslBumping) {
-         debugs(33, 5, "httpsAccept: accept transparent connection: " << params.conn);
+ void
+ ConnStateData::postHttpsAccept()
+ {
 -    // XXX: Remove these change-minimizing variables before commit
 -    ConnStateData *connState = this;
 -    const AnyP::PortCfgPointer s = port;
 -
 -    if (s->flags.tunnelSslBumping) {
++    if (port->flags.tunnelSslBumping) {
+         debugs(33, 5, "accept transparent connection: " << clientConnection);
  
          if (!Config.accessList.ssl_bump) {
--            httpsSslBumpAccessCheckDone(ACCESS_DENIED, connState);
++            httpsSslBumpAccessCheckDone(ACCESS_DENIED, this);
              return;
          }
  
@@@ -3674,19 -3658,19 +3654,19 @@@
          // using tproxy/intercept provided destination IP and port.
          HttpRequest *request = new HttpRequest();
          static char ip[MAX_IPSTRLEN];
-         assert(params.conn->flags & (COMM_TRANSPARENT | COMM_INTERCEPTION));
-         request->SetHost(params.conn->local.toStr(ip, sizeof(ip)));
-         request->port = params.conn->local.port();
-         request->myportname = s->name;
+         assert(clientConnection->flags & (COMM_TRANSPARENT | COMM_INTERCEPTION));
+         request->SetHost(clientConnection->local.toStr(ip, sizeof(ip)));
+         request->port = clientConnection->local.port();
 -        request->myportname = s->name;
++        request->myportname = port->name;
  
          ACLFilledChecklist *acl_checklist = new ACLFilledChecklist(Config.accessList.ssl_bump, request, NULL);
-         acl_checklist->src_addr = params.conn->remote;
-         acl_checklist->my_addr = s->s;
-         acl_checklist->nonBlockingCheck(httpsSslBumpAccessCheckDone, connState);
+         acl_checklist->src_addr = clientConnection->remote;
 -        acl_checklist->my_addr = s->s;
 -        acl_checklist->nonBlockingCheck(httpsSslBumpAccessCheckDone, connState);
++        acl_checklist->my_addr = port->s;
++        acl_checklist->nonBlockingCheck(httpsSslBumpAccessCheckDone, this);
          return;
      } else {
--        SSL_CTX *sslContext = s->staticSslContext.get();
--        httpsEstablish(connState, sslContext, Ssl::bumpNone);
++        SSL_CTX *sslContext = port->staticSslContext.get();
++        httpsEstablish(this, sslContext, Ssl::bumpNone);
      }
  }