Merge branch 'maint-0.2.9' into maint-0.3.1

diff --cc doc/tor.1.txt
index c4c569836ba6148c210c35a6d2900474400327b6,a7ee7d11cad68939d162b60eac6a818ffcef0a99..a6b4f2fc4b793adc039cf413a7853648db79e961
--- 1/doc/tor.1.txt
--- 2/doc/tor.1.txt
+++ b/doc/tor.1.txt
@@@ -2405,9 -2438,104 +2405,104 @@@ The following options are used to confi
      non-anonymous HiddenServiceSingleHopMode. Enables direct connections in the
      server-side hidden service protocol.  If you are using this option,
      you need to disable all client-side services on your Tor instance,
 -    including setting SOCKSPort to "0".
 -    (Default: 0)
 +    including setting SOCKSPort to "0". Can not be changed while tor is
 +    running. (Default: 0)
  
+ DENIAL OF SERVICE MITIGATION OPTIONS
+ ------------------------------------
+ 
+ The following options are useful only for a public relay. They control the
+ Denial of Service mitigation subsystem.
+ 
+ [[DoSCircuitCreationEnabled]] **DoSCircuitCreationEnabled** **0**|**1**|**auto**::
+ 
+     Enable circuit creation DoS mitigation. If enabled, tor will cache client
+     IPs along with statistics in order to detect circuit DoS attacks. If an
+     address is positively identified, tor will activate defenses against the
+     address. See the DoSCircuitCreationDefenseType option for more details.
+     This is a client to relay detection only. "auto" means use the consensus
+     parameter. If not defined in the consensus, the value is 0.
+     (Default: auto)
+ 
+ [[DoSCircuitCreationMinConnections]] **DoSCircuitCreationMinConnections** __NUM__::
+ 
+     Minimum threshold of concurrent connections before a client address can be
+     flagged as executing a circuit creation DoS. In other words, once a client
+     address reaches the circuit rate and has a minimum of NUM concurrent
+     connections, a detection is positive. "0" means use the consensus
+     parameter. If not defined in the consensus, the value is 3.
+     (Default: 0)
+ 
+ [[DoSCircuitCreationRate]] **DoSCircuitCreationRate** __NUM__::
+ 
+     The allowed circuit creation rate per second applied per client IP
+     address. If this option is 0, it obeys a consensus parameter. If not
+     defined in the consensus, the value is 3.
+     (Default: 0)
+ 
+ [[DoSCircuitCreationBurst]] **DoSCircuitCreationBurst** __NUM__::
+ 
+     The allowed circuit creation burst per client IP address. If the circuit
+     rate and the burst are reached, a client is marked as executing a circuit
+     creation DoS. "0" means use the consensus parameter. If not defined in the
+     consensus, the value is 90.
+     (Default: 0)
+ 
+ [[DoSCircuitCreationDefenseType]] **DoSCircuitCreationDefenseType** __NUM__::
+ 
+     This is the type of defense applied to a detected client address. The
+     possible values are:
+ 
+       1: No defense.
+       2: Refuse circuit creation for the DoSCircuitCreationDefenseTimePeriod period of time.
+ +
+     "0" means use the consensus parameter. If not defined in the consensus,
+     the value is 2.
+     (Default: 0)
+ 
+ [[DoSCircuitCreationDefenseTimePeriod]] **DoSCircuitCreationDefenseTimePeriod** __N__ **seconds**|**minutes**|**hours**::
+ 
+     The base time period in seconds that the DoS defense is activated for. The
+     actual value is selected randomly for each activation from N+1 to 3/2 * N.
+     "0" means use the consensus parameter. If not defined in the consensus,
+     the value is 3600 seconds (1 hour).  (Default: 0)
+ 
+ [[DoSConnectionEnabled]] **DoSConnectionEnabled** **0**|**1**|**auto**::
+ 
+     Enable the connection DoS mitigation. For client address only, this allows
+     tor to mitigate against large number of concurrent connections made by a
+     single IP address. "auto" means use the consensus parameter. If not
+     defined in the consensus, the value is 0.
+     (Default: auto)
+ 
+ [[DoSConnectionMaxConcurrentCount]] **DoSConnectionMaxConcurrentCount** __NUM__::
+ 
+     The maximum threshold of concurrent connection from a client IP address.
+     Above this limit, a defense selected by DoSConnectionDefenseType is
+     applied. "0" means use the consensus parameter. If not defined in the
+     consensus, the value is 100.
+     (Default: 0)
+ 
+ [[DoSConnectionDefenseType]] **DoSConnectionDefenseType** __NUM__::
+ 
+     This is the type of defense applied to a detected client address for the
+     connection mitigation. The possible values are:
+ 
+       1: No defense.
+       2: Immediately close new connections.
+ +
+     "0" means use the consensus parameter. If not defined in the consensus,
+     the value is 2.
+     (Default: 0)
+ 
+ [[DoSRefuseSingleHopClientRendezvous]] **DoSRefuseSingleHopClientRendezvous** **0**|**1**|**auto**::
+ 
+     Refuse establishment of rendezvous points for single hop clients. In other
+     words, if a client directly connects to the relay and sends an
+     ESTABLISH_RENDEZVOUS cell, it is silently dropped. "auto" means use the
+     consensus parameter. If not defined in the consensus, the value is 0.
+     (Default: auto)
+ 
  TESTING NETWORK OPTIONS
  -----------------------
  

diff --cc src/common/address.c
Simple merge

diff --cc src/common/address.h
Simple merge

diff --cc src/common/include.am
Simple merge

diff --cc src/common/log.c
index 6a5819064a203496897d693a44c1006c6ec11c42,4db1c9f0d011b222caa33ddf2eb15b2b6b4854c9..87c260799d9fb0d32c4566482803738ccbcee468
--- 1/src/common/log.c
--- 2/src/common/log.c
+++ b/src/common/log.c
@@@ -1177,7 -1177,7 +1177,7 @@@ static const char *domain_list[] = 
    "GENERAL", "CRYPTO", "NET", "CONFIG", "FS", "PROTOCOL", "MM",
    "HTTP", "APP", "CONTROL", "CIRC", "REND", "BUG", "DIR", "DIRSERV",
    "OR", "EDGE", "ACCT", "HIST", "HANDSHAKE", "HEARTBEAT", "CHANNEL",
-   "SCHED", "GUARD", "CONSDIFF", NULL
 -  "SCHED", "DOS", NULL
++  "SCHED", "GUARD", "CONSDIFF", "DOS", NULL
  };
  
  /** Return a bitmask for the log domain for which <b>domain</b> is the name,

diff --cc src/common/torlog.h
index 6e374b1c11a9772ddc308b1e9145ae5cbec598e0,20b7d938f03f0b1164faf956f110ac70a72d3526..0149ce9a5beb31c7a4c3830c0cdc52289a7448fe
--- 1/src/common/torlog.h
--- 2/src/common/torlog.h
+++ b/src/common/torlog.h
@@@ -99,12 -99,10 +99,14 @@@
  #define LD_CHANNEL   (1u<<21)
  /** Scheduler */
  #define LD_SCHED     (1u<<22)
 -#define LD_DOS       (1u<<23)
 +/** Guard nodes */
 +#define LD_GUARD     (1u<<23)
 +/** Generation and application of consensus diffs. */
 +#define LD_CONSDIFF  (1u<<24)
+ /** Denial of Service mitigation. */
++#define LD_DOS       (1u<<25)
  /** Number of logging domains in the code. */
- #define N_LOGGING_DOMAINS 25
 -#define N_LOGGING_DOMAINS 24
++#define N_LOGGING_DOMAINS 26
  
  /** This log message is not safe to send to a callback-based logger
   * immediately.  Used as a flag, not a log domain. */

diff --cc src/or/channel.c
index 2970b967911365418947a8327b64297fd3c7edd1,54e10666d2d91879e5555c6419334c3c1bb6dd04..9f652b5845681225ba5a071ef996c84a9ed23807
--- 1/src/or/channel.c
--- 2/src/or/channel.c
+++ b/src/or/channel.c
@@@ -2696,9 -2570,20 +2696,10 @@@ channel_do_open_actions(channel_t *chan
      router_set_status(chan->identity_digest, 1);
    } else {
      /* only report it to the geoip module if it's not a known router */
 -    if (!router_get_by_id_digest(chan->identity_digest)) {
 +    if (!connection_or_digest_is_known_relay(chan->identity_digest)) {
        if (channel_get_addr_if_possible(chan, &remote_addr)) {
          char *transport_name = NULL;
+         channel_tls_t *tlschan = BASE_CHAN_TO_TLS(chan);
          if (chan->get_transport_name(chan, &transport_name) < 0)
            transport_name = NULL;
  

diff --cc src/or/channel.h
Simple merge

diff --cc src/or/channeltls.c
Simple merge

diff --cc src/or/command.c
Simple merge

diff --cc src/or/config.c
index f11c183b88c961ff789991f05272b96429ce7e09,3b40274339814f971e6a584ca7671e7be056fc41..96cbbc7c7de80766e721f546b57e393d875d5a28
--- 1/src/or/config.c
--- 2/src/or/config.c
+++ b/src/or/config.c
@@@ -301,8 -240,21 +302,21 @@@ static config_var_t option_vars_[] = 
    OBSOLETE("DisableIOCP"),
    OBSOLETE("DisableV2DirectoryInfo_"),
    OBSOLETE("DynamicDHGroups"),
 -  VPORT(DNSPort,                     LINELIST, NULL),
 -  V(DNSListenAddress,            LINELIST, NULL),
 +  VPORT(DNSPort),
 +  OBSOLETE("DNSListenAddress"),
+   /* DoS circuit creation options. */
+   V(DoSCircuitCreationEnabled,   AUTOBOOL, "auto"),
+   V(DoSCircuitCreationMinConnections,      UINT, "0"),
+   V(DoSCircuitCreationRate,      UINT,     "0"),
+   V(DoSCircuitCreationBurst,     UINT,     "0"),
+   V(DoSCircuitCreationDefenseType,         INT,  "0"),
+   V(DoSCircuitCreationDefenseTimePeriod,   INTERVAL, "0"),
+   /* DoS connection options. */
+   V(DoSConnectionEnabled,        AUTOBOOL, "auto"),
+   V(DoSConnectionMaxConcurrentCount,       UINT, "0"),
+   V(DoSConnectionDefenseType,    INT,      "0"),
+   /* DoS single hop client options. */
+   V(DoSRefuseSingleHopClientRendezvous,    AUTOBOOL, "auto"),
    V(DownloadExtraInfo,           BOOL,     "0"),
    V(TestingEnableConnBwEvent,    BOOL,     "0"),
    V(TestingEnableCellStatsEvent, BOOL,     "0"),

diff --cc src/or/connection.c
Simple merge

diff --cc src/or/geoip.c
Simple merge

diff --cc src/or/geoip.h
Simple merge

diff --cc src/or/include.am
index 1ef5afa013555be692087d0f934a929ebb1ee36e,5108a08e5359b1fc0cc88912a952b905e7966fa2..5d5ff5c41d139ca42ab440d7940b9e72b3c018cc
--- 1/src/or/include.am
--- 2/src/or/include.am
+++ b/src/or/include.am
@@@ -48,12 -43,9 +48,13 @@@ LIBTOR_A_SOURCES = 
        src/or/dirvote.c                                \
        src/or/dns.c                                    \
        src/or/dnsserv.c                                \
+       src/or/dos.c                                    \
        src/or/fp_pair.c                                \
        src/or/geoip.c                                  \
 +      src/or/hs_intropoint.c          \
 +      src/or/hs_circuitmap.c          \
 +      src/or/hs_ntor.c                \
 +      src/or/hs_service.c             \
        src/or/entrynodes.c                             \
        src/or/ext_orport.c                             \
        src/or/hibernate.c                              \

diff --cc src/or/main.c
index 0d91803d4e555c95a76fb241f334176f0753ed55,fcd8dc90240fe3d5479e4fa569807edd29141bcc..197dfd4308c918e608868fdb6a05501a9816db59
--- 1/src/or/main.c
--- 2/src/or/main.c
+++ b/src/or/main.c
@@@ -3239,8 -2990,7 +3240,9 @@@ tor_free_all(int postfork
    control_free_all();
    sandbox_free_getaddrinfo_cache();
    protover_free_all();
 +  bridges_free_all();
 +  consdiffmgr_free_all();
+   dos_free_all();
    if (!postfork) {
      config_free_all();
      or_state_free_all();

diff --cc src/or/networkstatus.c
Simple merge

diff --cc src/or/nodelist.c
Simple merge

diff --cc src/or/nodelist.h
Simple merge

diff --cc src/or/or.h
index 95281e701f558edcab56d9f0d424b9e603801857,024a9cff0f32fdcc8d0405df8736f4359437a96a..9e7833386c61437aa60bfbb5168f5f2d032b8229
--- 1/src/or/or.h
--- 2/src/or/or.h
+++ b/src/or/or.h
@@@ -4562,22 -4515,34 +4566,51 @@@ typedef struct 
    /** If 1, we skip all OOS checks. */
    int DisableOOSCheck;
  
 +  /** Autobool: Should we include Ed25519 identities in extend2 cells?
 +   * If -1, we should do whatever the consensus parameter says. */
 +  int ExtendByEd25519ID;
 +
 +  /** Bool (default: 1): When testing routerinfos as a directory authority,
 +   * do we enforce Ed25519 identity match? */
 +  /* NOTE: remove this option someday. */
 +  int AuthDirTestEd25519LinkKeys;
 +
 +  /** Bool (default: 0): Tells if a %include was used on torrc */
 +  int IncludeUsed;
 +
 +  /** The seconds after expiration which we as a relay should keep old
 +   * consensuses around so that we can generate diffs from them.  If 0,
 +   * use the default. */
 +  int MaxConsensusAgeForDiffs;
++
+   /** Autobool: Is the circuit creation DoS mitigation subsystem enabled? */
+   int DoSCircuitCreationEnabled;
+   /** Minimum concurrent connection needed from one single address before any
+    * defense is used. */
+   int DoSCircuitCreationMinConnections;
+   /** Circuit rate used to refill the token bucket. */
+   int DoSCircuitCreationRate;
+   /** Maximum allowed burst of circuits. Reaching that value, the address is
+    * detected as malicious and a defense might be used. */
+   int DoSCircuitCreationBurst;
+   /** When an address is marked as malicous, what defense should be used
+    * against it. See the dos_cc_defense_type_t enum. */
+   int DoSCircuitCreationDefenseType;
+   /** For how much time (in seconds) the defense is applicable for a malicious
+    * address. A random time delta is added to the defense time of an address
+    * which will be between 1 second and half of this value. */
+   int DoSCircuitCreationDefenseTimePeriod;
+ 
+   /** Autobool: Is the DoS connection mitigation subsystem enabled? */
+   int DoSConnectionEnabled;
+   /** Maximum concurrent connection allowed per address. */
+   int DoSConnectionMaxConcurrentCount;
+   /** When an address is reaches the maximum count, what defense should be
+    * used against it. See the dos_conn_defense_type_t enum. */
+   int DoSConnectionDefenseType;
+ 
+   /** Autobool: Do we refuse single hop client rendezvous? */
+   int DoSRefuseSingleHopClientRendezvous;
  } or_options_t;
  
  /** Persistent state for an onion router, as saved to disk. */

diff --cc src/or/relay.c
index a86ae9bc4338768514316a0862a020fc73ef5c26,22ce7675238bae2c35d06d563bb4db827921191e..7a074d63daec951fe1a83980af58e3f340c82cd8
--- 1/src/or/relay.c
--- 2/src/or/relay.c
+++ b/src/or/relay.c
@@@ -2618,11 -2469,15 +2618,15 @@@ static time_t last_time_under_memory_pr
  STATIC int
  cell_queues_check_size(void)
  {
+   time_t now = time(NULL);
    size_t alloc = cell_queues_get_total_allocation();
    alloc += buf_get_total_allocation();
 -  alloc += tor_zlib_get_total_allocation();
 +  alloc += tor_compress_get_total_allocation();
    const size_t rend_cache_total = rend_cache_get_total_allocation();
    alloc += rend_cache_total;
+   const size_t geoip_client_cache_total =
+     geoip_client_cache_total_allocation();
+   alloc += geoip_client_cache_total;
    if (alloc >= get_options()->MaxMemInQueues_low_threshold) {
      last_time_under_memory_pressure = approx_time();
      if (alloc >= get_options()->MaxMemInQueues) {
@@@ -2632,8 -2487,16 +2636,14 @@@
        if (rend_cache_total > get_options()->MaxMemInQueues / 5) {
          const size_t bytes_to_remove =
            rend_cache_total - (size_t)(get_options()->MaxMemInQueues / 10);
 -        rend_cache_clean_v2_descs_as_dir(now, bytes_to_remove);
 -        alloc -= rend_cache_total;
 -        alloc += rend_cache_get_total_allocation();
 +        alloc -= hs_cache_handle_oom(time(NULL), bytes_to_remove);
        }
+       if (geoip_client_cache_total > get_options()->MaxMemInQueues / 5) {
+         const size_t bytes_to_remove =
+           geoip_client_cache_total -
+           (size_t)(get_options()->MaxMemInQueues / 10);
+         alloc -= geoip_client_cache_handle_oom(now, bytes_to_remove);
+       }
        circuits_handle_oom(alloc);
        return 1;
      }

diff --cc src/or/rendmid.c
index 23c3deddaa9b33d20045869e77bb698b04c2d6b6,441d5043ce2b45cdf15fae93ec3595082f87ebde..89739e12919b55cfc876168b1a5a195bf680c254
--- 1/src/or/rendmid.c
--- 2/src/or/rendmid.c
+++ b/src/or/rendmid.c
@@@ -11,7 -12,7 +12,8 @@@
  #include "circuitlist.h"
  #include "circuituse.h"
  #include "config.h"
 +#include "crypto.h"
+ #include "dos.h"
  #include "relay.h"
  #include "rendmid.h"
  #include "rephist.h"

diff --cc src/or/status.c
Simple merge

diff --cc src/test/include.am
index d5ae0bec1c0fd541d57ed8a2013439d3eb0f6b1a,0ee3d1169f681b44903ab25295b61b4b8a8d333b..723b4964e1b0cfe6e051a006a27a6b158cd1f909
--- 1/src/test/include.am
--- 2/src/test/include.am
+++ b/src/test/include.am
@@@ -104,7 -88,7 +105,8 @@@ src_test_test_SOURCES = 
        src/test/test_controller.c \
        src/test/test_controller_events.c \
        src/test/test_crypto.c \
 +      src/test/test_crypto_openssl.c \
+       src/test/test_dos.c \
        src/test/test_data.c \
        src/test/test_dir.c \
        src/test/test_dir_common.c \

diff --cc src/test/test.c
index 68f5f90fd77fb4b7c16725f41ab422b1721db6b3,0fef6979093a6224972a8f1ca26d4fd977692f36..911ef0c24eed515d7664aca3600c22ddbe33f04e
--- 1/src/test/test.c
--- 2/src/test/test.c
+++ b/src/test/test.c
@@@ -1203,7 -1216,7 +1222,8 @@@ struct testgroup_t testgroups[] = 
    { "control/", controller_tests },
    { "control/event/", controller_event_tests },
    { "crypto/", crypto_tests },
 +  { "crypto/openssl/", crypto_openssl_tests },
+   { "dos/", dos_tests },
    { "dir/", dir_tests },
    { "dir_handle_get/", dir_handle_get_tests },
    { "dir/md/", microdesc_tests },

diff --cc src/test/test.h
index 6abaf39e6fabe2ac6e5fa97e451840b9587f96c8,028082386e9aefbcb77716eebb2990d6367aacff..ea1b16adee14ae0efc3b595b401e3b977262d0a1
--- 1/src/test/test.h
--- 2/src/test/test.h
+++ b/src/test/test.h
@@@ -198,7 -191,7 +199,8 @@@ extern struct testcase_t container_test
  extern struct testcase_t controller_tests[];
  extern struct testcase_t controller_event_tests[];
  extern struct testcase_t crypto_tests[];
 +extern struct testcase_t crypto_openssl_tests[];
+ extern struct testcase_t dos_tests[];
  extern struct testcase_t dir_tests[];
  extern struct testcase_t dir_handle_get_tests[];
  extern struct testcase_t entryconn_tests[];