relay: Remove unused conn->ext_or_conn_id

This also incidently removes a use of uninitialized stack data from the
connection_or_set_ext_or_identifier() function.

Fixes #40648

Signed-off-by: David Goulet <dgoulet@torproject.org>

sandbox: Add my-consensus-<flavor-name> to sandbox for dirauth

Fixese #40663

Signed-off-by: David Goulet <dgoulet@torproject.org>

thread: Bump max detectable CPU from 16 to 128

Lets take advantage of those beefy machines ;).

Closes #40703

Signed-off-by: David Goulet <dgoulet@torproject.org>

Merge branch 'tor-gitlab/mr/645' into maint-0.4.7

Merge branch 'tor-gitlab/mr/644' into maint-0.4.7

metrics: Treat relay connections as gauge, not counter

Fixes #40699

Signed-off-by: David Goulet <dgoulet@torproject.org>

Changes file for 40683

Strip "__.SYMDEF*" before re-archiving in combine_libs on macOS and iOS.

This patch changes how combine_libs works on Darwin like platforms to
make sure we don't include any `__.SYMDEF` and `__.SYMDEF SORTED`
symbols on the archive before we repack and run ${RANLIB} on the
archive.

See: tpo/core/tor#40683.

changes: Update changes for ticket 40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

metrics: Add number of opened circuits to MetricsPort

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

relay: Add our consensus relay flag to MetricsPort

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

metrics: Add traffic related stats to MetricsPort

At this commit, bytes read and written are exported.

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

relay: Add DoS subsystem stats to MetricsPort

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

metrics: Fix naming and documentation

After nickm's review, minor changes to names and comments.

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

relay: Change the connection metrics name

Signed-off-by: David Goulet <dgoulet@torproject.org>

relay: Add CC RTT reset stats to MetricsPort

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

relay: Add total number of streams seen on MetricsPort

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

rephist: Track number of streams seen per type

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

changes: Ticket 40694

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs: Retry service rendezvous on circuit close

Move the retry from circuit_expire_building() to when the offending
circuit is being closed.

Fixes #40695

Signed-off-by: David Goulet <dgoulet@torproject.org>

circ: Get rid of hs_circ_has_timed_out

Logic is too convoluted and we can't efficiently apply a specific
timeout depending on the purpose.

Remove it and instead rely on the right circuit cutoff instead of
keeping this flagged circuit open forever.

Part of #40694

Signed-off-by: David Goulet <dgoulet@torproject.org>

circ: Set proper timeout cutoff for HS circuits

Explicitly set the S_CONNECT_REND purpose to a 4-hop cutoff.

As for the established rendezvous circuit waiting on the RENDEZVOUS2,
set one that is very long considering the possible waiting time for the
service to get the request and join our rendezvous.

Part of #40694

Signed-off-by: David Goulet <dgoulet@torproject.org>

hs: Retry rdv circuit if repurposed

This can happen if our measurement subsystem decides to snatch it.

Fixes #40696

Signed-off-by: David Goulet <dgoulet@torproject.org>

Merge branch 'tor-gitlab/mr/635' into maint-0.4.7

hs: Change the error for a collapsing client circuit

Change it to an "unreachable" error so the intro point can be retried
and not flagged as a failure and never retried again.

Closes #40692

Signed-off-by: David Goulet <dgoulet@torproject.org>

Merge branch 'maint-0.4.5' into maint-0.4.7

Merge branch 'tor-gitlab/mr/631' into maint-0.4.5

dirauth: Remove Faravahar

Closes #40688

Signed-off-by: David Goulet <dgoulet@torproject.org>

Merge branch 'maint-0.4.5' into maint-0.4.7

Merge branch 'tor-gitlab/mr/629' into maint-0.4.7

relay: Reduce the minimum circuit cell in queue limit

With congestion control, the flow control window is much lower than the
initial 1000.

Signed-off-by: David Goulet <dgoulet@torproject.org>

dos: Apply circuit creation defenses if circ max queue cell reached

This adds two consensus parameters to control the outbound max circuit
queue cell size limit and how many times it is allowed to reach that
limit for a single client IP.

Closes #40680

Signed-off-by: David Goulet <dgoulet@torproject.org>

dir auths now omit Measured= if rs->is_authority

Directory authorities stop voting a consensus "Measured" weight
for relays with the Authority flag. Now these relays will be
considered unmeasured, which should reserve their bandwidth
for their dir auth role and minimize distractions from other roles.

In place of the "Measured" weight, they now include a
"MeasuredButAuthority" weight (not used by anything) so the bandwidth
authority's opinion on this relay can be recorded for posterity.

Resolves ticket 40698.

back out most of commit b7992d4f

The AuthDirDontVoteOnDirAuthBandwidth torrc option never worked, and it
was implemented in a way that could have produced consensus conflicts
if it had.

Resolves bug 40700.

fix typo in #40673's changes file

dirauth: Change dizum IP address

Closes #40687

Signed-off-by: David Goulet <dgoulet@torproject.org>

Merge branch 'maint-0.4.5' into maint-0.4.7

Fix a completely wrong calculation in mach monotime_init_internal()

Bug 1: We were purporting to calculate milliseconds per tick, when we
*should* have been computing ticks per millisecond.

Bug 2: Instead of computing either one of those, we were _actually_
computing femtoseconds per tick.

These two bugs covered for one another on x86 hardware, where 1 tick
== 1 nanosecond.  But on M1 OSX, 1 tick is about 41 nanoseconds,
causing surprising results.

Fixes bug 40684; bugfix on 0.3.3.1-alpha.

relay: Add number of rejected connections to MetricsPort

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

relay: Add connection stats to MetricsPort

This adds the number of created and opened connections to the
MetricsPort for a relay for each connection type and direction.

Output looks like:

  # HELP tor_relay_connections Connections metrics of this relay
  # TYPE tor_relay_connections counter
  tor_relay_connections{type="OR listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="OR listener",direction="received",state="created"} 0
  tor_relay_connections{type="OR listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="OR listener",direction="received",state="opened"} 0
  tor_relay_connections{type="OR",direction="initiated",state="created"} 5
  tor_relay_connections{type="OR",direction="received",state="created"} 0
  tor_relay_connections{type="OR",direction="initiated",state="opened"} 5
  tor_relay_connections{type="OR",direction="received",state="opened"} 0
  tor_relay_connections{type="Exit",direction="initiated",state="created"} 0
  tor_relay_connections{type="Exit",direction="received",state="created"} 0
  tor_relay_connections{type="Exit",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Exit",direction="received",state="opened"} 0
  tor_relay_connections{type="Socks listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="Socks listener",direction="received",state="created"} 0
  tor_relay_connections{type="Socks listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Socks listener",direction="received",state="opened"} 0
  tor_relay_connections{type="Socks",direction="initiated",state="created"} 0
  tor_relay_connections{type="Socks",direction="received",state="created"} 0
  tor_relay_connections{type="Socks",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Socks",direction="received",state="opened"} 0
  tor_relay_connections{type="Directory listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="Directory listener",direction="received",state="created"} 0
  tor_relay_connections{type="Directory listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Directory listener",direction="received",state="opened"} 0
  tor_relay_connections{type="Directory",direction="initiated",state="created"} 0
  tor_relay_connections{type="Directory",direction="received",state="created"} 0
  tor_relay_connections{type="Directory",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Directory",direction="received",state="opened"} 0
  tor_relay_connections{type="Control listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="Control listener",direction="received",state="created"} 0
  tor_relay_connections{type="Control listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Control listener",direction="received",state="opened"} 0
  tor_relay_connections{type="Control",direction="initiated",state="created"} 0
  tor_relay_connections{type="Control",direction="received",state="created"} 0
  tor_relay_connections{type="Control",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Control",direction="received",state="opened"} 0
  tor_relay_connections{type="Transparent pf/netfilter listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="Transparent pf/netfilter listener",direction="received",state="created"} 0
  tor_relay_connections{type="Transparent pf/netfilter listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Transparent pf/netfilter listener",direction="received",state="opened"} 0
  tor_relay_connections{type="Transparent natd listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="Transparent natd listener",direction="received",state="created"} 0
  tor_relay_connections{type="Transparent natd listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Transparent natd listener",direction="received",state="opened"} 0
  tor_relay_connections{type="DNS listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="DNS listener",direction="received",state="created"} 0
  tor_relay_connections{type="DNS listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="DNS listener",direction="received",state="opened"} 0
  tor_relay_connections{type="Extended OR",direction="initiated",state="created"} 0
  tor_relay_connections{type="Extended OR",direction="received",state="created"} 0
  tor_relay_connections{type="Extended OR",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Extended OR",direction="received",state="opened"} 0
  tor_relay_connections{type="Extended OR listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="Extended OR listener",direction="received",state="created"} 0
  tor_relay_connections{type="Extended OR listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Extended OR listener",direction="received",state="opened"} 0
  tor_relay_connections{type="HTTP tunnel listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="HTTP tunnel listener",direction="received",state="created"} 0
  tor_relay_connections{type="HTTP tunnel listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="HTTP tunnel listener",direction="received",state="opened"} 0
  tor_relay_connections{type="Metrics listener",direction="initiated",state="created"} 0
  tor_relay_connections{type="Metrics listener",direction="received",state="created"} 1
  tor_relay_connections{type="Metrics listener",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Metrics listener",direction="received",state="opened"} 1
  tor_relay_connections{type="Metrics",direction="initiated",state="created"} 0
  tor_relay_connections{type="Metrics",direction="received",state="created"} 0
  tor_relay_connections{type="Metrics",direction="initiated",state="opened"} 0
  tor_relay_connections{type="Metrics",direction="received",state="opened"} 0

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

conn: Keep stats of opened and closed connections

Related to #40194

Signed-off-by: David Goulet <dgoulet@torproject.org>

Properly compute cell-drop overload fraction

Patch to address #40673. An additional check has been added to
onion_pending_add() in order to ensure that we avoid counting create
cells from clients.

In the cpuworker.c assign_onionskin_to_cpuworker
method if total_pending_tasks >= max_pending_tasks
and channel_is_client(circ->p_chan) returns false then
rep_hist_note_circuit_handshake_dropped() will be called and
rep_hist_note_circuit_handshake_assigned() will not be called. This
causes relays to run into errors due to the fact that the number of
dropped packets exceeds the total number of assigned packets.

To avoid this situation a check has been added to
onion_pending_add() to ensure that these erroneous calls to
rep_hist_note_circuit_handshake_dropped() are not made.

See the #40673 ticket for the conversation with armadev about this issue.

version: Bump version to 0.4.7.10-dev

Merge branch 'maint-0.4.6' into maint-0.4.7

version: Bump version to 0.4.6.12-dev

Merge branch 'maint-0.4.5' into maint-0.4.6

version: Bump version to 0.4.5.14-dev

version: Bump version to 0.4.7.10

Merge branch 'maint-0.4.6' into maint-0.4.7

version: Bump version to 0.4.6.12

Merge branch 'maint-0.4.6' into maint-0.4.7

Merge branch 'maint-0.4.5' into maint-0.4.6

version: Bump version to 0.4.5.14

Merge branch 'maint-0.4.6' into maint-0.4.7

Merge branch 'maint-0.4.5' into maint-0.4.6

geoip: Update geoip files with August 9th, 2022 database

Fixes #40658

Signed-off-by: David Goulet <dgoulet@torproject.org>

version: Bump version to 0.4.7.9-dev

Merge branch 'maint-0.4.6' into maint-0.4.7

version: Bump version to 0.4.6.11-dev

Merge branch 'maint-0.4.5' into maint-0.4.6

version: Bump version to 0.4.5.13-dev

version: Bump version to 0.4.7.9

Merge branch 'maint-0.4.6' into maint-0.4.7

version: Bump version to 0.4.6.11

Merge branch 'maint-0.4.6' into maint-0.4.7

Merge branch 'maint-0.4.5' into maint-0.4.6

version: Bump version to 0.4.5.13

Merge branch 'maint-0.4.6' into maint-0.4.7

Merge branch 'maint-0.4.5' into maint-0.4.6

fallbackdir: Update list generated on August 11, 2022

Update geoip files to match ipfire location db, 2022/08/11.

Merge branch 'tor-gitlab/mr/613' into maint-0.4.7

Tune congestion control parameters.

Add changes file for bug40642.

Reduce the number of vegas parameters.

We need to tune these, but we're not likely to need the subtle differences
between a few of them. Removing them will prevent our consensus parameter
string from becoming too long in the event of tuning.

Reset the min value if we hit cwnd_min.

This can avoid circuits getting stuck due to an abnormally low min value.

Use EWMA instead of bare rtt for min rtt.

This allows us to average out minimums due to lulls in activity a bit more.

Create slow-start max for n_ewma_cnt.

Since slow-start now checks every sendme, lower EWMA is better.

Implement RFC3742 Limited Slow Start

RFC3742 updates the cwnd every sendme during slow start, and backs off of the
exponential growth based on a cap parameter.

Merge branch 'maint-0.4.5' into maint-0.4.6

Merge branch 'maint-0.4.6' into maint-0.4.7

fallbackdirs: Update list from maint-0.4.7

Signed-off-by: David Goulet <dgoulet@torproject.org>

Merge branch 'maint-0.4.5' into maint-0.4.6

Merge branch 'maint-0.4.6' into maint-0.4.7

geoip: Update files from maint-0.4.7

Signed-off-by: David Goulet <dgoulet@torproject.org>

ignore families for L2 guard independence

mike is concerned that we would get too much exposure to adversaries,
if we enforce that none of our L2 guards can be in the same family.

this change set now essentially finishes the feature that commit a77727cdc
was attempting to add, but strips the "_and_family" part of that plan.

make L2 vanguards actually independent

We had omitted some checks for whether our vanguards (second layer
guards from proposal 333) overlapped or came from the same family.
Now make sure to pick each of them to be independent.

Fixes bug 40639; bugfix on 0.4.7.1-alpha.

man: Fix typo for AuthDirMiddleOnly option

Signed-off-by: David Goulet <dgoulet@torproject.org>

dirauth: Make voting flag threshold tunable via torrc

Remove UPTIME_TO_GUARANTEE_STABLE, MTBF_TO_GUARANTEE_STABLE,
TIME_KNOWN_TO_GUARANTEE_FAMILIAR WFU_TO_GUARANTEE_GUARD and replace each
of them with a tunnable torrc option.

Related to #40652

Signed-off-by: David Goulet <dgoulet@torproject.org>

dirauth: Add a AuthDirVoteGuard to pin Guard flags

Related to #40652

Signed-off-by: David Goulet <dgoulet@torproject.org>

Merge branch 'maint-0.4.5' into maint-0.4.6

Merge branch 'maint-0.4.6' into maint-0.4.7

Merge branch 'tor-gitlab/mr/608' into maint-0.4.5

Merge branch 'maint-0.4.5' into maint-0.4.6

Merge branch 'maint-0.4.6' into maint-0.4.7

relay: Don't send DESTROY remote reason backward or forward

Fixes #40649

Signed-off-by: David Goulet <dgoulet@torproject.org>

Changes file for bug 40644.

Add an underflow check to a cwnd error condition.

conn: Notify btrack subsys on normal OR conn close

Fixes #40604

Signed-off-by: David Goulet <dgoulet@torproject.org>

Merge branch 'maint-0.4.5' into maint-0.4.6