Nick Mathewson [Wed, 12 Jul 2017 16:23:33 +0000 (12:23 -0400)]
Always start with one additional worker thread
Now that half the threads are permissive and half are strict, we
need to make sure we have at least two threads, so that we'll
have at least one of each kind.
Nick Mathewson [Wed, 12 Jul 2017 16:18:33 +0000 (12:18 -0400)]
Make the chance for priority inversion thread-specific
Instead of choosing a lower-priority job with a 1/37 chance, have
the chance be 1/37 for half the threads, and 1/2147483647 for the
other half. This way if there are very slow jobs of low priority,
they shouldn't be able to grab all the threads when there is better
work to do.
Nick Mathewson [Wed, 12 Jul 2017 15:47:01 +0000 (11:47 -0400)]
Add support for multi-priority workqueues
Each piece of queued work now has an associated priority value; each
priority goes on a separate queue.
With probability (N-1)/N, the workers will take work from the highest
priority nonempty queue. Otherwise, they'll look for work in a
queue of lower priority. This behavior is meant to prevent
starvation for lower-priority tasks.
Nick Mathewson [Wed, 26 Jul 2017 19:37:53 +0000 (15:37 -0400)]
Merge branch 'maint-0.3.0' into maint-0.3.1
This is an "ours" merge to avoid taking any changs from maint-0.3.0:
we don't want to take the travis file, since there is a separate branch
for putting it in 0.3.1 and later.
Nick Mathewson [Tue, 25 Jul 2017 14:14:24 +0000 (10:14 -0400)]
Fix build warnings from Coverity related to our BUG macro
In the Linux kernel, the BUG() macro causes an instant panic. Our
BUG() macro is different, however: it generates a nonfatal assertion
failure, and is usable as an expression.
Additionally, this patch tells util_bug.h to make all assertion
failures into fatal conditions when we're building with a static
analysis tool, so that the analysis tool can look for instances
where they're reachable.
Nick Mathewson [Mon, 24 Jul 2017 18:32:59 +0000 (14:32 -0400)]
Rename the hybrid_encrypt/decrypt functions; label them as dangerous
We need to keep these around for TAP and old-style hidden services,
but they're obsolete, and we shouldn't encourage anyone to use them.
So I've added "obsolete" to their names, and a comment explaining
what the problem is.
Nick Mathewson [Mon, 24 Jul 2017 18:15:40 +0000 (14:15 -0400)]
Improve comment about why we disable TLS compression.
Closes bug 22964. Based on Teor's replacement there, but tries
to put the comment in a more logical place, and explain why we're
actually disabling compression in the first place.
David Goulet [Mon, 24 Jul 2017 15:31:54 +0000 (11:31 -0400)]
test: Remove buggy unit test in test_hs_service
There isn't much of a point of this buggy test afterall to add twice the same
service object but with a different key which ultinately can end up failing
the test because 1/N_BUCKETS of probability that we end up to put the service
in the same bucket.
Fixes #23023
Signed-off-by: David Goulet <dgoulet@torproject.org>
* CHANGE .travis.yml so that commands for different purposes (e.g. getting
dependencies, building, testing) are in separate config lines and sections.
* CHANGE .travis.yml to use their mechanism for installing dependencies via
apt. [0] This also allows us to not need sudo (the "sudo: false" line).
* CHANGE Travis CI tests (the "script:" section) to build and run tests in the
same manner as Jenkins (i.e. with --enable-fatal-warnings and
--disable-silent-rules and run `make check`).
* ADD Travis configuration to do all the target builds with both GCC and clang.
* ADD make flags to build with both of the cores available.
* ADD notifications for IRC, and configure email notifications (to the author
of the commit) only if the branch was previously building successfully and
the latest commit broke it.
* ADD the ability to run the Travis build matrix for OSX as well, but leave it
commented out by default (because it takes roughly ten times longer, due to a
shortage of OSX build machines).
* ADD Travis config option to cancel/fail the build early if one target has
already failed ("fast_finish: true").
* ADD comments to describe what our Travis config is doing and why it is
configured that way.
* CHANGE .travis.yml so that commands for different purposes (e.g. getting
dependencies, building, testing) are in separate config lines and sections.
* CHANGE .travis.yml to use their mechanism for installing dependencies via
apt. [0] This also allows us to not need sudo (the "sudo: false" line).
* CHANGE Travis CI tests (the "script:" section) to build and run tests in the
same manner as Jenkins (i.e. with --enable-fatal-warnings and
--disable-silent-rules and run `make check`).
* CHANGE Travis config to install nightly rustc and cargo.
* CHANGE Travis config to split rust install into commands for getting
dependencies ("before_install:") and commands for installing them
("install:").
* REMOVE shell redirection when downloading the rustup.sh script.
* CHANGE cargo to be in "online mode" so that we can get our Rust dependencies.
There's not really a way to get the dependencies without using cargo
right now. See https://bugs.torproject.org/22830 for more info.
* REMOVE cargo "offline mode" envvars from rustup.sh invocation.
* ADD commands to get more info about rustc and cargo before building.
* FIX sourcing the cargo/toolchain environment that rustup creates. (Without
this, our build scripts don't know about anything called "rustc" or "cargo".)
* ADD Travis configuration to do all the target builds with both GCC and clang.
* ADD make flags to build with both of the cores available.
* ADD notifications for IRC, and configure email notifications (to the author
of the commit) only if the branch was previously building successfully and
the latest commit broke it.
* ADD the ability to run the Travis build matrix for OSX as well, but leave it
commented out by default (because it takes roughly ten times longer, due to a
shortage of OSX build machines).
* ADD Travis config option to cancel/fail the build early if one target has
already failed ("fast_finish: true").
* ADD comments to describe what our Travis config is doing and why it is
configured that way.
Nick Mathewson [Fri, 14 Jul 2017 20:21:51 +0000 (16:21 -0400)]
Fix zstd 1.3.0 trouble: Be more respectful of its state machine
In zstd 1.3.0, once you have called ZSTD_endStream and been told
that your putput buffer is full, it really doesn't want you to call
ZSTD_compressStream again. ZSTD 1.2.0 didn't seem to mind about
this.
This patch fixes the issue by making sure never to call
ZSTD_endStream if there's any more data on the input buffer to
process, by flushing even when we're about to call "endStream", and
by never calling "compress" or "flush" after "endStream".
Nick Mathewson [Fri, 14 Jul 2017 17:56:40 +0000 (13:56 -0400)]
Fix a signed integer overflow in dir/download_status_random_backoff
Fix for 22924. Bugfix on 0.2.9.1-alpha when the test was introducd
-- though it couldn't actually overflow until we fixed 17750.
Additionally, this only seems to overflow on 32-bit, and only when
the compiler doesn't re-order the (possibly dead) assignment out of
the way. We ran into it on a 32-bit ubuntu trusty builder.
Nick Mathewson [Thu, 13 Jul 2017 21:49:48 +0000 (17:49 -0400)]
Fix compiler warnings with openssl-scrypt/libscrypt test on clang
Clang didn't like that we were passing uint64_t values to an API
that wanted uint32_t. GCC has either not cared, or has figured out
that the values in question were safe to cast to uint32_t.