Ben Darnell [Tue, 14 Nov 2023 03:02:32 +0000 (22:02 -0500)]
iostream_test: Test check_hostname functionality.
In #3337, the removal of ssl.match_hostname revealed that we did not
have any test coverage of hostname checking in tornado.iostream.
Since we were forced to remove the manual check that we had in place
for old versions of Python, we need a test to make sure that we didn't
inadvertently break hostname checking.
Ben Darnell [Wed, 11 Oct 2023 00:39:25 +0000 (20:39 -0400)]
test: Close the thread pool in run_on_executor test
If this executor was left around it would be GC'd at an unpredictable
time and would often be reported as a failure in other circlerefs tests.
(For unknown reasons this would occur most often in i686 (i.e. 32-bit)
linux builds).
Ben Darnell [Sat, 14 Oct 2023 02:39:41 +0000 (22:39 -0400)]
*: Lint on the newest version of python too.
We previously only typechecked on the oldest version of python we
supported, incorrectly assuming nothing we depended on would be
removed. Now we typecheck on the latest version of python.
Assume support for modern version of ssl and remove some pre-SNI
code paths which rely on functions that are now removed.
Ben Darnell [Sat, 14 Oct 2023 01:27:20 +0000 (21:27 -0400)]
docs: Update intersphinx references for python 3.12
Intersphinx links are currently an unpinned dependency, so when
a new version of python is released it's possible (although relatively
rare) for it to break our links. 3.12 removed a few members of
the ssl module.
Ben Darnell [Wed, 23 Aug 2023 01:27:05 +0000 (21:27 -0400)]
escape: Use the standard library where possible
Many of these functions were necessary in Python 2, but are now
redundant. We can simply use the standard library in many cases.
The only major change is in xhtml_unescape, where we now reject
invalid character references such as surrogates and control characters.
Update docs throughout to be more specific about differences from the
standard library. Also be more complete about the ``plus`` option to
the url escaping functions.
Ben Darnell [Tue, 22 Aug 2023 03:03:39 +0000 (23:03 -0400)]
ioloop,concurrent: Fix reference cycles
In a few places we were referring to a future via a closure instead
of using the reference passed as an argument to the callback. This
sometimes causes a reference cycle that can slow GC. This commit
adds a test which covers two of the cases (chain_future and the
concurrent.future branch of add_future) while the third was found by
inspecting other calls to add_done_callback for obvious instances of
this pattern.
This test has recently become flaky on windows CI, and before
investigating further, see if it's just because the CI machines are
overloaded and subprocesses are slower on windows.
Ben Darnell [Fri, 11 Aug 2023 01:41:40 +0000 (21:41 -0400)]
httpserver_test: Add ExpectLog to fix CI
The github security advisory feature lets you make private PRs but
it apparently doesn't support CI so this log failure wasn't caught
until after the PR was merged.
Ben Darnell [Wed, 9 Aug 2023 01:55:02 +0000 (21:55 -0400)]
http1connection: Make content-length parsing more strict
Content-length and chunk size parsing now strictly matches the RFCs.
We previously used the python int() function which accepted leading
plus signs and internal underscores, which are not allowed by the
HTTP RFCs (it also accepts minus signs, but these are less problematic
in this context since they'd result in errors elsewhere)
It is important to fix this because when combined with certain proxies,
the lax parsing could result in a request smuggling vulnerability (if
both Tornado and the proxy accepted an invalid content-length but
interpreted it differently). This is known to occur with old versions
of haproxy, although the current version of haproxy is unaffected.
Ben Darnell [Thu, 27 Jul 2023 00:15:12 +0000 (20:15 -0400)]
autoreload: Add --until-success flag
This flag terminates the autoreload loop after the first successful
run. This makes it possible to cleanly shut down a process that is using
"python -m tornado.autoreload" without printing a traceback.
build(deps): bump certifi from 2022.12.7 to 2023.7.22
Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.12.7 to 2023.7.22.
- [Commits](https://github.com/certifi/python-certifi/compare/2022.12.07...2023.07.22)
Ben Darnell [Sun, 23 Jul 2023 02:10:18 +0000 (22:10 -0400)]
autoreload: Support directories in CLI wrapper
A previous commit added support for using autoreload within programs
that were started as directories; this commit supports them when
run with the -m tornado.autoreload wrapper.
This change may have side effects for file mode since we now use
runpy.run_path instead of executing the file by hand (I don't think
the run_path function existed when this code was originally written).
Ben Darnell [Fri, 14 Jul 2023 00:57:11 +0000 (20:57 -0400)]
autoreload: Support the ability to run a directory instead of a module
Running a directory has some but not all of the behavior of
running a module, including setting __spec__, so we must be careful
not to break things by assuming that __spec__ means module mode.
Ben Darnell [Sat, 8 Jul 2023 01:19:18 +0000 (21:19 -0400)]
asyncio: Remove atexit hook
This hook was added because of an only-in-CI issue, but we have since
improved our cleanup of the selector thread. As long as this passes
CI, I think we can remove the atexit hook.
Ben Darnell [Sat, 8 Jul 2023 00:04:27 +0000 (20:04 -0400)]
auth: Update facebook scope
The read_stream scope was replaced with user_posts; this change
was made to demos/facebook/facebook.py in #1674 but the corresponding
comment was not updated. The offline_access scope has also been removed
but seems irrelvant to this comment.
Ben Darnell [Fri, 9 Jun 2023 02:52:19 +0000 (22:52 -0400)]
*: Adapt to deprecation of datetime utc methods
Python 3.12 deprecates the utcnow and utcfromtimestamp methods and
discourages the use of naive datetimes to represent UTC. This was
previously the main way that Tornado used datetimes (since it was
the only option available in Python 2 before the introduction
of datetime.timezone.utc in Python 3.2).
- httpclient_test: Test-only change to test that both kinds of datetimes
are supported in If-Modified-Since (this just calls
httputil.format_timestamp)
- httputil: No functional changes, but format_timestamp's
support for both naive and aware datetimes is now tested.
- locale: format_timestamp now supports aware datetimes (in
addition to the existing support for naive datetimes).
- web: Cookie expirations internally use aware datetimes.
StaticFileHandler.get_modified_time now supports both and the
standard implementation returns aware.
It feels fragile that "naive" and "aware" datetimes are not distinct
types but subject to data-dependent behavior. This change uses
"aware" datetimes throughout Tornado, but some operations (comparisons
and subtraction) fail with mixed datetime types and if I missed any
in this change may cause errors if naive datetimes were used (where
previously naive datetimes would have been required). But that's
apparently the API we have to work with.
Ben Darnell [Mon, 19 Jun 2023 19:28:45 +0000 (15:28 -0400)]
asyncio_test: Use inequality when checking thread leaks
Sometimes we have a net reduction in the thread count
because there was an extra thread running at the time captured
the starting count, so use inequality instead of exact matches.
Ben Darnell [Wed, 17 May 2023 00:57:50 +0000 (20:57 -0400)]
asyncio: Manage the selector thread with an async generator
Async generators have a special shutdown protocol which allows
us to detect the end of the event loop and stop our thread.
This lets us clean up the thread reliably when the event loop
is started/stopped via the tornado IOLoop interfaces (which
explicitly know about the selector thread), or when the
latest asyncio interfaces are used (asyncio.run or manually
calling shutdown_asyncgens).
The thread is still leaked when older versions of the asyncio
interfaces are used (loop.close *without* shutdown_asyncgens), but
I've been unable to find a solution that does not print leak warnings
even in the event of a clean shutdown. Use of shutdown_asyncgens is
now effectively required for apps combining asyncio and tornado.
This is unfortunate since leaking a thread is relatively expensive
compared to the usual consequences of failing to call
shutdown_asyncgens, but it seems to be the best we can do.
projects / thirdparty / tornado.git / log
