.. toctree::
gen
- concurrent
locks
queues
process
-``tornado.gen`` --- Simplify asynchronous code
+``tornado.gen`` --- Generator-based coroutines
==============================================
.. testsetup::
executor = concurrent.futures.ThreadPoolExecutor()
await tornado.gen.convert_yielded(executor.submit(g))
-While native coroutines are not visibly tied to a particular framework
-(i.e. they do not use a decorator like `tornado.gen.coroutine` or
-`asyncio.coroutine`), not all coroutines are compatible with each
-other. There is a *coroutine runner* which is selected by the first
-coroutine to be called, and then shared by all coroutines which are
-called directly with ``await``. The Tornado coroutine runner is
-designed to be versatile and accept awaitable objects from any
-framework; other coroutine runners may be more limited (for example,
-the ``asyncio`` coroutine runner does not accept coroutines from other
-frameworks). For this reason, it is recommended to use the Tornado
-coroutine runner for any application which combines multiple
-frameworks. To call a coroutine using the Tornado runner from within a
-coroutine that is already using the asyncio runner, use the
-`tornado.platform.asyncio.to_asyncio_future` adapter.
-
-
How it works
~~~~~~~~~~~~
Coroutine patterns
~~~~~~~~~~~~~~~~~~
-Interaction with callbacks
-^^^^^^^^^^^^^^^^^^^^^^^^^^
-
-To interact with asynchronous code that uses callbacks instead of
-`.Future`, wrap the call in a `.Task`. This will add the callback
-argument for you and return a `.Future` which you can yield:
-
-.. testcode::
-
- @gen.coroutine
- def call_task():
- # Note that there are no parens on some_function.
- # This will be translated by Task into
- # some_function(other_args, callback=callback)
- yield gen.Task(some_function, other_args)
-
-.. testoutput::
- :hide:
-
Calling blocking functions
^^^^^^^^^^^^^^^^^^^^^^^^^^
The simplest way to call a blocking function from a coroutine is to
-use a `~concurrent.futures.ThreadPoolExecutor`, which returns
+use `.IOLoop.run_in_executor`, which returns
``Futures`` that are compatible with coroutines::
- thread_pool = ThreadPoolExecutor(4)
-
@gen.coroutine
def call_blocking():
- yield thread_pool.submit(blocking_func, args)
+ yield IOLoop.current().run_in_executor(blocking_func, args)
Parallelism
^^^^^^^^^^^
-The coroutine decorator recognizes lists and dicts whose values are
+The `.coroutine` decorator recognizes lists and dicts whose values are
``Futures``, and waits for all of those ``Futures`` in parallel:
.. testcode::
.. testoutput::
:hide:
+Lists and dicts must be wrapped in `tornado.gen.multi` for use with
+``await``::
+
+ async def parallel_fetch(url1, url2):
+ resp1, resp2 = await gen.multi([http_client.fetch(url1),
+ http_client.fetch(url2)])
+
Interleaving
^^^^^^^^^^^^
Looping
^^^^^^^
-Looping is tricky with coroutines since there is no way in Python
-to ``yield`` on every iteration of a ``for`` or ``while`` loop and
-capture the result of the yield. Instead, you'll need to separate
-the loop condition from accessing the results, as in this example
-from `Motor <https://motor.readthedocs.io/en/stable/>`_::
+In native coroutines, ``async for`` can be used. In older versions of
+Python, looping is tricky with coroutines since there is no way to
+``yield`` on every iteration of a ``for`` or ``while`` loop and
+capture the result of the yield. Instead, you'll need to separate the
+loop condition from accessing the results, as in this example from
+`Motor <https://motor.readthedocs.io/en/stable/>`_::
import motor
db = motor.MotorClient().test
asynchronous techniques in `.RequestHandler` subclasses.
The simplest way to make a handler asynchronous is to use the
-`.coroutine` decorator. This allows you to perform non-blocking I/O
-with the ``yield`` keyword, and no response will be sent until the
-coroutine has returned. See :doc:`coroutines` for more details.
+`.coroutine` decorator or ``async def``. This allows you to perform
+non-blocking I/O with the ``yield`` or ``await`` keywords, and no
+response will be sent until the coroutine has returned. See
+:doc:`coroutines` for more details.
In some cases, coroutines may be less convenient than a
callback-oriented style, in which case the `.tornado.web.asynchronous`
.. versionadded:: 4.2
-Coordinate coroutines with synchronization primitives analogous to those the
-standard library provides to threads.
+Coordinate coroutines with synchronization primitives analogous to
+those the standard library provides to threads. These classes are very
+similar to those provided in the standard library's `asyncio package
+<https://docs.python.org/3/library/asyncio-sync.html>`_.
.. warning::
.. toctree::
autoreload
+ concurrent
log
options
stack_context
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
-"""Utilities for working with threads and ``Futures``.
+"""Utilities for working with ``Future`` objects.
``Futures`` are a pattern for concurrent programming introduced in
-Python 3.2 in the `concurrent.futures` package. This package defines
-a mostly-compatible `Future` class designed for use from coroutines,
-as well as some utility functions for interacting with the
-`concurrent.futures` package.
+Python 3.2 in the `concurrent.futures` package, and also adopted (in a
+slightly different form) in Python 3.4's `asyncio` package. This
+package defines a ``Future`` class that is an alias for `asyncio.Future`
+when available, and a compatible implementation for older versions of
+Python. It also includes some utility functions for interacting with
+``Future`` objects.
+
+While this package is an important part of Tornado's internal
+implementation, applications rarely need to interact with it
+directly.
"""
from __future__ import absolute_import, division, print_function
pass
This decorator should not be confused with the similarly-named
- `.IOLoop.run_in_executor`.
+ `.IOLoop.run_in_executor`. In general, using ``run_in_executor``
+ when *calling* a blocking method is recommended instead of using
+ this decorator when *defining* a method. If compatibility with older
+ versions of Tornado is required, consider defining an executor
+ and using ``executor.submit()`` at the call site.
.. versionchanged:: 4.2
Added keyword arguments to use alternative attributes.
"""Decorator to make a function that returns via callback return a
`Future`.
+ This decorator was provided to ease the transition from
+ callback-oriented code to coroutines. It is not recommended for
+ new code.
+
The wrapped function should take a ``callback`` keyword argument
and invoke it with one argument when it has finished. To signal failure,
the function can simply raise an exception (which will be
Note that ``@return_future`` and ``@gen.engine`` can be applied to the
same function, provided ``@return_future`` appears first. However,
consider using ``@gen.coroutine`` instead of this combination.
+
"""
replacer = ArgReplacer(f, 'callback')
-"""``tornado.gen`` is a generator-based interface to make it easier to
-work in an asynchronous environment. Code using the ``gen`` module
-is technically asynchronous, but it is written as a single generator
+"""``tornado.gen`` implements generator-based coroutines.
+
+.. note::
+
+ The "decorator and generator" approach in this module is a
+ precursor to native coroutines (using ``async def`` and ``await``)
+ which were introduced in Python 3.5. Applications that do not
+ require compatibility with older versions of Python should use
+ native coroutines instead. Some parts of this module are still
+ useful with native coroutines, notably `multi`, `sleep`,
+ `WaitIterator`, and `with_timeout`. Some of these functions have
+ counterparts in the `asyncio` module which may be used as well,
+ although the two may not necessarily be 100% compatible.
+
+Coroutines provide an easier way to work in an asynchronous
+environment than chaining callbacks. Code using coroutines is
+technically asynchronous, but it is written as a single generator
instead of a collection of separate functions.
For example, the following asynchronous handler:
This function is available under the names ``multi()`` and ``Multi()``
for historical reasons.
+ Cancelling a `.Future` returned by ``multi()`` does not cancel its
+ children. `asyncio.gather` is similar to ``multi()``, but it does
+ cancel its children.
+
.. versionchanged:: 4.2
If multiple yieldables fail, any exceptions after the first
(which is raised) will be logged. Added the ``quiet_exceptions``
Does not support `YieldPoint` subclasses.
+ The wrapped `.Future` is not canceled when the timeout expires,
+ permitting it to be reused. `asyncio.wait_for` is similar to this
+ function but it does cancel the wrapped `.Future` on timeout.
+
.. versionadded:: 4.0
.. versionchanged:: 4.1
.. versionchanged:: 4.4
Added support for yieldable objects other than `.Future`.
+
"""
# TODO: allow YieldPoints in addition to other yieldables?
# Tricky to do with stack_context semantics.
# License for the specific language governing permissions and limitations
# under the License.
-"""Asynchronous queues for coroutines.
+"""Asynchronous queues for coroutines. These classes are very similar
+to those provided in the standard library's `asyncio package
+<https://docs.python.org/3/library/asyncio-queue.html>`_.
.. warning::
are *not* thread-safe. To use these queues from another thread,
use `.IOLoop.add_callback` to transfer control to the `.IOLoop` thread
before calling any queue methods.
+
"""
from __future__ import absolute_import, division, print_function
-------------------
In general, methods on `RequestHandler` and elsewhere in Tornado are
-not thread-safe. In particular, methods such as
+not thread-safe. In particular, methods such as
`~RequestHandler.write()`, `~RequestHandler.finish()`, and
-`~RequestHandler.flush()` must only be called from the main thread. If
+`~RequestHandler.flush()` must only be called from the main thread. If
you use multiple threads it is important to use `.IOLoop.add_callback`
to transfer control back to the main thread before finishing the
-request.
+request, or to limit your use of other threads to
+`.IOLoop.run_in_executor` and ensure that your callbacks running in
+the executor do not refer to Tornado objects.
"""
projects / thirdparty / tornado.git / commitdiff
