]
# I wish this could go in a per-module file...
coverage_ignore_classes = [
+ # tornado.concurrent
+ "TracebackFuture",
+
# tornado.gen
"Multi",
"Runner",
.. autoclass:: YieldPoint
:members:
+ .. autofunction:: with_timeout
+ .. autoexception:: TimeoutError
+
+ .. autofunction:: maybe_future
+
Other classes
-------------
--- /dev/null
+``tornado.http1connection`` -- HTTP/1.x client/server implementation
+====================================================================
+
+.. automodule:: tornado.http1connection
+ :members:
----------
.. autoexception:: StreamClosedError
+ .. autoexception:: UnsatisfiableReadError
gen
ioloop
iostream
+ http1connection
httpclient
netutil
tcpserver
of the old ``TracebackFuture`` class. ``TracebackFuture`` is now
simply an alias for ``Future``.
-`tornado.httpclient`
-~~~~~~~~~~~~~~~~~~~~
-
-* The command-line HTTP client (``python -m tornado.httpclient $URL``)
- now works on Python 3.
-
`tornado.gen`
~~~~~~~~~~~~~
* Performance of coroutines has been improved.
* Coroutines no longer generate ``StackContexts`` by default, but they
will be created on demand when needed.
+* New function `.with_timeout` wraps a `.Future` and raises an exception
+ if it doesn't complete in a given amount of time.
+
+`tornado.http1connection`
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+* New module contains the HTTP implementation shared by `tornado.httpserver`
+ and ``tornado.simple_httpclient``.
+
+`tornado.httpclient`
+~~~~~~~~~~~~~~~~~~~~
+
+* The command-line HTTP client (``python -m tornado.httpclient $URL``)
+ now works on Python 3.
+
+`tornado.httpserver`
+~~~~~~~~~~~~~~~~~~~~
+
+* ``tornado.httpserver.HTTPRequest`` has moved to
+ `tornado.httputil.HTTPServerRequest`.
+* HTTP implementation has been unified with ``tornado.simple_httpclient``
+ in `tornado.http1connection`.
+* Now supports ``Transfer-Encoding: chunked`` for request bodies.
+* Now supports ``Content-Encoding: gzip`` for request bodies if ``gzip=True``
+ is passed to the `.HTTPServer` constructor.
+* The ``connection`` attribute of `.HTTPServerRequest` is now documented
+ for public use; applications are expected to write their responses
+ via the `.HTTPConnection` interface.
+* The `.HTTPServerRequest.write` and `.HTTPServerRequest.finish` methods
+ are now deprecated.
+* `.HTTPServer` now supports `.HTTPServerConnectionDelegate` in addition to
+ the old ``request_callback`` interface. The delegate interface supports
+ streaming of request bodies.
+* `.HTTPServer` now detects the error of an application sending a
+ ``Content-Length`` error that is inconsistent with the actual content.
+* New constructor arguments ``max_header_size`` and ``max_body_size``
+ allow separate limits to be set for different parts of the request.
+ ``max_body_size`` is applied even in streaming mode.
+* New constructor argument ``chunk_size`` can be used to limit the amount
+ of data read into memory at one time per request.
+* New constructor arguments ``idle_connection_timeout`` and ``body_timeout``
+ allow time limits to be placed on the reading of requests.
+
+`tornado.httputil`
+~~~~~~~~~~~~~~~~~~
+
+* `.HTTPServerRequest` was moved to this module from `tornado.httpserver`.
+* New base classes `.HTTPConnection`, `.HTTPServerConnectionDelegate`,
+ and `.HTTPMessageDelegate` define the interaction between applications
+ and the HTTP implementation.
+
`tornado.ioloop`
~~~~~~~~~~~~~~~~
(when possible) to avoid a garbage-collection-related problem in unit tests.
* New method `.IOLoop.clear_instance` makes it possible to uninstall the
singleton instance.
+* `.IOLoop.add_timeout` is now a bit more efficient.
`tornado.iostream`
~~~~~~~~~~~~~~~~~~
for use with coroutines.
* No longer gets confused when an ``IOError`` or ``OSError`` without
an ``errno`` attribute is raised.
+* `.BaseIOStream.read_bytes` now accepts a ``partial`` keyword argument,
+ which can be used to return before the full amount has been read.
+ This is a more coroutine-friendly alternative to ``streaming_callback``.
+* `.BaseIOStream.read_until` and ``read_until_regex`` now acept a
+ ``max_bytes`` keyword argument which will cause the request to fail if
+ it cannot be satisfied from the given number of bytes.
+* `.IOStream` no longer reads from the socket into memory if it does not
+ need data to satisfy a pending read. As a side effect, the close callback
+ will not be run immediately if the other side closes the connection
+ while there is unconsumed data in the buffer.
+* The default ``chunk_size`` has been increased to 64KB (from 4KB)
`tornado.netutil`
~~~~~~~~~~~~~~~~~
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Improved default cipher suite selection (Python 2.7+).
-
+* HTTP implementation has been unified with ``tornado.httpserver``
+ in `tornado.http1connection`
+* Streaming request bodies are now supported via the ``body_producer``
+ keyword argument to `tornado.httpclient.HTTPRequest`.
+* The ``expect_100_continue`` keyword argument to
+ `tornado.httpclient.HTTPRequest` allows the use of the HTTP ``Expect:
+ 100-continue`` feature.
`tornado.stack_context`
~~~~~~~~~~~~~~~~~~~~~~~
* When gzip support is enabled, all ``text/*`` mime types will be compressed,
not just those on a whitelist.
+* `.Application` now implements the `.HTTPMessageDelegate` interface.
+* It is now possible to support streaming request bodies with the
+ `.stream_request_body` decorator and the new `.RequestHandler.data_received`
+ method.
+* `.RequestHandler.flush` now returns a `.Future` if no callback is given.
`tornado.websocket`
~~~~~~~~~~~~~~~~~~~
messages larger than 2GB on 64-bit systems.
* The fallback mechanism for detecting a missing C compiler now
works correctly on Mac OS X.
+* Arguments to `.WebSocketHandler.open` are now decoded in the same way
+ as arguments to `.RequestHandler.get` and similar methods.
+
+`tornado.wsgi`
+~~~~~~~~~~~~~~
+
+* New class `.WSGIAdapter` supports running a Tornado `.Application` on
+ a WSGI server in a way that is more compatible with Tornado's non-WSGI
+ `.HTTPServer`. `.WSGIApplication` is deprecated in favor of using
+ `.WSGIAdapter` with a regular `.Application`.
+* `.WSGIAdapter` now supports gzipped output.
.. automethod:: RequestHandler.send_error
.. automethod:: RequestHandler.write_error
.. automethod:: RequestHandler.clear
+ .. automethod:: RequestHandler.data_received
Cookies
.. autofunction:: authenticated
.. autofunction:: addslash
.. autofunction:: removeslash
+ .. autofunction:: stream_request_body
Everything else
---------------
relative to `.IOLoop.time`)
Currently only supports Futures, not other `YieldPoint` classes.
+
+ .. versionadded:: 3.3
"""
# TODO: allow yield points in addition to futures?
# Tricky to do with stack_context semantics.
# License for the specific language governing permissions and limitations
# under the License.
+"""Client and server implementations of HTTP/1.x.
+
+.. versionadded:: 3.3
+"""
+
from __future__ import absolute_import, division, print_function, with_statement
from tornado.concurrent import Future
from tornado.util import GzipDecompressor
class HTTP1ConnectionParameters(object):
+ """Parameters for `.HTTP1Connection` and `.HTTP1ServerConnection`.
+ """
def __init__(self, no_keep_alive=False, protocol=None, chunk_size=None,
max_header_size=None, header_timeout=None, max_body_size=None,
body_timeout=None, use_gzip=False):
+ """
+ :arg bool no_keep_alive: If true, always close the connection after
+ one request.
+ :arg str protocol: "http" or "https"
+ :arg int chunk_size: how much data to read into memory at once
+ :arg int max_header_size: maximum amount of data for HTTP headers
+ :arg float header_timeout: how long to wait for all headers (seconds)
+ :arg int max_body_size: maximum amount of data for body
+ :arg float body_timeout: how long to wait while reading body (seconds)
+ :arg bool use_gzip: if true, decode incoming ``Content-Encoding: gzip``
+ """
self.no_keep_alive = no_keep_alive
self.protocol = protocol
self.chunk_size = chunk_size or 65536
self.use_gzip = use_gzip
class HTTP1Connection(object):
- """Handles a connection to an HTTP client, executing HTTP requests.
+ """Implements the HTTP/1.x protocol.
- We parse HTTP headers and bodies, and execute the request callback
- until the HTTP conection is closed.
+ This class can be on its own for clients, or via `HTTP1ServerConnection`
+ for servers.
"""
def __init__(self, stream, is_client, params=None, context=None):
+ """
+ :arg stream: an `.IOStream`
+ :arg bool is_client: client or server
+ :arg params: a `.HTTP1ConnectionParameters` instance or ``None``
+ :arg context: an opaque application-defined object that can be accessed
+ as ``connection.context``.
+ """
self.is_client = is_client
self.stream = stream
if params is None:
self._expected_content_remaining = None
def read_response(self, delegate):
+ """Read a single HTTP response.
+
+ Typical client-mode usage is to write a request using `write_headers`,
+ `write`, and `finish`, and then call ``read_response``.
+
+ :arg delegate: a `.HTTPMessageDelegate`
+
+ Returns a `.Future` that resolves to None after the full response has
+ been read.
+ """
if self.params.use_gzip:
delegate = _GzipMessageDelegate(delegate, self.params.chunk_size)
return self._read_message(delegate)
def set_close_callback(self, callback):
"""Sets a callback that will be run when the connection is closed.
- Use this instead of accessing
- `HTTPConnection.stream.set_close_callback
- <.BaseIOStream.set_close_callback>` directly (which was the
- recommended approach prior to Tornado 3.0).
+ .. deprecated:: 3.3
+ Use `.HTTPMessageDelegate.on_connection_close` instead.
"""
self._close_callback = stack_context.wrap(callback)
self._clear_callbacks()
def detach(self):
+ """Take control of the underlying stream.
+
+ Returns the underlying `.IOStream` object and stops all further
+ HTTP processing. May only be called during
+ `.HTTPMessageDelegate.headers_received`. Intended for implementing
+ protocols like websockets that tunnel over an HTTP handshake.
+ """
stream = self.stream
self.stream = None
return stream
def set_body_timeout(self, timeout):
+ """Sets the body timeout for a single request.
+
+ Overrides the value from `.HTTP1ConnectionParameters`.
+ """
self._body_timeout = timeout
def set_max_body_size(self, max_body_size):
+ """Sets the body size limit for a single request.
+
+ Overrides the value from `.HTTP1ConnectionParameters`.
+ """
self._max_body_size = max_body_size
def write_headers(self, start_line, headers, chunk=None, callback=None,
has_body=True):
+ """Implements `.HTTPConnection.write_headers`."""
if self.is_client:
self._request_start_line = start_line
# Client requests with a non-empty body must have either a
return chunk
def write(self, chunk, callback=None):
- """Writes a chunk of output to the stream."""
+ """Implements `.HTTPConnection.write`."""
if self.stream.closed():
self._write_future = Future()
self._write_future.set_exception(iostream.StreamClosedError())
return self._write_future
def finish(self):
- """Finishes the request."""
+ """Implements `.HTTPConnection.finish`."""
if (self._expected_content_remaining is not None and
self._expected_content_remaining != 0 and
not self.stream.closed()):
class HTTP1ServerConnection(object):
+ """An HTTP/1.x server."""
def __init__(self, stream, params=None, context=None):
+ """
+ :arg stream: an `.IOStream`
+ :arg params: a `.HTTP1ConnectionParameters` or None
+ :arg context: an opaque application-defined object that is accessible
+ as ``connection.context``
+ """
self.stream = stream
if params is None:
params = HTTP1ConnectionParameters()
@gen.coroutine
def close(self):
+ """Closes the connection.
+
+ Returns a `.Future` that resolves after the serving loop has exited.
+ """
self.stream.close()
# Block until the serving loop is done, but ignore any exceptions
# (start_serving is already responsible for logging them).
pass
def start_serving(self, delegate):
+ """Starts serving requests on this connection.
+
+ :arg delegate: a `.HTTPServerConnectionDelegate`
+ """
assert isinstance(delegate, httputil.HTTPServerConnectionDelegate)
self._serving_future = self._server_request_loop(delegate)
# Register the future on the IOLoop so its errors get logged.
:arg body: HTTP request body as a string (byte or unicode; if unicode
the utf-8 encoding will be used)
:arg body_producer: Callable used for lazy/asynchronous request bodies.
- TODO: document the interface.
+ It is called with one argument, a ``write`` function, and should
+ return a `.Future`. It should call the write function with new
+ data as it becomes available. The write function returns a
+ `.Future` which can be used for flow control.
Only one of ``body`` and ``body_producer`` may
be specified. ``body_producer`` is not supported on
``curl_httpclient``. When using ``body_producer`` it is recommended
to pass a ``Content-Length`` in the headers as otherwise chunked
encoding will be used, and many servers do not support chunked
- encoding on requests.
+ encoding on requests. New in Tornado 3.3
:arg string auth_username: Username for HTTP authentication
:arg string auth_password: Password for HTTP authentication
:arg string auth_mode: Authentication mode; default is "basic".
.. versionadded:: 3.1
The ``auth_mode`` argument.
+
+ .. versionadded:: 3.3
+ The ``body_producer`` and ``expect_100_continue`` arguments.
"""
# Note that some of these attributes go through property setters
# defined below.
class HTTPServer(TCPServer, httputil.HTTPServerConnectionDelegate):
r"""A non-blocking, single-threaded HTTP server.
- A server is defined by a request callback that takes an HTTPRequest
- instance as an argument and writes a valid HTTP response with
- `.HTTPServerRequest.write`. `.HTTPServerRequest.finish` finishes the request (but does
- not necessarily close the connection in the case of HTTP/1.1 keep-alive
- requests). A simple example server that echoes back the URI you
- requested::
+ A server is defined by either a request callback that takes a
+ `.HTTPServerRequest` as an argument or a `.HTTPServerConnectionDelegate`
+ instance.
+
+ A simple example server that echoes back the URI you requested::
import tornado.httpserver
import tornado.ioloop
def handle_request(request):
message = "You requested %s\n" % request.uri
- request.write("HTTP/1.1 200 OK\r\nContent-Length: %d\r\n\r\n%s" % (
- len(message), message))
- request.finish()
+ request.connection.write_headers(
+ httputil.ResponseStartLine('HTTP/1.1', 200, 'OK'),
+ {"Content-Length": str(len(message))})
+ request.connection.write(message)
+ request.connection.finish()
http_server = tornado.httpserver.HTTPServer(handle_request)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()
- `HTTPServer` is a very basic connection handler. It parses the request
- headers and body, but the request callback is responsible for producing
- the response exactly as it will appear on the wire. This affords
- maximum flexibility for applications to implement whatever parts
- of HTTP responses are required.
+ Applications should use the methods of `.HTTPConnection` to write
+ their response.
`HTTPServer` supports keep-alive connections by default
(automatically for HTTP/1.1, or for HTTP/1.0 when the client
- requests ``Connection: keep-alive``). This means that the request
- callback must generate a properly-framed response, using either
- the ``Content-Length`` header or ``Transfer-Encoding: chunked``.
- Applications that are unable to frame their responses properly
- should instead return a ``Connection: close`` header in each
- response and pass ``no_keep_alive=True`` to the `HTTPServer`
- constructor.
+ requests ``Connection: keep-alive``).
If ``xheaders`` is ``True``, we support the
``X-Real-Ip``/``X-Forwarded-For`` and
servers if you want to create your listening sockets in some
way other than `tornado.netutil.bind_sockets`.
+ .. versionchanged:: 3.3
+ Added ``gzip``, ``chunk_size``, ``max_header_size``,
+ ``idle_connection_timeout``, ``body_timeout``, ``max_body_size``
+ arguments. Added support for `.HTTPServerConnectionDelegate`
+ instances as ``request_callback``.
"""
def __init__(self, request_callback, no_keep_alive=False, io_loop=None,
xheaders=False, ssl_options=None, protocol=None, gzip=False,
be accessed through the "connection" attribute. Since connections
are typically kept open in HTTP/1.1, multiple requests can be handled
sequentially on a single connection.
+
+ .. versionchanged:: 3.3
+ Moved from ``tornado.httpserver.HTTPRequest``.
"""
def __init__(self, method=None, uri=None, version="HTTP/1.0", headers=None,
body=None, host=None, files=None, connection=None,
self.body_arguments = {}
def supports_http_1_1(self):
- """Returns True if this request supports HTTP/1.1 semantics"""
+ """Returns True if this request supports HTTP/1.1 semantics.
+
+ .. deprecated:: 3.3
+ Applications are less likely to need this information with the
+ introduction of `.HTTPConnection`. If you still need it, access
+ the ``version`` attribute directly.
+ """
return self.version == "HTTP/1.1"
@property
return self._cookies
def write(self, chunk, callback=None):
- """Writes the given chunk to the response stream."""
+ """Writes the given chunk to the response stream.
+
+ .. deprecated:: 3.3
+ Use ``request.connection`` and the `.HTTPConnection` methods
+ to write the response.
+ """
assert isinstance(chunk, bytes_type)
self.connection.write(chunk, callback=callback)
def finish(self):
- """Finishes this HTTP request on the open connection."""
+ """Finishes this HTTP request on the open connection.
+
+ .. deprecated:: 3.3
+ Use ``request.connection`` and the `.HTTPConnection` methods
+ to write the response.
+ """
self.connection.finish()
self._finish_time = time.time()
class HTTPInputException(Exception):
"""Exception class for malformed HTTP requests or responses
from remote sources.
+
+ .. versionadded:: 3.3
"""
pass
class HTTPOutputException(Exception):
- """Exception class for errors in HTTP output."""
+ """Exception class for errors in HTTP output.
+
+ .. versionadded:: 3.3
+ """
pass
class HTTPServerConnectionDelegate(object):
+ """Implement this interface to handle requests from `.HTTPServer`.
+
+ .. versionadded:: 3.3
+ """
def start_request(self, server_conn, request_conn):
+ """This method is called by the server when a new request has started.
+
+ :arg server_conn: is an opaque object representing the long-lived
+ (e.g. tcp-level) connection.
+ :arg request_conn: is a `.HTTPConnection` object for a single
+ request/response exchange.
+
+ This method should return a `.HTTPMessageDelegate`.
+ """
raise NotImplementedError()
def on_close(self, server_conn):
+ """This method is called when a connection has been closed.
+
+ :arg server_conn: is a server connection that has previously been
+ passed to ``start_request``.
+ """
pass
class HTTPMessageDelegate(object):
+ """Implement this interface to handle an HTTP request or response.
+
+ .. versionadded:: 3.3
+ """
def headers_received(self, start_line, headers):
+ """Called when the HTTP headers have been received and parsed.
+
+ :arg start_line: a `.RequestStartLine` or `.ResponseStartLine`
+ depending on whether this is a client or server message.
+ :arg headers: a `.HTTPHeaders` instance.
+
+ Some `.HTTPConnection` methods can only be called during
+ ``headers_received``.
+
+ May return a `.Future`; if it does the body will not be read
+ until it is done.
+ """
pass
def data_received(self, chunk):
+ """Called when a chunk of data has been received.
+
+ May return a `.Future` for flow control.
+ """
pass
def finish(self):
+ """Called after the last chunk of data has been received."""
pass
def on_connection_close(self):
+ """Called if the connection is closed without finishing the request.
+
+ If ``headers_received`` is called, either ``finish`` or
+ ``on_connection_close`` will be called, but not both.
+ """
pass
+class HTTPConnection(object):
+ """Applications use this interface to write their responses.
+
+ .. versionadded:: 3.3
+ """
+ def write_headers(self, start_line, headers, chunk=None, callback=None,
+ has_body=True):
+ """Write an HTTP header block.
+
+ :arg start_line: a `.RequestStartLine` or `.ResponseStartLine`.
+ :arg headers: a `.HTTPHeaders` instance.
+ :arg chunk: the first (optional) chunk of data. This is an optimization
+ so that small responses can be written in the same call as their
+ headers.
+ :arg callback: a callback to be run when the write is complete.
+ :arg has_body: as an optimization, may be ``False`` to indicate
+ that no further writes will be coming.
+
+ Returns a `.Future` if no callback is given.
+ """
+ raise NotImplementedError()
+
+ def write(self, chunk, callback=None):
+ """Writes a chunk of body data.
+
+ The callback will be run when the write is complete. If no callback
+ is given, returns a Future.
+ """
+ raise NotImplementedError()
+
+ def finish(self):
+ """Indicates that the last body data has been written.
+ """
+ raise NotImplementedError()
+
+
def url_concat(url, args):
"""Concatenate url and argument dictionary regardless of whether
url has existing query parameters.
@staticmethod
def clear_instance():
- """Clear the global `IOLoop` instance."""
+ """Clear the global `IOLoop` instance.
+
+ .. versionadded:: 3.3
+ """
if hasattr(IOLoop, "_instance"):
del IOLoop._instance
Note that only one flush callback can be outstanding at a time;
if another flush occurs before the previous flush's callback
has been run, the previous callback will be discarded.
+
+ .. versionchanged:: 3.3
+ Now returns a `.Future` if no callback is given.
"""
chunk = b"".join(self._write_buffer)
self._write_buffer = []
# in a finally block to avoid GC issues prior to Python 3.4.
self._prepared_future.set_result(None)
+ def data_received(self, chunk):
+ """Implement this method to handle streamed request data.
+
+ Requires the `.stream_request_body` decorator.
+ """
+ raise NotImplementedError()
+
def _log(self):
"""Logs the current request.
"""Apply to `RequestHandler` subclasses to enable streaming body support.
This decorator implies the following changes:
+
* `.HTTPServerRequest.body` is undefined, and body arguments will not
be included in `RequestHandler.get_argument`.
* `RequestHandler.prepare` is called when the request headers have been
that it is not possible to use `.AsyncHTTPClient`, or the
`tornado.auth` or `tornado.websocket` modules.
+ .. versionadded:: 3.3
"""
def __init__(self, application):
if isinstance(application, WSGIApplication):
projects / thirdparty / tornado.git / commitdiff
