===============
The response returned by urlopen (or the :exc:`~urllib.error.HTTPError` instance) has two
-useful methods :meth:`info` and :meth:`geturl` and is defined in the module
-:mod:`urllib.response`..
+useful methods :meth:`!info` and :meth:`!geturl` and is defined in the module
+:mod:`urllib.response`.
-**geturl** - this returns the real URL of the page fetched. This is useful
-because ``urlopen`` (or the opener object used) may have followed a
-redirect. The URL of the page fetched may not be the same as the URL requested.
+* **geturl** - this returns the real URL of the page fetched. This is useful
+ because ``urlopen`` (or the opener object used) may have followed a
+ redirect. The URL of the page fetched may not be the same as the URL requested.
-**info** - this returns a dictionary-like object that describes the page
-fetched, particularly the headers sent by the server. It is currently an
-:class:`http.client.HTTPMessage` instance.
+* **info** - this returns a dictionary-like object that describes the page
+ fetched, particularly the headers sent by the server. It is currently an
+ :class:`http.client.HTTPMessage` instance.
Typical headers include 'Content-length', 'Content-type', and so on. See the
`Quick Reference to HTTP Headers <https://jkorpela.fi/http.html>`_
In the above example we only supplied our ``HTTPBasicAuthHandler`` to
``build_opener``. By default openers have the handlers for normal situations
- -- ``ProxyHandler`` (if a proxy setting such as an :envvar:`http_proxy`
+ -- ``ProxyHandler`` (if a proxy setting such as an :envvar:`!http_proxy`
environment variable is set), ``UnknownHandler``, ``HTTPHandler``,
``HTTPDefaultErrorHandler``, ``HTTPRedirectHandler``, ``FTPHandler``,
``FileHandler``, ``DataHandler``, ``HTTPErrorProcessor``.
:class:`UnknownHandler` to ensure this never happens).
In addition, if proxy settings are detected (for example, when a ``*_proxy``
- environment variable like :envvar:`http_proxy` is set),
+ environment variable like :envvar:`!http_proxy` is set),
:class:`ProxyHandler` is default installed and makes sure the requests are
handled through the proxy.
.. versionchanged:: 3.10
HTTPS connection now send an ALPN extension with protocol indicator
``http/1.1`` when no *context* is given. Custom *context* should set
- ALPN protocols with :meth:`~ssl.SSLContext.set_alpn_protocol`.
+ ALPN protocols with :meth:`~ssl.SSLContext.set_alpn_protocols`.
.. deprecated:: 3.6
the actual HTTP code, for example :meth:`http_error_404` would handle HTTP
404 errors.
- * :meth:`<protocol>_open` --- signal that the handler knows how to open *protocol*
+ * :meth:`!<protocol>_open` --- signal that the handler knows how to open *protocol*
URLs.
See |protocol_open|_ for more information.
- * :meth:`http_error_\<type\>` --- signal that the handler knows how to handle HTTP
+ * :meth:`!http_error_\<type\>` --- signal that the handler knows how to handle HTTP
errors with HTTP error code *type*.
See |http_error_nnn|_ for more information.
- * :meth:`<protocol>_error` --- signal that the handler knows how to handle errors
+ * :meth:`!<protocol>_error` --- signal that the handler knows how to handle errors
from (non-\ ``http``) *protocol*.
- * :meth:`<protocol>_request` --- signal that the handler knows how to pre-process
+ * :meth:`!<protocol>_request` --- signal that the handler knows how to pre-process
*protocol* requests.
See |protocol_request|_ for more information.
- * :meth:`<protocol>_response` --- signal that the handler knows how to
+ * :meth:`!<protocol>_response` --- signal that the handler knows how to
post-process *protocol* responses.
See |protocol_response|_ for more information.
Handle an error of the given protocol. This will call the registered error
handlers for the given protocol with the given arguments (which are protocol
specific). The HTTP protocol is a special case which uses the HTTP response
- code to determine the specific error handler; refer to the :meth:`http_error_\<type\>`
+ code to determine the specific error handler; refer to the :meth:`!http_error_\<type\>`
methods of the handler classes.
Return values and exceptions raised are the same as those of :func:`urlopen`.
The order in which these methods are called within each stage is determined by
sorting the handler instances.
-#. Every handler with a method named like :meth:`<protocol>_request` has that
+#. Every handler with a method named like :meth:`!<protocol>_request` has that
method called to pre-process the request.
-#. Handlers with a method named like :meth:`<protocol>_open` are called to handle
+#. Handlers with a method named like :meth:`!<protocol>_open` are called to handle
the request. This stage ends when a handler either returns a non-\ :const:`None`
value (ie. a response), or raises an exception (usually
:exc:`~urllib.error.URLError`). Exceptions are allowed to propagate.
In fact, the above algorithm is first tried for methods named
- :meth:`default_open`. If all such methods return :const:`None`, the algorithm
- is repeated for methods named like :meth:`<protocol>_open`. If all such methods
+ :meth:`~BaseHandler.default_open`. If all such methods return :const:`None`, the algorithm
+ is repeated for methods named like :meth:`!<protocol>_open`. If all such methods
return :const:`None`, the algorithm is repeated for methods named
- :meth:`unknown_open`.
+ :meth:`~BaseHandler.unknown_open`.
Note that the implementation of these methods may involve calls of the parent
:class:`OpenerDirector` instance's :meth:`~OpenerDirector.open` and
:meth:`~OpenerDirector.error` methods.
-#. Every handler with a method named like :meth:`<protocol>_response` has that
+#. Every handler with a method named like :meth:`!<protocol>_response` has that
method called to post-process the response.
the return value of the :meth:`~OpenerDirector.open` method of :class:`OpenerDirector`, or ``None``.
It should raise :exc:`~urllib.error.URLError`, unless a truly exceptional
thing happens (for example, :exc:`MemoryError` should not be mapped to
- :exc:`URLError`).
+ :exc:`~urllib.error.URLError`).
This method will be called before any protocol-specific open method.
define it if they want to handle URLs with the given protocol.
This method, if defined, will be called by the parent :class:`OpenerDirector`.
- Return values should be the same as for :meth:`default_open`.
+ Return values should be the same as for :meth:`~BaseHandler.default_open`.
.. method:: BaseHandler.unknown_open(req)
Subclasses should override this method to handle specific HTTP errors.
Arguments, return values and exceptions raised should be the same as for
- :meth:`http_error_default`.
+ :meth:`~BaseHandler.http_error_default`.
.. _protocol_request:
is the case, :exc:`~urllib.error.HTTPError` is raised. See :rfc:`2616` for
details of the precise meanings of the various redirection codes.
- An :class:`HTTPError` exception raised as a security consideration if the
+ An :exc:`~urllib.error.HTTPError` exception raised as a security consideration if the
HTTPRedirectHandler is presented with a redirected URL which is not an HTTP,
HTTPS or FTP URL.
.. method:: ProxyHandler.<protocol>_open(request)
:noindex:
- The :class:`ProxyHandler` will have a method :meth:`<protocol>_open` for every
+ The :class:`ProxyHandler` will have a method :meth:`!<protocol>_open` for every
*protocol* which has a proxy in the *proxies* dictionary given in the
constructor. The method will modify requests to go through the proxy, by
calling ``request.set_proxy()``, and call the next handler in the chain to
For 200 error codes, the response object is returned immediately.
For non-200 error codes, this simply passes the job on to the
- :meth:`http_error_\<type\>` handler methods, via :meth:`OpenerDirector.error`.
+ :meth:`!http_error_\<type\>` handler methods, via :meth:`OpenerDirector.error`.
Eventually, :class:`HTTPDefaultErrorHandler` will raise an
:exc:`~urllib.error.HTTPError` if no other handler handles the error.
:func:`build_opener` provides many handlers by default, including a
:class:`ProxyHandler`. By default, :class:`ProxyHandler` uses the environment
variables named ``<scheme>_proxy``, where ``<scheme>`` is the URL scheme
-involved. For example, the :envvar:`http_proxy` environment variable is read to
+involved. For example, the :envvar:`!http_proxy` environment variable is read to
obtain the HTTP proxy's URL.
This example replaces the default :class:`ProxyHandler` with one that uses
points to a local file, the object will not be copied unless filename is supplied.
Return a tuple ``(filename, headers)`` where *filename* is the
local file name under which the object can be found, and *headers* is whatever
- the :meth:`info` method of the object returned by :func:`urlopen` returned (for
+ the :meth:`!info` method of the object returned by :func:`urlopen` returned (for
a remote object). Exceptions are the same as for :func:`urlopen`.
The second argument, if present, specifies the file location to copy to (if
:mimetype:`application/x-www-form-urlencoded` format; see the
:func:`urllib.parse.urlencode` function.
- :func:`urlretrieve` will raise :exc:`ContentTooShortError` when it detects that
+ :func:`urlretrieve` will raise :exc:`~urllib.error.ContentTooShortError` when it detects that
the amount of data available was less than the expected amount (which is the
size reported by a *Content-Length* header). This can occur, for example, when
the download is interrupted.
urlretrieve reads more data, but if less data is available, it raises the
exception.
- You can still retrieve the downloaded data in this case, it is stored in the
- :attr:`content` attribute of the exception instance.
+ You can still retrieve the downloaded data in this case, it is stored in the
+ :attr:`!content` attribute of the exception instance.
If no *Content-Length* header was supplied, urlretrieve can not check the size
of the data it has downloaded, and just returns it. In this case you just have
authentication is performed. For the 30x response codes, recursion is bounded
by the value of the *maxtries* attribute, which defaults to 10.
- For all other response codes, the method :meth:`http_error_default` is called
+ For all other response codes, the method :meth:`~BaseHandler.http_error_default` is called
which you can override in subclasses to handle the error appropriately.
.. note::