>>> f.read().decode('utf-8')
- urlretrieve reads more data, but if less data is available, it raises the
- exception.
+Legacy interface
+----------------
+
+The following functions and classes are ported from the Python 2 module
+``urllib`` (as opposed to ``urllib2``). They might become deprecated at
+some point in the future.
+
+
+.. function:: urlretrieve(url, filename=None, reporthook=None, data=None)
+
+ Copy a network object denoted by a URL to a local file, if necessary. If the URL
+ points to a local file, or a valid cached copy of the object exists, the object
+ is not copied. Return a tuple ``(filename, headers)`` where *filename* is the
+ local file name under which the object can be found, and *headers* is whatever
+ the :meth:`info` method of the object returned by :func:`urlopen` returned (for
+ a remote object, possibly cached). Exceptions are the same as for
+ :func:`urlopen`.
+
+ The second argument, if present, specifies the file location to copy to (if
+ absent, the location will be a tempfile with a generated name). The third
+ argument, if present, is a hook function that will be called once on
+ establishment of the network connection and once after each block read
+ thereafter. The hook will be passed three arguments; a count of blocks
+ transferred so far, a block size in bytes, and the total size of the file. The
+ third argument may be ``-1`` on older FTP servers which do not return a file
+ size in response to a retrieval request.
+
+ If the *url* uses the :file:`http:` scheme identifier, the optional *data*
+ argument may be given to specify a ``POST`` request (normally the request type
+ is ``GET``). The *data* argument must in standard
+ :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
+ function below.
+
+ :func:`urlretrieve` will raise :exc:`ContentTooShortError` when it detects that
+ the amount of data available was less than the expected amount (which is the
+ size reported by a *Content-Length* header). This can occur, for example, when
+ the download is interrupted.
+
+ The *Content-Length* is treated as a lower bound: if there's more data to read,
- If no *Content-Length* header was supplied, urlretrieve can not check the size
- of the data it has downloaded, and just returns it. In this case you just have
- to assume that the download was successful.
++ :func:`urlretrieve` reads more data, but if less data is available, it raises
++ the exception.
+
+ You can still retrieve the downloaded data in this case, it is stored in the
+ :attr:`content` attribute of the exception instance.
+
++ If no *Content-Length* header was supplied, :func:`urlretrieve` can not check
++ the size of the data it has downloaded, and just returns it. In this case
++ you just have to assume that the download was successful.
+
+.. function:: urlcleanup()
+
+ Clear the cache that may have been built up by previous calls to
+ :func:`urlretrieve`.
+
+.. class:: URLopener(proxies=None, **x509)
+
+ Base class for opening and reading URLs. Unless you need to support opening
+ objects using schemes other than :file:`http:`, :file:`ftp:`, or :file:`file:`,
+ you probably want to use :class:`FancyURLopener`.
+
+ By default, the :class:`URLopener` class sends a :mailheader:`User-Agent` header
+ of ``urllib/VVV``, where *VVV* is the :mod:`urllib` version number.
+ Applications can define their own :mailheader:`User-Agent` header by subclassing
+ :class:`URLopener` or :class:`FancyURLopener` and setting the class attribute
+ :attr:`version` to an appropriate string value in the subclass definition.
+
+ The optional *proxies* parameter should be a dictionary mapping scheme names to
+ proxy URLs, where an empty dictionary turns proxies off completely. Its default
+ value is ``None``, in which case environmental proxy settings will be used if
+ present, as discussed in the definition of :func:`urlopen`, above.
+
+ Additional keyword parameters, collected in *x509*, may be used for
+ authentication of the client when using the :file:`https:` scheme. The keywords
+ *key_file* and *cert_file* are supported to provide an SSL key and certificate;
+ both are needed to support client authentication.
+
+ :class:`URLopener` objects will raise an :exc:`IOError` exception if the server
+ returns an error code.
+
+ .. method:: open(fullurl, data=None)
+
+ Open *fullurl* using the appropriate protocol. This method sets up cache and
+ proxy information, then calls the appropriate open method with its input
+ arguments. If the scheme is not recognized, :meth:`open_unknown` is called.
+ The *data* argument has the same meaning as the *data* argument of
+ :func:`urlopen`.
+
+
+ .. method:: open_unknown(fullurl, data=None)
+
+ Overridable interface to open unknown URL types.
+
+
+ .. method:: retrieve(url, filename=None, reporthook=None, data=None)
+
+ Retrieves the contents of *url* and places it in *filename*. The return value
+ is a tuple consisting of a local filename and either a
+ :class:`email.message.Message` object containing the response headers (for remote
+ URLs) or ``None`` (for local URLs). The caller must then open and read the
+ contents of *filename*. If *filename* is not given and the URL refers to a
+ local file, the input filename is returned. If the URL is non-local and
+ *filename* is not given, the filename is the output of :func:`tempfile.mktemp`
+ with a suffix that matches the suffix of the last path component of the input
+ URL. If *reporthook* is given, it must be a function accepting three numeric
+ parameters. It will be called after each chunk of data is read from the
+ network. *reporthook* is ignored for local URLs.
+
+ If the *url* uses the :file:`http:` scheme identifier, the optional *data*
+ argument may be given to specify a ``POST`` request (normally the request type
+ is ``GET``). The *data* argument must in standard
+ :mimetype:`application/x-www-form-urlencoded` format; see the :func:`urlencode`
+ function below.
+
+
+ .. attribute:: version
+
+ Variable that specifies the user agent of the opener object. To get
+ :mod:`urllib` to tell servers that it is a particular user agent, set this in a
+ subclass as a class variable or in the constructor before calling the base
+ constructor.
+
+
+.. class:: FancyURLopener(...)
+
+ :class:`FancyURLopener` subclasses :class:`URLopener` providing default handling
+ for the following HTTP response codes: 301, 302, 303, 307 and 401. For the 30x
+ response codes listed above, the :mailheader:`Location` header is used to fetch
+ the actual URL. For 401 response codes (authentication required), basic HTTP
+ authentication is performed. For the 30x response codes, recursion is bounded
+ by the value of the *maxtries* attribute, which defaults to 10.
+
+ For all other response codes, the method :meth:`http_error_default` is called
+ which you can override in subclasses to handle the error appropriately.
+
+ .. note::
+
+ According to the letter of :rfc:`2616`, 301 and 302 responses to POST requests
+ must not be automatically redirected without confirmation by the user. In
+ reality, browsers do allow automatic redirection of these responses, changing
+ the POST to a GET, and :mod:`urllib` reproduces this behaviour.
+
+ The parameters to the constructor are the same as those for :class:`URLopener`.
+
+ .. note::
+
+ When performing basic authentication, a :class:`FancyURLopener` instance calls
+ its :meth:`prompt_user_passwd` method. The default implementation asks the
+ users for the required information on the controlling terminal. A subclass may
+ override this method to support more appropriate behavior if needed.
+
+ The :class:`FancyURLopener` class offers one additional method that should be
+ overloaded to provide the appropriate behavior:
+
+ .. method:: prompt_user_passwd(host, realm)
+
+ Return information needed to authenticate the user at the given host in the
+ specified security realm. The return value should be a tuple, ``(user,
+ password)``, which can be used for basic authentication.
+
+ The implementation prompts for this information on the terminal; an application
+ should override this method to use an appropriate interaction model in the local
+ environment.
+
+
:mod:`urllib.request` Restrictions
----------------------------------
projects / thirdparty / Python / cpython.git / commitdiff
