:synopsis: Implementation of the ElementTree API.
.. moduleauthor:: Fredrik Lundh <fredrik@pythonware.com>
-**Source code:** :source:`Lib/xml/etree/ElementTree.py`
+The :mod:`xml.etree.ElementTree` module implements a simple and efficient API
+for parsing and creating XML data.
---------------
-
-The :class:`Element` type is a flexible container object, designed to store
-hierarchical data structures in memory. The type can be described as a cross
-between a list and a dictionary.
+.. versionchanged:: 3.3
+ This module will use a fast implementation whenever available.
+ The :mod:`xml.etree.cElementTree` module is deprecated.
+
+ .. warning::
+
+ The :mod:`xml.etree.ElementTree` module is not secure against
+ maliciously constructed data. If you need to parse untrusted or
+ unauthenticated data see :ref:`xml-vulnerabilities`.
+
+Tutorial
+--------
+
+This is a short tutorial for using :mod:`xml.etree.ElementTree` (``ET`` in
+short). The goal is to demonstrate some of the building blocks and basic
+concepts of the module.
+
+XML tree and elements
+^^^^^^^^^^^^^^^^^^^^^
+
+XML is an inherently hierarchical data format, and the most natural way to
+represent it is with a tree. ``ET`` has two classes for this purpose -
+:class:`ElementTree` represents the whole XML document as a tree, and
+:class:`Element` represents a single node in this tree. Interactions with
+the whole document (reading and writing to/from files) are usually done
+on the :class:`ElementTree` level. Interactions with a single XML element
+and its sub-elements are done on the :class:`Element` level.
+
+.. _elementtree-parsing-xml:
+
+Parsing XML
+^^^^^^^^^^^
+
+We'll be using the following XML document as the sample data for this section:
+
+.. code-block:: xml
+
+ <?xml version="1.0"?>
+ <data>
+ <country name="Liechtenstein">
+ <rank>1</rank>
+ <year>2008</year>
+ <gdppc>141100</gdppc>
+ <neighbor name="Austria" direction="E"/>
+ <neighbor name="Switzerland" direction="W"/>
+ </country>
+ <country name="Singapore">
+ <rank>4</rank>
+ <year>2011</year>
+ <gdppc>59900</gdppc>
+ <neighbor name="Malaysia" direction="N"/>
+ </country>
+ <country name="Panama">
+ <rank>68</rank>
+ <year>2011</year>
+ <gdppc>13600</gdppc>
+ <neighbor name="Costa Rica" direction="W"/>
+ <neighbor name="Colombia" direction="E"/>
+ </country>
+ </data>
+
+We can import this data by reading from a file::
+
+ import xml.etree.ElementTree as ET
+ tree = ET.parse('country_data.xml')
+ root = tree.getroot()
+
+Or directly from a string::
+
+ root = ET.fromstring(country_data_as_string)
+
+:func:`fromstring` parses XML from a string directly into an :class:`Element`,
+which is the root element of the parsed tree. Other parsing functions may
+create an :class:`ElementTree`. Check the documentation to be sure.
+
+As an :class:`Element`, ``root`` has a tag and a dictionary of attributes::
+
+ >>> root.tag
+ 'data'
+ >>> root.attrib
+ {}
+
+It also has children nodes over which we can iterate::
+
+ >>> for child in root:
+ ... print(child.tag, child.attrib)
+ ...
+ country {'name': 'Liechtenstein'}
+ country {'name': 'Singapore'}
+ country {'name': 'Panama'}
+
+Children are nested, and we can access specific child nodes by index::
+
+ >>> root[0][1].text
+ '2008'
+
+Finding interesting elements
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+:class:`Element` has some useful methods that help iterate recursively over all
+the sub-tree below it (its children, their children, and so on). For example,
+:meth:`Element.iter`::
+
+ >>> for neighbor in root.iter('neighbor'):
+ ... print(neighbor.attrib)
+ ...
+ {'name': 'Austria', 'direction': 'E'}
+ {'name': 'Switzerland', 'direction': 'W'}
+ {'name': 'Malaysia', 'direction': 'N'}
+ {'name': 'Costa Rica', 'direction': 'W'}
+ {'name': 'Colombia', 'direction': 'E'}
+
+:meth:`Element.findall` finds only elements with a tag which are direct
+children of the current element. :meth:`Element.find` finds the *first* child
+with a particular tag, and :meth:`Element.text` accesses the element's text
+content. :meth:`Element.get` accesses the element's attributes::
+
+ >>> for country in root.findall('country'):
+ ... rank = country.find('rank').text
+ ... name = country.get('name')
+ ... print(name, rank)
+ ...
+ Liechtenstein 1
+ Singapore 4
+ Panama 68
+
+More sophisticated specification of which elements to look for is possible by
+using :ref:`XPath <elementtree-xpath>`.
+
+Modifying an XML File
+^^^^^^^^^^^^^^^^^^^^^
+
+:class:`ElementTree` provides a simple way to build XML documents and write them to files.
+The :meth:`ElementTree.write` method serves this purpose.
+
+Once created, an :class:`Element` object may be manipulated by directly changing
+its fields (such as :attr:`Element.text`), adding and modifying attributes
+(:meth:`Element.set` method), as well as adding new children (for example
+with :meth:`Element.append`).
+
+Let's say we want to add one to each country's rank, and add an ``updated``
+attribute to the rank element::
+
+ >>> for rank in root.iter('rank'):
+ ... new_rank = int(rank.text) + 1
+ ... rank.text = str(new_rank)
+ ... rank.set('updated', 'yes')
+ ...
+ >>> tree.write('output.xml')
+
+Our XML now looks like this:
+
+.. code-block:: xml
+
+ <?xml version="1.0"?>
+ <data>
+ <country name="Liechtenstein">
+ <rank updated="yes">2</rank>
+ <year>2008</year>
+ <gdppc>141100</gdppc>
+ <neighbor name="Austria" direction="E"/>
+ <neighbor name="Switzerland" direction="W"/>
+ </country>
+ <country name="Singapore">
+ <rank updated="yes">5</rank>
+ <year>2011</year>
+ <gdppc>59900</gdppc>
+ <neighbor name="Malaysia" direction="N"/>
+ </country>
+ <country name="Panama">
+ <rank updated="yes">69</rank>
+ <year>2011</year>
+ <gdppc>13600</gdppc>
+ <neighbor name="Costa Rica" direction="W"/>
+ <neighbor name="Colombia" direction="E"/>
+ </country>
+ </data>
+
+We can remove elements using :meth:`Element.remove`. Let's say we want to
+remove all countries with a rank higher than 50::
+
+ >>> for country in root.findall('country'):
+ ... rank = int(country.find('rank').text)
+ ... if rank > 50:
+ ... root.remove(country)
+ ...
+ >>> tree.write('output.xml')
+
+Our XML now looks like this:
+
+.. code-block:: xml
+
+ <?xml version="1.0"?>
+ <data>
+ <country name="Liechtenstein">
+ <rank updated="yes">2</rank>
+ <year>2008</year>
+ <gdppc>141100</gdppc>
+ <neighbor name="Austria" direction="E"/>
+ <neighbor name="Switzerland" direction="W"/>
+ </country>
+ <country name="Singapore">
+ <rank updated="yes">5</rank>
+ <year>2011</year>
+ <gdppc>59900</gdppc>
+ <neighbor name="Malaysia" direction="N"/>
+ </country>
+ </data>
+
+Building XML documents
+^^^^^^^^^^^^^^^^^^^^^^
+
+The :func:`SubElement` function also provides a convenient way to create new
+sub-elements for a given element::
+
+ >>> a = ET.Element('a')
+ >>> b = ET.SubElement(a, 'b')
+ >>> c = ET.SubElement(a, 'c')
+ >>> d = ET.SubElement(c, 'd')
+ >>> ET.dump(a)
+ <a><b /><c><d /></c></a>
+
+Additional resources
+^^^^^^^^^^^^^^^^^^^^
-Each element has a number of properties associated with it:
-
-* a tag which is a string identifying what kind of data this element represents
- (the element type, in other words).
-
-* a number of attributes, stored in a Python dictionary.
-
-* a text string.
-
-* an optional tail string.
-
-* a number of child elements, stored in a Python sequence
-
-To create an element instance, use the :class:`Element` constructor or the
-:func:`SubElement` factory function.
-
-The :class:`ElementTree` class can be used to wrap an element structure, and
-convert it from and to XML.
+See http://effbot.org/zone/element-index.htm for tutorials and links to other
+docs.
-A C implementation of this API is available as :mod:`xml.etree.cElementTree`.
-See http://effbot.org/zone/element-index.htm for tutorials and links to other
-docs. Fredrik Lundh's page is also the location of the development version of
-the xml.etree.ElementTree.
+.. _elementtree-xpath:
-.. versionchanged:: 3.2
- The ElementTree API is updated to 1.3. For more information, see
- `Introducing ElementTree 1.3
- <http://effbot.org/zone/elementtree-13-intro.htm>`_.
+XPath support
+-------------
+This module provides limited support for
+`XPath expressions <http://www.w3.org/TR/xpath>`_ for locating elements in a
+tree. The goal is to support a small subset of the abbreviated syntax; a full
+XPath engine is outside the scope of the module.
+
+Example
+^^^^^^^
+
+Here's an example that demonstrates some of the XPath capabilities of the
+module. We'll be using the ``countrydata`` XML document from the
+:ref:`Parsing XML <elementtree-parsing-xml>` section::
+
+ import xml.etree.ElementTree as ET
+
+ root = ET.fromstring(countrydata)
+
+ # Top-level elements
+ root.findall(".")
+
+ # All 'neighbor' grand-children of 'country' children of the top-level
+ # elements
+ root.findall("./country/neighbor")
+
+ # Nodes with name='Singapore' that have a 'year' child
+ root.findall(".//year/..[@name='Singapore']")
+
+ # 'year' nodes that are children of nodes with name='Singapore'
+ root.findall(".//*[@name='Singapore']/year")
+
+ # All 'neighbor' nodes that are the second child of their parent
+ root.findall(".//neighbor[2]")
+
+Supported XPath syntax
+^^^^^^^^^^^^^^^^^^^^^^
+
++-----------------------+------------------------------------------------------+
+| Syntax | Meaning |
++=======================+======================================================+
+| ``tag`` | Selects all child elements with the given tag. |
+| | For example, ``spam`` selects all child elements |
+| | named ``spam``, ``spam/egg`` selects all |
+| | grandchildren named ``egg`` in all children named |
+| | ``spam``. |
++-----------------------+------------------------------------------------------+
+| ``*`` | Selects all child elements. For example, ``*/egg`` |
+| | selects all grandchildren named ``egg``. |
++-----------------------+------------------------------------------------------+
+| ``.`` | Selects the current node. This is mostly useful |
+| | at the beginning of the path, to indicate that it's |
+| | a relative path. |
++-----------------------+------------------------------------------------------+
+| ``//`` | Selects all subelements, on all levels beneath the |
+| | current element. For example, ``.//egg`` selects |
+| | all ``egg`` elements in the entire tree. |
++-----------------------+------------------------------------------------------+
+| ``..`` | Selects the parent element. Returns ``None`` if the |
+| | path attempts to reach the ancestors of the start |
+| | element (the element ``find`` was called on). |
++-----------------------+------------------------------------------------------+
+| ``[@attrib]`` | Selects all elements that have the given attribute. |
++-----------------------+------------------------------------------------------+
+| ``[@attrib='value']`` | Selects all elements for which the given attribute |
+| | has the given value. The value cannot contain |
+| | quotes. |
++-----------------------+------------------------------------------------------+
+| ``[tag]`` | Selects all elements that have a child named |
+| | ``tag``. Only immediate children are supported. |
++-----------------------+------------------------------------------------------+
+| ``[position]`` | Selects all elements that are located at the given |
+| | position. The position can be either an integer |
+| | (1 is the first position), the expression ``last()`` |
+| | (for the last position), or a position relative to |
+| | the last position (e.g. ``last()-1``). |
++-----------------------+------------------------------------------------------+
+
+Predicates (expressions within square brackets) must be preceded by a tag
+name, an asterisk, or another predicate. ``position`` predicates must be
+preceded by a tag name.
+
+Reference
+---------
.. _elementtree-functions:
XML Processing Modules
======================
+ .. module:: xml
+ :synopsis: Package containing XML processing modules
+ .. sectionauthor:: Christian Heimes <christian@python.org>
+ .. sectionauthor:: Georg Brandl <georg@python.org>
+
+
Python's interfaces for processing XML are grouped in the ``xml`` package.
+ .. warning::
+
+ The XML modules are not secure against erroneous or maliciously
+ constructed data. If you need to parse untrusted or unauthenticated data see
+ :ref:`xml-vulnerabilities`.
+
++
It is important to note that modules in the :mod:`xml` package require that
there be at least one SAX-compliant XML parser available. The Expat parser is
included with Python, so the :mod:`xml.parsers.expat` module will always be
* :mod:`xml.sax`: SAX2 base classes and convenience functions
* :mod:`xml.parsers.expat`: the Expat parser binding
+
+
+ .. _xml-vulnerabilities:
+
+ XML vulnerabilities
+ ===================
+
+ The XML processing modules are not secure against maliciously constructed data.
+ An attacker can abuse vulnerabilities for e.g. denial of service attacks, to
+ access local files, to generate network connections to other machines, or
+ to or circumvent firewalls. The attacks on XML abuse unfamiliar features
+ like inline `DTD`_ (document type definition) with entities.
+
+
+ ========================= ======== ========= ========= ======== =========
+ kind sax etree minidom pulldom xmlrpc
+ ========================= ======== ========= ========= ======== =========
+ billion laughs **True** **True** **True** **True** **True**
+ quadratic blowup **True** **True** **True** **True** **True**
+ external entity expansion **True** False (1) False (2) **True** False (3)
+ DTD retrieval **True** False False **True** False
+ decompression bomb False False False False **True**
+ ========================= ======== ========= ========= ======== =========
+
+ 1. :mod:`xml.etree.ElementTree` doesn't expand external entities and raises a
+ ParserError when an entity occurs.
+ 2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns
+ the unexpanded entity verbatim.
+ 3. :mod:`xmlrpclib` doesn't expand external entities and omits them.
+
+
+ billion laughs / exponential entity expansion
+ The `Billion Laughs`_ attack -- also known as exponential entity expansion --
+ uses multiple levels of nested entities. Each entity refers to another entity
+ several times, the final entity definition contains a small string. Eventually
+ the small string is expanded to several gigabytes. The exponential expansion
+ consumes lots of CPU time, too.
+
+ quadratic blowup entity expansion
+ A quadratic blowup attack is similar to a `Billion Laughs`_ attack; it abuses
+ entity expansion, too. Instead of nested entities it repeats one large entity
+ with a couple of thousand chars over and over again. The attack isn't as
+ efficient as the exponential case but it avoids triggering countermeasures of
+ parsers against heavily nested entities.
+
+ external entity expansion
+ Entity declarations can contain more than just text for replacement. They can
+ also point to external resources by public identifiers or system identifiers.
+ System identifiers are standard URIs or can refer to local files. The XML
+ parser retrieves the resource with e.g. HTTP or FTP requests and embeds the
+ content into the XML document.
+
+ DTD retrieval
+ Some XML libraries like Python's mod:'xml.dom.pulldom' retrieve document type
+ definitions from remote or local locations. The feature has similar
+ implications as the external entity expansion issue.
+
+ decompression bomb
+ The issue of decompression bombs (aka `ZIP bomb`_) apply to all XML libraries
+ that can parse compressed XML stream like gzipped HTTP streams or LZMA-ed
+ files. For an attacker it can reduce the amount of transmitted data by three
+ magnitudes or more.
+
+ The documentation of `defusedxml`_ on PyPI has further information about
+ all known attack vectors with examples and references.
+
+ defused packages
+ ----------------
+
+ `defusedxml`_ is a pure Python package with modified subclasses of all stdlib
+ XML parsers that prevent any potentially malicious operation. The courses of
+ action are recommended for any server code that parses untrusted XML data. The
+ package also ships with example exploits and an extended documentation on more
+ XML exploits like xpath injection.
+
+ `defusedexpat`_ provides a modified libexpat and patched replacment
+ :mod:`pyexpat` extension module with countermeasures against entity expansion
+ DoS attacks. Defusedexpat still allows a sane and configurable amount of entity
+ expansions. The modifications will be merged into future releases of Python.
+
+ The workarounds and modifications are not included in patch releases as they
+ break backward compatibility. After all inline DTD and entity expansion are
+ well-definied XML features.
+
+
+ .. _defusedxml: <https://pypi.python.org/pypi/defusedxml/>
+ .. _defusedexpat: <https://pypi.python.org/pypi/defusedexpat/>
+ .. _Billion Laughs: http://en.wikipedia.org/wiki/Billion_laughs
+ .. _ZIP bomb: http://en.wikipedia.org/wiki/Zip_bomb
+ .. _DTD: http://en.wikipedia.org/wiki/Document_Type_Definition
++
Build
-----
-- Issue #10325: Fix two issues in the fallback definitions for PY_ULLONG_MAX and
- PY_LLONG_MAX that made them unsuitable for use in preprocessor conditionals.
-
-Documentation
--------------
-
-- Issue #10299: List the built-in functions in a table in functions.rst.
-
-
-What's New in Python 3.2 Alpha 4?
-=================================
-
-*Release date: 13-Nov-2010*
-
-Core and Builtins
------------------
-
-- Issue #10372: Import the warnings module only after the IO library is
- initialized, so as to avoid bootstrap issues with the '-W' option.
-
-- Issue #10293: Remove obsolete field in the PyMemoryView structure, unused
- undocumented value PyBUF_SHADOW, and strangely-looking code in
- PyMemoryView_GetContiguous.
-
-- Issue #6081: Add str.format_map(), similar to ``str.format(**mapping)``.
+- Issue #17550: Fix the --enable-profiling configure switch.
-- If FileIO.__init__ fails, close the file descriptor.
+- Issue #17425: Build with openssl 1.0.1d on Windows.
-- Issue #10221: dict.pop(k) now has a key error message that includes the
- missing key (same message d[k] returns for missing keys).
-
-- Issue #5437: A preallocated MemoryError instance should not keep traceback
- data (including local variables caught in the stack trace) alive infinitely.
-
-- Issue #10186: Fix the SyntaxError caret when the offset is equal to the length
- of the offending line.
-
-- Issue #10089: Add support for arbitrary -X options on the command line. They
- can be retrieved through a new attribute ``sys._xoptions``.
-
-- Issue #4388: On Mac OS X, decode command line arguments from UTF-8, instead of
- the locale encoding. If the LANG (and LC_ALL and LC_CTYPE) environment
- variable is not set, the locale encoding is ISO-8859-1, whereas most programs
- (including Python) expect UTF-8. Python already uses UTF-8 for the filesystem
- encoding and to encode command line arguments on this OS.
-
-- Issue #9713, #10114: Parser functions (e.g. PyParser_ASTFromFile) expect
- filenames encoded to the filesystem encoding with the surrogateescape error
- handler (to support undecodable bytes), instead of UTF-8 in strict mode.
-
-- Issue #9997: Don't let the name "top" have special significance in scope
- resolution.
-
-- Issue #9862: Compensate for broken PIPE_BUF in AIX by hard coding its value as
- the default 512 when compiling on AIX.
-
-- Use locale encoding instead of UTF-8 to encode and decode filenames if
- Py_FileSystemDefaultEncoding is not set.
-
-- Issue #10095: fp_setreadl() doesn't reopen the file, instead reuse the file
- descriptor.
-
-- Issue #9418: Moved private string methods ``_formatter_parser`` and
- ``_formatter_field_name_split`` into a new ``_string`` module.
-
-- Issue #9992: Remove PYTHONFSENCODING environment variable.
-
-Library
--------
-
-- Issue #10465: fix broken delegating of attributes by gzip._PaddedFile.
-
-- Issue #10356: Decimal.__hash__(-1) should return -2.
-
-- Issue #1553375: logging: Added stack_info kwarg to display stack information.
-
-- Issue #5111: IPv6 Host in the Header is wrapped inside [ ]. Patch by Chandru.
-
-- Fix Fraction.__hash__ so that Fraction.__hash__(-1) is -2. (See also issue
- #10356.)
-
-- Issue #4471: Add the IMAP.starttls() method to enable encryption on standard
- IMAP4 connections. Original patch by Lorenzo M. Catucci.
-
-- Issue #1466065: Add 'validate' option to base64.b64decode to raise an error if
- there are non-base64 alphabet characters in the input.
-
-- Issue #10386: Add __all__ to token module; this simplifies importing in
- tokenize module and prevents leaking of private names through ``import *``.
-
-- Issue #4471: Properly shutdown socket in IMAP.shutdown(). Patch by Lorenzo
- M. Catucci.
-
-- Fix IMAP.login() to work properly.
-
-- Issue #9244: multiprocessing pool worker processes could terminate
- unexpectedly if the return value of a task could not be pickled. Only the
- ``repr`` of such errors are now sent back, wrapped in an
- ``MaybeEncodingError`` exception.
-
-- Issue #9244: The ``apply_async()`` and ``map_async()`` methods of
- ``multiprocessing.Pool`` now accepts a ``error_callback`` argument. This can
- be a callback with the signature ``callback(exc)``, which will be called if
- the target raises an exception.
-
-- Issue #10022: The dictionary returned by the ``getpeercert()`` method of SSL
- sockets now has additional items such as ``issuer`` and ``notBefore``.
-
-- ``usenetrc`` is now false by default for NNTP objects.
-
-- Issue #1926: Add support for NNTP over SSL on port 563, as well as STARTTLS.
- Patch by Andrew Vant.
-
-- Issue #10335: Add tokenize.open(), detect the file encoding using
- tokenize.detect_encoding() and open it in read only mode.
-
-- Issue #10321: Add support for binary data to smtplib.SMTP.sendmail, and a new
- method send_message to send an email.message.Message object.
-
-- Issue #6011: sysconfig and distutils.sysconfig use the surrogateescape error
- handler to parse the Makefile file. Avoid a UnicodeDecodeError if the source
- code directory name contains a non-ASCII character and the locale encoding is
- ASCII.
-
-- Issue #10329: The trace module writes reports using the input Python script
- encoding, instead of the locale encoding. Patch written by Alexander
- Belopolsky.
-
-- Issue #10126: Fix distutils' test_build when Python was built with
- --enable-shared.
-
-- Issue #9281: Prevent race condition with mkdir in distutils. Patch by
- Arfrever.
-
-- Issue #10229: Fix caching error in gettext.
-
-- Issue #10252: Close file objects in a timely manner in distutils code and
- tests. Patch by Brian Brazil, completed by Éric Araujo.
-
-- Issue #10180: Pickling file objects is now explicitly forbidden, since
- unpickling them produced nonsensical results.
-
-- Issue #10311: The signal module now restores errno before returning from its
- low-level signal handler. Patch by Hallvard B Furuseth.
-
-- Issue #10282: Add a ``nntp_implementation`` attribute to NNTP objects.
-
-- Issue #10283: Add a ``group_pattern`` argument to NNTP.list().
-
-- Issue #10155: Add IISCGIHandler to wsgiref.handlers to support IIS CGI
- environment better, and to correct unicode environment values for WSGI 1.0.1.
-
-- Issue #10281: nntplib now returns None for absent fields in the OVER/XOVER
- response, instead of raising an exception.
-
-- wsgiref now implements and validates PEP 3333, rather than an experimental
- extension of PEP 333. (Note: earlier versions of Python 3.x may have
- incorrectly validated some non-compliant applications as WSGI compliant; if
- your app validates with Python <3.2b1+, but not on this version, it is likely
- the case that your app was not compliant.)
-
-- Issue #10280: NNTP.nntp_version should reflect the highest version advertised
- by the server.
-
-- Issue #10184: Touch directories only once when extracting a tarfile.
-
-- Issue #10199: New package, ``turtledemo`` now contains selected demo scripts
- that were formerly found under Demo/turtle.
-
-- Issue #10265: Close file objects explicitly in sunau. Patch by Brian Brazil.
-
-- Issue #10266: uu.decode didn't close in_file explicitly when it was given as a
- filename. Patch by Brian Brazil.
-
-- Issue #10110: Queue objects didn't recognize full queues when the maxsize
- parameter had been reduced.
-
-- Issue #10160: Speed up operator.attrgetter. Patch by Christos Georgiou.
-
-- logging: Added style option to basicConfig() to allow %, {} or $-formatting.
-
-- Issue #5729: json.dumps() now supports using a string such as '\t' for
- pretty-printing multilevel objects.
-
-- Issue #10253: FileIO leaks a file descriptor when trying to open a file for
- append that isn't seekable. Patch by Brian Brazil.
-
-- Support context manager protocol for file-like objects returned by mailbox
- ``get_file()`` methods.
-
-- Issue #10246: uu.encode didn't close file objects explicitly when filenames
- were given to it. Patch by Brian Brazil.
-
-- Issue #10198: fix duplicate header written to wave files when writeframes() is
- called without data.
-
-- Close file objects in modulefinder in a timely manner.
-
-- Close a io.TextIOWrapper object in email.parser in a timely manner.
-
-- Close a file object in distutils.sysconfig in a timely manner.
+- Issue #16754: Fix the incorrect shared library extension on linux. Introduce
+ two makefile macros SHLIB_SUFFIX and EXT_SUFFIX. SO now has the value of
+ SHLIB_SUFFIX again (as in 2.x and 3.1). The SO macro is removed in 3.4.
-- Close a file object in pkgutil in a timely manner.
+- Issue #5033: Fix building of the sqlite3 extension module when the
+ SQLite library version has "beta" in it. Patch by Andreas Pelme.
-- Issue #10233: Close file objects in a timely manner in the tarfile module and
- its test suite.
+- Issue #17228: Fix building without pymalloc.
-- Issue #10093: ResourceWarnings are now issued when files and sockets are
- deallocated without explicit closing. These warnings are silenced by default,
- except in pydebug mode.
+- Issue #3718: Use AC_ARG_VAR to set MACHDEP in configure.ac.
-- tarfile.py: Add support for all missing variants of the GNU sparse extensions
- and create files with holes when extracting sparse members.
+- Issue #17031: Fix running regen in cross builds.
-- Issue #10218: Return timeout status from ``Condition.wait`` in threading.
+- Issue #3754: fix typo in pthread AC_CACHE_VAL.
-- Issue #7351: Add ``zipfile.BadZipFile`` spelling of the exception name and
- deprecate the old name ``zipfile.BadZipfile``.
+- Issue #15484: Fix _PYTHON_PROJECT_BASE for srcdir != builddir builds;
+ use _PYTHON_PROJECT_BASE in distutils/sysconfig.py.
-- Issue #5027: The standard ``xml`` namespace is now understood by
- xml.sax.saxutils.XMLGenerator as being bound to
- http://www.w3.org/XML/1998/namespace. Patch by Troy J. Farrell.
+- Issue #17029: Let h2py search the multiarch system include directory.
-- Issue #5975: Add csv.unix_dialect class.
+- Issue #16953: Fix socket module compilation on platforms with
+ HAVE_BROKEN_POLL. Patch by Jeffrey Armstrong.
-- Issue #7761: telnetlib.interact failures on Windows fixed.
+- Issue #16836: Enable IPv6 support even if IPv6 is disabled on the build host.
-- logging: Added style option to Formatter to allow %, {} or $-formatting.
+- Cross compiling needs host and build settings. configure no longer
+ creates a broken PYTHON_FOR_BUILD variable when --build is missing.
-- Issue #5178: Added tempfile.TemporaryDirectory class that can be used as a
- context manager.
+- Fix cross compiling issue in setup.py, ensure that lib_dirs and inc_dirs are
+ defined in cross compiling mode, too.
-- Issue #1349106: Generator (and BytesGenerator) flatten method and Header
- encode method now support a 'linesep' argument.
+- Issue #16593: Have BSD 'make -s' do the right thing, thanks to Daniel Shahaf
-- Issue #5639: Add a *server_hostname* argument to ``SSLContext.wrap_socket`` in
- order to support the TLS SNI extension. ``HTTPSConnection`` and ``urlopen()``
- also use this argument, so that HTTPS virtual hosts are now supported.
+- Issue #16262: fix out-of-src-tree builds, if mercurial is not installed.
-- Issue #10166: Avoid recursion in pstats Stats.add() for many stats items.
+- Issue #15298: ensure _sysconfigdata is generated in build directory, not
+ source directory.
-- Issue #10163: Skip unreadable registry keys during mimetypes initialization.
+- Issue #15833: Fix a regression in 3.3 that resulted in exceptions being
+ raised if importlib failed to write byte-compiled files. This affected
+ attempts to build Python out-of-tree from a read-only source directory.
-- logging: Made StreamHandler terminator configurable.
+- Issue #15923: Fix a mistake in ``asdl_c.py`` that resulted in a TypeError
+ after 2801bf875a24 (see #15801).
-- logging: Allowed filters to be just callables.
+- Issue #15819: Make sure we can build Python out-of-tree from a read-only
+ source directory. (Somewhat related to issue #9860.)
-- logging: Added tests for _logRecordClass changes.
+- Issue #15587: Enable Tk high-resolution text rendering on Macs with
+ Retina displays. Applies to Tkinter apps, such as IDLE, on OS X
+ framework builds linked with Cocoa Tk 8.5.
-- Issue #10092: Properly reset locale in calendar.Locale*Calendar classes.
+- Issue #17161: make install now also installs a python3 man page.
-- logging: Added _logRecordClass, getLogRecordClass, setLogRecordClass to
- increase flexibility of LogRecord creation.
+Tools/Demos
+-----------
-- Issue #5117: Case normalization was needed on ntpath.relpath(). Also fixed
- root directory issue on posixpath.relpath(). (Ported working fixes from
- ntpath.)
+- Issue #13301: use ast.literal_eval() instead of eval() in Tools/i18n/msgfmt.py
+ Patch by Serhiy Storchaka.
-- Issue #1343: xml.sax.saxutils.XMLGenerator now has an option
- short_empty_elements to direct it to use self-closing tags when appropriate.
+Documentation
+-------------
-- Issue #9807 (part 1): Expose the ABI flags in sys.abiflags. Add --abiflags
- switch to python-config for command line access.
++- Issue 17538: Document XML vulnerabilties
+
-- Issue #6098: Don't claim DOM level 3 conformance in minidom.
+- Issue #16642: sched.scheduler timefunc initial default is time.monotonic.
+ Patch by Ramchandra Apte
-- Issue #5762: Fix AttributeError raised by ``xml.dom.minidom`` when an empty
- XML namespace attribute is encountered.
+- Issue #17047: remove doubled words in docs and docstrings
+ reported by Serhiy Storchaka and Matthew Barnett.
-- Issue #2830: Add the ``html.escape()`` function, which quotes all problematic
- characters by default. Deprecate ``cgi.escape()``.
+- Issue #15465: Document the versioning macros in the C API docs rather than
+ the standard library docs. Patch by Kushal Das.
-- Issue #9409: Fix the regex to match all kind of filenames, for interactive
- debugging in doctests.
+- Issue #16406: Combine the pages for uploading and registering to PyPI.
-- Issue #9183: ``datetime.timezone(datetime.timedelta(0))`` will now return the
- same instance as ``datetime.timezone.utc``.
+- Issue #16403: Document how distutils uses the maintainer field in
+ PKG-INFO. Patch by Jyrki Pulliainen.
-- Issue #7523: Add SOCK_CLOEXEC and SOCK_NONBLOCK to the socket module, where
- supported by the system. Patch by Nikita Vetoshkin.
+- Issue #16695: Document how glob handles filenames starting with a
+ dot. Initial patch by Jyrki Pulliainen.
-- Issue #10063: file:// scheme will stop accessing remote hosts via ftp
- protocol. file:// urls had fallback to access remote hosts via ftp. This was
- not correct, change is made to raise a URLError when a remote host is tried to
- access via file:// scheme.
+- Issue #8890: Stop advertising an insecure practice by replacing uses
+ of the /tmp directory with better alternatives in the documentation.
+ Patch by Geoff Wilson.
-- Issue #1710703: Write structures for an empty ZIP archive when a ZipFile is
- created in modes 'a' or 'w' and then closed without adding any files. Raise
- BadZipfile (rather than IOError) when opening small non-ZIP files.
+- Issue #17203: add long option names to unittest discovery docs.
-- Issue #10041: The signature of optional arguments in socket.makefile() didn't
- match that of io.open(), and they also didn't get forwarded properly to
- TextIOWrapper in text mode. Patch by Kai Zhu.
+- Issue #13094: add "Why do lambdas defined in a loop with different values
+ all return the same result?" programming FAQ.
-- Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and
- urllib.request.urlopen now take optional arguments to allow for server
- certificate checking, as recommended in public uses of HTTPS.
+- Issue #14901: Update portions of the Windows FAQ.
+ Patch by Ashish Nitin Patil.
-- Issue #6612: Fix site and sysconfig to catch os.getcwd() error, eg. if the
- current directory was deleted. Patch written by W. Trevor King.
+- Issue #16267: Better document the 3.3+ approach to combining
+ @abstractmethod with @staticmethod, @classmethod and @property
-- Issue #3873: Speed up unpickling from file objects that have a peek() method.
+- Issue #15209: Clarify exception chaining description in exceptions module
+ documentation
-- Issue #10075: Add a session_stats() method to SSLContext objects.
+- Issue #15990: Improve argument/parameter documentation.
-- Issue #9948: Fixed problem of losing filename case information.
+- Issue #16209: Move the documentation for the str built-in function to a new
+ str class entry in the "Text Sequence Type" section.
-Extension Modules
------------------
+- Issue #13538: Improve str() and object.__str__() documentation.
-- Issue #5109: array.array constructor will now use fast code when
- initial data is provided in an array object with correct type.
+- Issue #16489: Make it clearer that importlib.find_loader() requires any and
+ all packages to be separately imported.
-- Issue #6317: Now winsound.PlaySound only accepts unicode.
+- Issue #16400: Update the description of which versions of a given package
+ PyPI displays.
-- Issue #6317: Now winsound.PlaySound can accept non ascii filename.
+- Issue #15677: Document that zlib and gzip accept a compression level of 0 to
+ mean 'no compression'. Patch by Brian Brazil.
-- Issue #9377: Use Unicode API for gethostname on Windows.
+- Issue #8040: added a version switcher to the documentation. Patch by
+ Yury Selivanov.
-- Issue #10143: Update "os.pathconf" values.
+- Additional comments and some style changes in the concurrent.futures URL
+ retrieval example
-- Issue #6518: Support context manager protcol for ossaudiodev types.
+- Issue #16115: Improve subprocess.Popen() documentation around args, shell,
+ and executable arguments.
-- Issue #678250: Make mmap flush a noop on ACCESS_READ and ACCESS_COPY.
+- Issue #15533: Clarify docs and add tests for `subprocess.Popen()`'s cwd
+ argument.
-- Issue #9054: Fix a crash occurring when using the pyexpat module with expat
- version 2.0.1.
+- Issue #15979: Improve timeit documentation.
-- Issue #5355: Provide mappings from Expat error numbers to string descriptions
- and backwards, in order to actually make it possible to analyze error codes
- provided by ExpatError.
+- Issue #16036: Improve documentation of built-in `int()`'s signature and
+ arguments.
-- The Unicode database was updated to 6.0.0.
+- Issue #15935: Clarification of `argparse` docs, re: add_argument() type and
+ default arguments. Patch contributed by Chris Jerdonek.
-C-API
------
+- Issue #11964: Document a change in v3.2 to the behavior of the indent
+ parameter of json encoding operations.
-- Issue #10288: The deprecated family of "char"-handling macros
- (ISLOWER()/ISUPPER()/etc) have now been removed: use Py_ISLOWER() etc instead.
+- Issue #15116: Remove references to appscript as it is no longer being
+ supported.
-- Issue #9778: Hash values are now always the size of pointers. A new Py_hash_t
- type has been introduced.
+- Issue #15116: Remove references to appscript as it is no longer being
+ supported.
Tools/Demos
-----------
projects / thirdparty / Python / cpython.git / commitdiff
