From: Antoine Pitrou Date: Fri, 27 Jan 2012 08:50:45 +0000 (+0100) Subject: Issue #13885: CVE-2011-3389: the _ssl module would always disable the CBC IV attack... X-Git-Tag: v3.2.3rc1~121 X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=3f366314e831e0babca220abd734f8ae02776925;p=thirdparty%2FPython%2Fcpython.git Issue #13885: CVE-2011-3389: the _ssl module would always disable the CBC IV attack countermeasure. --- 3f366314e831e0babca220abd734f8ae02776925 diff --cc Misc/NEWS index 5a52b9fc055a,b70b097cb754..8a85fe440943 --- a/Misc/NEWS +++ b/Misc/NEWS @@@ -111,3872 -13,935 +111,3875 @@@ Core and Builtin Library ------- + - Issue #13885: CVE-2011-3389: the _ssl module would always disable the CBC + IV attack countermeasure. + -- Issue #11603: Fix a crash when __str__ is rebound as __repr__. Patch by - Andreas Stührk. +- Issue #13772: In os.symlink() under Windows, do not try to guess the link + target's type (file or directory). The detection was buggy and made the + call non-atomic (therefore prone to race conditions). +- Issue #6631: Disallow relative file paths in urllib urlopen methods. -What's New in Python 3.1.4? -=========================== +- Issue #13722: Avoid silencing ImportErrors when initializing the codecs + registry. -*Release date: 2011-06-11* +- Issue #13781: Fix GzipFile bug that caused an exception to be raised when + opening for writing using a fileobj returned by os.fdopen(). -Library -------- +- Issue #13803: Under Solaris, distutils doesn't include bitness + in the directory name. -- Issue #12502: asyncore: fix polling loop with AF_UNIX sockets. +- Issue #13589: Fix some serialization primitives in the aifc module. + Patch by Oleg Plakhotnyuk. -- Issue #12009: Fixed regression in netrc file comment handling. +- Issue #13642: Unquote before b64encoding user:password during Basic + Authentication. Patch contributed by Joonas Kuorilehto. -Extension Modules ------------------ +- Issue #13726: Fix the ambiguous -S flag in regrtest. It is -o/--slow for slow + tests. -- Issue #12221: Replace pyexpat.__version__ with the Python version. +- Issue #12364: Fix a hang in concurrent.futures.ProcessPoolExecutor. + The hang would occur when retrieving the result of a scheduled future after + the executor had been shut down. -What's New in Python 3.1.4 release candidate 1? -=============================================== +- Issue #13502: threading: Fix a race condition in Event.wait() that made it + return False when the event was set and cleared right after. -*Release date: 2011-05-29* +- Issue #12926: Fix a bug in tarfile's link extraction. -Core and Builtins ------------------ +- Issue #13696: Fix the 302 Relative URL Redirection problem. -- Issue #9670: Increase the default stack size for secondary threads on - Mac OS X and FreeBSD to reduce the chances of a crash instead of a - "maximum recursion depth" RuntimeError exception. - (patch by Ronald Oussoren) +- Issue #13636: Weak ciphers are now disabled by default in the ssl module + (except when SSLv2 is explicitly asked for). -- Correct lookup of __dir__ on objects. Among other things, this causes errors - besides AttributeError found on lookup to be propagated. +- Issue #12798: Updated the mimetypes documentation. -- Issue #12060: Use sig_atomic_t type and volatile keyword in the signal - module. Patch written by Charles-François Natali. +- Issue #11006: Don't issue low level warning in subprocess when pipe2() fails. -- Issue #1195: Fix input() if it is interrupted by CTRL+d and then CTRL+c, - clear the end-of-file indicator after CTRL+d. +- Issue #11829: Fix code execution holes in inspect.getattr_static for + metaclasses with metaclasses. Patch by Andreas Stührk. -- Issue #9756: When calling a method descriptor or a slot wrapper descriptor, - the check of the object type doesn't read the __class__ attribute anymore. - Fix a crash if a class override its __class__ attribute (e.g. a proxy of the - str type). Patch written by Andreas Stührk. +- Issue #1785: Fix inspect and pydoc with misbehaving descriptors. -- Issue #6780: fix starts/endswith error message to mention that tuples are - accepted too. +- Issue #11813: Fix inspect.getattr_static for modules. Patch by Andreas + Stührk. -- Issue #5057: fix a bug in the peepholer that led to non-portable pyc files - between narrow and wide builds while optimizing BINARY_SUBSCR on non-BMP - chars (e.g. "\U00012345"[0]). +- Issue #7502: Fix equality comparison for DocTestCase instances. Patch by + Cédric Krier. -- Issue #11650: PyOS_StdioReadline() retries fgets() if it was interrupted - (EINTR), for example if the program is stopped with CTRL+z on Mac OS X. Patch - written by Charles-Francois Natali. +- Issue #11870: threading: Properly reinitialize threads internal locks and + condition variables to avoid deadlocks in child processes. -- Issue #8651: PyArg_Parse*() functions raise an OverflowError if the file - doesn't have PY_SSIZE_T_CLEAN define and the size doesn't fit in an int - (length bigger than 2^31-1 bytes). +- Issue #8035: urllib: Fix a bug where the client could remain stuck after a + redirection or an error. -- Issue #11450: Don't truncate hg version info in Py_GetBuildInfo() when - there are many tags (e.g. when using mq). Patch by Nadeem Vawda. +- Issue #10350: Read and save errno before calling a function which might + overwrite it. Original patch by Hallvard B Furuseth. -- Issue #10451: memoryview objects could allow to mutate a readable buffer. - Initial patch by Ross Lagerwall. +- Issue #13591: A bug in importlib has been fixed that caused import_module + to load a module twice. -- Issue #10892: Don't segfault when trying to delete __abstractmethods__ from a - class. +- Issue #4625: If IDLE cannot write to its recent file or breakpoint + files, display a message popup and continue rather than crash. + (original patch by Roger Serwy) -- Issue #8020: Avoid a crash where the small objects allocator would read - non-Python managed memory while it is being modified by another thread. - Patch by Matt Bandy. +- Issue #5905: time.strftime() is now using the locale encoding, instead of + UTF-8, if the wcsftime() function is not available. -- Issue #8278: On Windows and with a NTFS filesystem, os.stat() and os.utime() - can now handle dates after 2038. +- Issue #8641: Update IDLE 3 syntax coloring to recognize b".." and not u"..". + Patch by Tal Einat. -- issue #11828: startswith and endswith don't accept None as slice index. - Patch by Torsten Becker. +- tarfile.py: Correctly detect bzip2 compressed streams with blocksizes + other than 900k. -- Issue #4236: PyModule_Create2 now checks the import machinery directly - rather than the Py_IsInitialized flag, avoiding a Fatal Python - error in certain circumstances when an import is done in __del__. +- Issue #13439: Fix many errors in turtle docstrings. -- Issue #10596: Fix float.__mod__ to have the same behaviour as - float.__divmod__ with respect to signed zeros. -4.0 % 4.0 should be - 0.0, not -0.0. +- Issue #13487: Make inspect.getmodule robust against changes done to + sys.modules while it is iterating over it. -- Issue #5587: add a repr to dict_proxy objects. Patch by David Stanek and - Daniel Urban. +- Issue #12618: Fix a bug that prevented py_compile from creating byte + compiled files in the current directory. Initial patch by Sjoerd de Vries. -- Issue #11506: Trying to assign to a bytes literal should result in a - SyntaxError. +- Issue #13444: When stdout has been closed explicitly, we should not attempt + to flush it at shutdown and print an error. -Library -------- +- Issue #12856: Ensure child processes do not inherit the parent's random + seed for filename generation in the tempfile module. Patch by Brian + Harring. -- Issue #12352: Fix a deadlock in multiprocessing.Heap when a block is freed by - the garbage collector while the Heap lock is held. +- Issue #13458: Fix a memory leak in the ssl module when decoding a + certificate with a subjectAltName. Patch by Robert Xiao. -- Issue #985064: Make plistlib more resilient to faulty input plists. - Patch by Mher Movsisyan. +- Issue #13415: os.unsetenv() doesn't ignore errors anymore. -- Issue #12175: RawIOBase.readall() now returns None if read() returns None. +- Issue #13322: Fix BufferedWriter.write() to ensure that BlockingIOError is + raised when the wrapped raw file is non-blocking and the write would block. + Previous code assumed that the raw write() would raise BlockingIOError, but + RawIOBase.write() is defined to returned None when the call would block. + Patch by sbt. -- Issue #12175: FileIO.readall() now raises a ValueError instead of an IOError - if the file is closed. +- Issue #13358: HTMLParser now calls handle_data only once for each CDATA. -- Issue #12100: Don't reset incremental encoders of CJK codecs at each call to - their encode() method anymore, but continue to call the reset() method if the - final argument is True. +- Issue #4147: minidom's toprettyxml no longer adds whitespace around a text + node when it is the only child of an element. Initial patch by Dan + Kenigsberg. -- Issue #5715: In socketserver, close the server socket in the child process. +- Issues #1745761, #755670, #13357, #12629, #1200313: HTMLParser now correctly + handles non-valid attributes, including adjacent and unquoted attributes. -- Issue #12124: zipimport doesn't keep a reference to zlib.decompress() anymore - to be able to unload the module. +- Issue #13193: Fix distutils.filelist.FileList under Windows. The + "recursive-include" directive now recognizes both legal path separators. -- Issue #10801: In zipfile, support different encodings for the header and - the filenames. +- Issue #13384: Remove unnecessary __future__ import in Lib/random.py -- Issue #10154, #10090: change the normalization of UTF-8 to "UTF-8" instead - of "UTF8" in the locale module as the latter is not supported MacOSX and OpenBSD. +- Issue #13373: multiprocessing.Queue.get() could sometimes block indefinitely + when called with a timeout. Patch by Arnaud Ysmal. -- Issue #10756: atexit normalizes the exception before displaying it. Patch by - Andreas Stührk. +- Issue #13254: Fix Maildir initialization so that maildir contents + are read correctly. -- Issue #8650: Make zlib module 64-bit clean. compress(), decompress() and - their incremental counterparts now raise OverflowError if given an input - larger than 4GB, instead of silently truncating the input and returning - an incorrect result. +- Issue #3067: locale.setlocale() now raises TypeError if the second + argument is an invalid iterable. Its documentation and docstring + were also updated. Initial patch by Jyrki Pulliainen. -- Issue #12050: zlib.decompressobj().decompress() now clears the unconsumed_tail - attribute when called without a max_length argument. +- Issue #13140: Fix the daemon_threads attribute of ThreadingMixIn. -- Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence - on a file opened in read+write mode (namely: reading, seeking a bit forward, - writing, then seeking before the previous write but still within buffered - data, and writing again). +- Issue #13339: Fix compile error in posixmodule.c due to missing semicolon. + Thanks to Robert Xiao. -- Issue #1028: Tk returns invalid Unicode null in %A: UnicodeDecodeError. - With Tk < 8.5 _tkinter.c:PythonCmd() raised UnicodeDecodeError, caused - IDLE to exit. Converted to valid Unicode null in PythonCmd(). +- Issue #10570: curses.putp() and curses.tparm() are now expecting a byte + string, instead of a Unicode string. -- Issue #10419: Fix build_scripts command of distutils to handle correctly - non-ASCII scripts. Open and write the script in binary mode, but ensure that - the shebang is decodable from UTF-8 and from the encoding of the script. +- Issue #2892: preserve iterparse events in case of SyntaxError. -- Issue #12012: ssl.PROTOCOL_SSLv2 becomes optional. +- Issue #670664: Fix HTMLParser to correctly handle the content of + ```` and ````. -- Issue #11164: Stop trying to use _xmlplus in the xml module. +- Issue #10817: Fix urlretrieve function to raise ContentTooShortError even + when reporthook is None. Patch by Jyrki Pulliainen. -- Issue #11927: SMTP_SSL now uses port 465 by default as documented. Patch - by Kasun Herath. +- Issue #13296: Fix IDLE to clear compile __future__ flags on shell restart. + (Patch by Roger Serwy) -- Issue #12002: ftplib's abort() method raises TypeError. +- Issue #13293: Better error message when trying to marshal bytes using + xmlrpc.client. -- Issue #11999: fixed sporadic sync failure mailbox.Maildir due to its trying to - detect mtime changes by comparing to the system clock instead of to the - previous value of the mtime. +- Issue #13291: NameError in xmlrpc package. -- Issue #10684: shutil.move used to delete a folder on case insensitive - filesystems when the source and destination name where the same except - for the case. +- Issue #13258: Use callable() built-in in the standard library. -- Issue #11277: mmap.mmap() calls fcntl(fd, F_FULLFSYNC) on Mac OS X to get - around a mmap bug with sparse files. Patch written by Steffen Daode Nurpmeso. +- Issue #13273: fix a bug that prevented HTMLParser to properly detect some + tags when strict=False. -- Issue #11763: don't use difflib in TestCase.assertMultiLineEqual if the - strings are too long. +- Issue #10332: multiprocessing: fix a race condition when a Pool is closed + before all tasks have completed. -- Issue #11236: getpass.getpass responds to ctrl-c or ctrl-z on terminal. +- Issue #13255: wrong docstrings in array module. -- Issue #11768: The signal handler of the signal module only calls - Py_AddPendingCall() for the first signal to fix a deadlock on reentrant or - parallel calls. PyErr_SetInterrupt() writes also into the wake up file. +- Issue #9168: now smtpd is able to bind privileged port. -- Issue #11467: Fix urlparse behavior when handling urls which contains scheme - specific part only digits. Patch by Santoso Wijaya. +- Issue #12529: fix cgi.parse_header issue on strings with double-quotes and + semicolons together. Patch by Ben Darnell and Petri Lehtinen. -- Issue #11875: collections.OrderedDict's __reduce__ was temporarily - mutating the object instead of just working on a copy. +- Issue #12448: smtplib now flushes stdout while running ``python -m smtplib`` + in order to display the prompt correctly. -- collections.Counter().copy() now works correctly for subclasses. +- Issue #6090: zipfile raises a ValueError when a document with a timestamp + earlier than 1980 is provided. Patch contributed by Petri Lehtinen. -- Issue #11474: Fix the bug with url2pathname() handling of '/C|/' on Windows. - Patch by Santoso Wijaya. +- Issue #13194: zlib.compressobj().copy() and zlib.decompressobj().copy() are + now available on Windows. -- Issue #9233: Fix json to work properly even when _json is not available. +- Issue #13158: Fix decoding and encoding of GNU tar specific base-256 number + fields in tarfile. -- Issue #11703: urllib2.geturl() does not return correct url when the original - url contains #fragment. +- Issue #13177: Functools lru_cache() no longer calls the original function + inside an exception handler. This makes tracebacks easier to read because + chained exceptions are avoided. -- Issue #10019: Fixed regression in json module where an indent of 0 stopped - adding newlines and acted instead like 'None'. +- Issue #13025: mimetypes is now reading MIME types using the UTF-8 encoding, + instead of the locale encoding. -- Issue #5162: Treat services like frozen executables to allow child spawning - from multiprocessing.forking on Windows. +- Issue #10653: On Windows, use strftime() instead of wcsftime() because + wcsftime() doesn't format time zone correctly. -- Issue #10963: Ensure that subprocess.communicate() never raises EPIPE. +- Issue #11171: Fix distutils.sysconfig.get_makefile_filename when Python was + configured with different prefix and exec-prefix. -- Issue #11696: Fix ID generation in msilib. +- Issue #11254: Teach distutils to compile .pyc and .pyo files in + PEP 3147-compliant __pycache__ directories. -- Issue #9696: Fix exception incorrectly raised by xdrlib.Packer.pack_int when - trying to pack a negative (in-range) integer. +- Issue #11250: Back port fix from 3.3 branch, so that 2to3 can handle files + with line feeds. This was ported from the sandbox to the 3.3 branch, but + didn't make it into 3.2. -- Issue #11675: multiprocessing.[Raw]Array objects created from an integer size - are now zeroed on creation. This matches the behaviour specified by the - documentation. +- Issue #7367: Fix pkgutil.walk_paths to skip directories whose + contents cannot be read. -- Issue #7639: Fix short file name generation in bdist_msi. +- Issue #13099: Fix sqlite3.Cursor.lastrowid under a Turkish locale. + Reported and diagnosed by Thomas Kluyver. -- Issue #11659: Fix ResourceWarning in test_subprocess introduced by #11459. - Patch by Ben Hayden. +- Issue #13087: BufferedReader.seek() now always raises UnsupportedOperation + if the underlying raw stream is unseekable, even if the seek could be + satisfied using the internal buffer. Patch by John O'Connor. -- Issue #11662: Make urllib and urllib2 ignore redirections if the - scheme is not HTTP, HTTPS or FTP (CVE-2011-1521). +- Issue #7689: Allow pickling of dynamically created classes when their + metaclass is registered with copyreg. Patch by Nicolas M. Thiéry and Craig + Citro. -- Issue #5537: Fix time2isoz() and time2netscape() functions of - httplib.cookiejar for expiration year greater than 2038 on 32-bit systems. +- Issue #4147: minidom's toprettyxml no longer adds whitespace to text nodes. -- Issue #11459: A ``bufsize`` value of 0 in subprocess.Popen() really creates - unbuffered pipes, such that select() works properly on them. +- Issue #13034: When decoding some SSL certificates, the subjectAltName + extension could be unreported. -- Issue #5421: Fix misleading error message when one of socket.sendto()'s - arguments has the wrong type. Patch by Nikita Vetoshkin. +- Issue #9871: Prevent IDLE 3 crash when given byte stings + with invalid hex escape sequences, like b'\x0'. + (Original patch by Claudiu Popa.) -- Issue #11401: fix handling of headers with no value; this fixes a regression - relative to Python2 and the result is now the same as it was in Python2. +- Issue #8933: distutils' PKG-INFO files will now correctly report + Metadata-Version: 1.1 instead of 1.0 if a Classifier or Download-URL field is + present. -- Issue #9298: base64 bodies weren't being folded to line lengths less than 78, - which was a regression relative to Python2. Unlike Python2, the last line - of the folded body now ends with a carriage return. +- Issue #9561: distutils now reads and writes egg-info files using UTF-8, + instead of the locale encoding. -- Issue #11569: use absolute path to the sysctl command in multiprocessing to - ensure that it will be found regardless of the shell PATH. This ensures - that multiprocessing.cpu_count works on default installs of MacOSX. +- Issue #12888: Fix a bug in HTMLParser.unescape that prevented it to escape + more than 128 entities. Patch by Peter Otten. -- Issue #11501: disutils.archive_utils.make_zipfile no longer fails if zlib is - not installed. Instead, the zipfile.ZIP_STORED compression is used to create - the ZipFile. Patch by Natalia B. Bidart. +- Issue #12878: Expose a __dict__ attribute on io.IOBase and its subclasses. -- Issue #11491: dbm.error is no longer raised when dbm.open is called with - the "n" as the flag argument and the file exists. The behavior matches - the documentation and general logic. +- Issue #12636: IDLE reads the coding cookie when executing a Python script. -- Issue #11131: Fix sign of zero in decimal.Decimal plus and minus - operations when the rounding mode is ROUND_FLOOR. +- Issue #12847: Fix a crash with negative PUT and LONG_BINPUT arguments in + the C pickle implementation. -- Issue #5622: Fix curses.wrapper to raise correct exception if curses - initialization fails. +- Issue #11564: Avoid crashes when trying to pickle huge objects or containers + (more than 2**31 items). Instead, in most cases, an OverflowError is raised. -- Issue #11391: Writing to a mmap object created with - ``mmap.PROT_READ|mmap.PROT_EXEC`` would segfault instead of raising a - TypeError. Patch by Charles-François Natali. +- Issue #12287: Fix a stack corruption in ossaudiodev module when the FD is + greater than FD_SETSIZE. -- Issue #11265: asyncore now correctly handles EPIPE, EBADF and EAGAIN errors - on accept(), send() and recv(). +- Issue #11657: Fix sending file descriptors over 255 over a multiprocessing + Pipe. -- Issue #10276: Fix the results of zlib.crc32() and zlib.adler32() on buffers - larger than 4GB. Patch by Nadeem Vawda. +- Issue #13007: whichdb should recognize gdbm 1.9 magic numbers. -- Issue #4681: Allow mmap() to work on file sizes and offsets larger than - 4GB, even on 32-bit builds. Initial patch by Ross Lagerwall, adapted for - 32-bit Windows. +- Issue #12213: Fix a buffering bug with interleaved reads and writes that + could appear on BufferedRandom streams. -- email.header.Header was incorrectly encoding folding white space when - rfc2047-encoding header values with embedded newlines, leaving them - without folding whitespace. It now uses the continuation_ws, as it - does for continuation lines that it creates itself. +- Issue #12650: Fix a race condition where a subprocess.Popen could leak + resources (FD/zombie) when killed at the wrong time. -- Issue #10360: In WeakSet, do not raise TypeErrors when testing for - membership of non-weakrefable objects. +- Issue #10860: http.client now correctly handles an empty port after port + delimiter in URLs. -- Issue #10549: Fix pydoc traceback when text-documenting certain classes. +Build +----- -- Issue #11110: Fix _sqlite to not deref a NULL when module creation fails. +- Issue #13326: Clean __pycache__ directories correctly on OpenBSD. -- Issue #11089: Fix performance issue limiting the use of ConfigParser() - with large config files. +Tools/Demos +----------- -- Issue #8275: Fix passing of callback arguments with ctypes under Win64. - Patch by Stan Mihai. +- Issue #13628: python-gdb.py is now able to retrieve more frames in the Python + traceback if Python is optimized. -- Issue #11053: Fix IDLE "Syntax Error" windows to behave as in 2.x, - preventing a confusing hung appearance on OS X with the windows - obscured. +Tests +----- -- Issue #11052: Correct IDLE menu accelerators on Mac OS X for Save - commands. +- Issue #13786: Remove unimplemented 'trace' long option from regrtest.py. -- Issue #11020: Command-line pyclbr was broken because of missing 2-to-3 - conversion. +- Issue #13725: Fix regrtest to recognize the documented -d flag. + Patch by Erno Tukia. -- Issue #10974: IDLE no longer crashes if its recent files list includes files - with non-ASCII characters in their path names. +- Issue #13304: Skip test case if user site-packages disabled (-s or + PYTHONNOUSERSITE). (Patch by Carl Meyer) -- Issue #10987: Fix the recursion limit handling in the _pickle module. +- Issue #13218: Fix test_ssl failures on Debian/Ubuntu. -- Issue #10949: Improved robustness of rotating file handlers. +- Issue #12821: Fix test_fcntl failures on OpenBSD 5. -- Issue #10955: Fix a potential crash when trying to mmap() a file past its - length. Initial patch by Ross Lagerwall. +- Re-enable lib2to3's test_parser.py tests, though with an expected failure + (see issue 13125). -- Issue #10898: Allow compiling the posix module when the C library defines - a symbol named FSTAT. +Extension Modules +----------------- -- Issue #10916: mmap should not segfault when a file is mapped using 0 as - length and a non-zero offset, and an attempt to read past the end of file - is made (IndexError is raised instead). Patch by Ross Lagerwall. +- Issue #13840: The error message produced by ctypes.create_string_buffer + when given a Unicode string has been fixed. -- Issue #10899: No function type annotations in the standard library. - Removed function type annotations from _pyio.py. +- Issue #9975: socket: Fix incorrect use of flowinfo and scope_id. Patch by + Vilmos Nebehaj. -- Issue #10875: Update Regular Expression HOWTO; patch by 'SilentGhost'. +- Issue #13159: FileIO and BZ2File now use a linear-time buffer growth + strategy instead of a quadratic-time one. -- Issue #10869: Fixed bug where ast.increment_lineno modified the root - node twice. +- Issue #13070: Fix a crash when a TextIOWrapper caught in a reference cycle + would be finalized after the reference to its underlying BufferedRWPair's + writer got cleared by the GC. -- Issue #5871: email.header.Header.encode now raises an error if any - continuation line in the formatted value has no leading white space - and looks like a header. Since Generator uses Header to format all - headers, this check is made for all headers in any serialized message - at serialization time. This provides protection against header - injection attacks. +- Issue #12881: ctypes: Fix segfault with large structure field names. -- Issue #7858: Raise an error properly when os.utime() fails under Windows - on an existing file. +- Issue #13058: ossaudiodev: fix a file descriptor leak on error. Patch by + Thomas Jarosch. -- Issue #3839: wsgiref should not override a Content-Length header set by - the application. Initial patch by Clovis Fabricio. +- Issue #13013: ctypes: Fix a reference leak in PyCArrayType_from_ctype. + Thanks to Suman Saha for finding the bug and providing a patch. -- Issue #10790: email.header.Header.append's charset logic now works correctly - for charsets whose output codec is different from its input codec. +- Issue #13022: Fix: _multiprocessing.recvfd() doesn't check that + file descriptor was actually received. -- Issue #6643: Reinitialize locks held within the threading module after fork - to avoid a potential rare deadlock or crash on some platforms. +- Issue #12483: ctypes: Fix a crash when the destruction of a callback + object triggers the garbage collector. -- Issue #10806, issue #9905: Fix subprocess pipes when some of the standard - file descriptors (0, 1, 2) are closed in the parent process. Initial - patch by Ross Lagerwall. +- Issue #12950: Fix passing file descriptors in multiprocessing, under + OpenIndiana/Illumos. -- Issue #10753 - Characters ';', '=' and ',' in the PATH_INFO environment - variable won't be quoted when the URI is constructed by the wsgiref.util's - request_uri method. According to RFC 3986, these characters can be a part of - params in PATH component of URI and need not be quoted. +Documentation +------------- -- Issue #10738: Fix webbrowser.Opera.raise_opts +- Issue #13883: PYTHONCASEOK also used on OS X and OS/2. -- Issue #9824: SimpleCookie now encodes , and ; in values to cater to how - browsers actually parse cookies. +- Issue #12949: Document the kwonlyargcount argument for the PyCode_New + C API function. -- Issue #5258/#10642: if site.py encounters a .pth file that generates an error, - it now prints the filename, line number, and traceback to stderr and skips - the rest of that individual file, instead of stopping processing entirely. +- Issue #2134: The tokenize documentation has been clarified to explain why + all operator and delimiter tokens are treated as token.OP tokens. + +- Issue #13513: Fix io.IOBase documentation to correctly link to the + io.IOBase.readline method instead of the readline module. + +- Issue #13237: Reorganise subprocess documentation to emphasise convenience + functions and the most commonly needed arguments to Popen. + +- Issue #13141: Demonstrate recommended style for socketserver examples. + + +What's New in Python 3.2.2? +=========================== + +*Release date: 03-Sep-2011* + +Core and Builtins +----------------- + +- Issue #12326: sys.platform is now always 'linux2' on Linux, even if Python + is compiled on Linux 3. + +- Accept bytes for the AST string type. This is temporary until a proper fix in + 3.3. + +Library +------- + +- Issue #8286: The distutils command sdist will print a warning message instead + of crashing when an invalid path is given in the manifest template. + +- Issue #12841: tarfile unnecessarily checked the existence of numerical user + and group ids on extraction. If one of them did not exist the respective id + of the current user (i.e. root) was used for the file and ownership + information was lost. + +- Issue #10946: The distutils commands bdist_dumb, bdist_wininst and bdist_msi + now respect a --skip-build option given to bdist. + +- Issue #12839: Fix crash in zlib module due to version mismatch. + Fix by Richard M. Tew. + +Extension Modules +----------------- + +- Issue #9651: Fix a crash when ctypes.create_string_buffer(0) was passed to + some functions like file.write(). + + + +What's New in Python 3.2.2 release candidate 1? +=============================================== + +*Release date: 14-Aug-2011* + +Core and Builtins +----------------- + +- Issue #12732: In narrow unicode builds, allow Unicode identifiers which fall + outside the BMP. + +- Issue #11603: Fix a crash when __str__ is rebound as __repr__. Patch by + Andreas Stührk. + +- Issue #11321: Fix a crash with multiple imports of the _pickle module when + embedding Python. Patch by Andreas Stührk. + +- Verify the types of AST strings and identifiers provided by the user before + compiling them. + +- Issue #12579: str.format_map() now raises a ValueError if used on a + format string that contains positional fields. Initial patch by + Julian Berman. + +- Issue #11627: Fix segfault when __new__ on a exception returns a + non-exception class. + +- Issue #12149: Update the method cache after a type's dictionary gets + cleared by the garbage collector. This fixes a segfault when an instance + and its type get caught in a reference cycle, and the instance's + deallocator calls one of the methods on the type (e.g. when subclassing + IOBase). Diagnosis and patch by Davide Rizzo. + +- When a generator yields, do not retain the caller's exception state on the + generator. + +- Issue #12475: Prevent generators from leaking their exception state into the + caller's frame as they return for the last time. + +Library +------- + +- Issue #11513: Fix exception handling ``tarfile.TarFile.gzopen()`` when + the file cannot be opened. + +- Issue #12687: Fix a possible buffering bug when unpickling text mode + (protocol 0, mostly) pickles. + +- Issue #10087: Fix the html output format of the calendar module. + +- Issue #12540: Prevent zombie IDLE processes on Windows due to changes + in os.kill(). + +- Issue #12683: urlparse updated to include svn as schemes that uses relative + paths. (svn from 1.5 onwards support relative path). + +- Issues #11104, #8688: Fix the behavior of distutils' sdist command with + manually-maintained MANIFEST files. + +- Issue #12464: tempfile.TemporaryDirectory.cleanup() should not follow + symlinks: fix it. Patch by Petri Lehtinen. + +- Issue #8887: "pydoc somebuiltin.somemethod" (or help('somebuiltin.somemethod') + in Python code) now finds the doc of the method. + +- Issue #12603: Fix pydoc.synopsis() on files with non-negative st_mtime. + +- Issue #12514: Use try/finally to assure the timeit module restores garbage + collections when it is done. + +- Issue #12607: In subprocess, fix issue where if stdin, stdout or stderr is + given as a low fd, it gets overwritten. + +- Issue #12590: IDLE editor window now always displays the first line + when opening a long file. With Tk 8.5, the first line was hidden. + +- Issue #12576: Fix urlopen behavior on sites which do not send (or obfuscates) + Connection:close header. + +- Issue #1813: Fix codec lookup under Turkish locales. + +- Issue #12591: Improve support of "universal newlines" in the subprocess + module: the piped streams can now be properly read from or written to. + +- Issue #12591: Allow io.TextIOWrapper to work with raw IO objects (without + a read1() method), and add an undocumented *write_through* parameter to + mandate unbuffered writes. + +- Issue #9611, #9015: FileIO.read() clamps the length to INT_MAX on Windows. + +- Issue #10883: Fix socket leaks in urllib.request when using FTP. + +- Issue #12571: Add a plat-linux3 directory mirroring the plat-linux2 + directory, so that "import DLFCN" and other similar imports work on + Linux 3.0. + +- Issue #7484: smtplib no longer puts <> around addresses in VRFY and EXPN + commands; they aren't required and in fact postfix doesn't support that form. + +- Close the call queue in concurrent.futures.ProcessPoolExecutor when + shutdown() is called, without waiting for the garbage collector to kick in. + +- Issue #12502: asyncore: fix polling loop with AF_UNIX sockets. + +- Issue #4376: ctypes now supports nested structures with an endianness + different than that of the parent structure. Patch by Vlad Riscutia. + +- Raise ValueError when attempting to set the _CHUNK_SIZE attribute of a + TextIOWrapper to a huge value, not TypeError. + +- Issue #12493: subprocess: Popen.communicate() now also handles EINTR errors + if the process has only one pipe. + +- Issue #12451: pydoc: html_getfile() now uses tokenize.open() to support + Python modules using a encoding different than UTF-8 (reading the coding + cookie of the module). + +- Issue #12451: pydoc: importfile() now opens the Python module in binary mode, + instead of text mode using the locale encoding, to avoid encoding issues. + +- Issue #12451: runpy: run_path() now opens the Python module in binary mode, + instead of text mode using the locale encoding, to support other encodings + than UTF-8 (modules using the coding cookie). + +- Issue #12451: xml.dom.pulldom: parse() now opens files in binary mode instead + of the text mode (using the locale encoding) to avoid encoding issues. + +Extension Modules +----------------- + +- Issue #12764: Fix a crash in ctypes when the name of a Structure field is not + a string. + +- Issue #11241: subclasses of ctypes.Array can now be subclassed. + +- Issue #10309: Define _GNU_SOURCE so that mremap() gets the proper + signature. Without this, architectures where sizeof void* != sizeof int are + broken. Patch given by Hallvard B Furuseth. + +C-API +----- + +Build +----- + +- Issue #12560: Build libpython.so on OpenBSD. Patch by Stefan Sperling. + +- Issue #12592: Make Python build on OpenBSD 5 (and future major releases). + +- Issue #12372: POSIX semaphores are broken on AIX: don't use them. + +Tools/Demos +----------- + +- Issue #10639: reindent.py no longer converts newlines and will raise + an error if attempting to convert a file with mixed newlines. + +Tests +----- + +- Issue #12331: The test suite for lib2to3 can now run from an installed + Python. + +- Issue #12626: In regrtest, allow to filter tests using a glob filter + with the ``-m`` (or ``--match``) option. This works with all test cases + using the unittest module. This is useful with long test suites + such as test_io or test_subprocess. + +- Issue #12624: It is now possible to fail after the first failure when + running in verbose mode (``-v`` or ``-W``), by using the ``--failfast`` + (or ``-G``) option to regrtest. This is useful with long test suites + such as test_io or test_subprocess. + +- Issue #12587: Correct faulty test file and reference in test_tokenize. + (Patch by Robert Xiao) + +- Try harder to reap dangling threads in test.support.reap_threads(). + +- Issue #12573: Add resource checks for dangling Thread and Process objects. + +- Issue #12549: Correct test_platform to not fail when OS X returns 'x86_64' + as the processor type on some Mac systems. + +- Avoid failing in test_robotparser when mueblesmoraleda.com is flaky and + an overzealous DNS service (e.g. OpenDNS) redirects to a placeholder + Web site. + +- Issue #12440: When testing whether some bits in SSLContext.options can be + reset, check the version of the OpenSSL headers Python was compiled against, + rather than the runtime version of the OpenSSL library. + +- Issue #12497: Install test/data to prevent failures of the various codecmaps + tests. + +- Issue #12496: Install test/capath directory to prevent test_connect_capath + testcase failure in test_ssl. + +- Issue #12469: Run "wakeup" signal tests in subprocess to run the test in a + fresh process with only one thread and to not change signal handling of the + parent process. + +- Issue #8716: Avoid crashes caused by Aqua Tk on OSX when attempting to run + test_tk or test_ttk_guionly under a username that is not currently logged + in to the console windowserver (as may be the case under buildbot or ssh). + + +What's New in Python 3.2.1? +=========================== + +*Release date: 10-Jul-2011* + +Library +------- + +- Issue #12467: warnings: fix a race condition if a warning is emitted at + shutdown, if globals()['__file__'] is None. + +Tests +----- + +- Skip network tests when getaddrinfo() returns EAI_AGAIN, meaning a temporary + failure in name resolution. + +- Issue #11812: Solve transient socket failure to connect to 'localhost' + in test_telnetlib.py. + +- Solved a potential deadlock in test_telnetlib.py. Related to issue #11812. + +- Avoid failing in test_urllibnet.test_bad_address when some overzealous + DNS service (e.g. OpenDNS) resolves a non-existent domain name. The test + is now skipped instead. + + +What's New in Python 3.2.1 release candidate 2? +=============================================== + +*Release date: 03-Jul-2011* + +Core and Builtins +----------------- + +- Issue #12291: You can now load multiple marshalled objects from a stream, with + other data interleaved between marshalled objects. + +- Issue #12084: os.stat on Windows now works properly with relative symbolic + links when called from any directory. + +- Issue #1195: my_fgets() now always clears errors before calling fgets(). Fix + the following case: sys.stdin.read() stopped with CTRL+d (end of file), + raw_input() interrupted by CTRL+c. + +- Issue #9670: Increase the default stack size for secondary threads on Mac OS X + and FreeBSD to reduce the chances of a crash instead of a "maximum recursion + depth" RuntimeError exception (patch by Ronald Oussoren). + +Library +------- + +- Issue #12147: Adjust the new-in-3.2 smtplib.send_message method for better + conformance to the RFCs: correctly handle Sender and Resent headers. + +- Issue #12352: Fix a deadlock in multiprocessing.Heap when a block is freed by + the garbage collector while the Heap lock is held. + +- Issue #12451: The XInclude default loader of xml.etree now decodes files from + UTF-8 instead of the locale encoding if the encoding is not specified. It now + also opens XML files for the parser in binary mode instead of the text mode to + avoid encoding issues. + +- Issue #12451: doctest.debug_script() doesn't create a temporary file anymore + to avoid encoding issues. + +- Issue #12451: pydoc.synopsis() now reads the encoding cookie if available, to + read the Python module from the right encoding. + +- Issue #12451: distutils now opens the setup script in binary mode to read the + encoding cookie, instead of opening it in UTF-8. + +- Issue #9516: On Mac OS X, change Distutils to no longer globally attempt to + check or set the MACOSX_DEPLOYMENT_TARGET environment variable for the + interpreter process. This could cause failures in non-Distutils subprocesses + and was unreliable since tests or user programs could modify the interpreter + environment after Distutils set it. Instead, have Distutils set the the + deployment target only in the environment of each build subprocess. It is + still possible to globally override the default by setting + MACOSX_DEPLOYMENT_TARGET before launching the interpreter; its value must be + greater or equal to the default value, the value with which the interpreter + was built. + +- Issue #12383: Fix subprocess module with env={}: don't copy the environment + variables, start with an empty environment. + +- Issue #11584: email.header.decode_header no longer fails if the header passed + to it is a Header object, and Header/make_header no longer fail if given + binary unknown-8bit input. + +- Issue #11700: mailbox proxy object close methods can now be called multiple + times without error. + +- Issue #11767: Correct file descriptor leak in mailbox's __getitem__ method. + +- Issue #12133: AbstractHTTPHandler.do_open() of urllib.request closes the HTTP + connection if its getresponse() method fails with a socket error. Patch + written by Ezio Melotti. + +- Issue #9284: Allow inspect.findsource() to find the source of doctest + functions. + +- Issue #12009: Fixed regression in netrc file comment handling. + +- Issue #10694: zipfile now ignores garbage at the end of a zipfile. + +- Issue #12283: Fixed regression in smtplib quoting of leading dots in DATA. + +- Issue #12168: SysLogHandler now allows NUL termination to be controlled using + a new 'append_nul' attribute on the handler. + +- Issue #11583: Speed up os.path.isdir on Windows by using GetFileAttributes + instead of os.stat. + +- Named tuples now work correctly with vars(). + +- Issue #12085: Fix an attribute error in subprocess.Popen destructor if the + constructor has failed, e.g. because of an undeclared keyword argument. Patch + written by Oleg Oshmyan. + +- Issue #985064: Make plistlib more resilient to faulty input plists. Patch by + Mher Movsisyan. + +- Issue #12175: RawIOBase.readall() now returns None if read() returns None. + +- Issue #12175: FileIO.readall() now raises a ValueError instead of an IOError + if the file is closed. + +- Issue #12070: Fix the Makefile parser of the sysconfig module to handle + correctly references to "bogus variable" (e.g. "prefix=$/opt/python"). + +- Issue #12100: Don't reset incremental encoders of CJK codecs at each call to + their encode() method anymore, but continue to call the reset() method if the + final argument is True. + +- Issue #5715: In socketserver, close the server socket in the child process. + +- Correct lookup of __dir__ on objects. Among other things, this causes errors + besides AttributeError found on lookup to be propagated. + +- Issue #12124: zipimport doesn't keep a reference to zlib.decompress() anymore + to be able to unload the module. + +- Issue #12065: connect_ex() on an SSL socket now returns the original errno + when the socket's timeout expires (it used to return None). + +Extension Modules +----------------- + +- Issue #12404: Remove C89 incompatible code from mmap module. Patch by Akira + Kitada. + +- Issue #12221: Replace pyexpat.__version__ with the Python version. + +Build +----- + +- Issue #8746: Correct faulty configure checks so that os.chflags() and + os.lchflags() are once again built on systems that support these functions + (*BSD and OS X). Also add new stat file flags for OS X (UF_HIDDEN and + UF_COMPRESSED). + +- Issue #11217: For 64-bit/32-bit Mac OS X universal framework builds, ensure + "make install" creates symlinks in --prefix bin for the "-32" files in the + framework bin directory like the installer does. + + +Tests +----- + +- Issue #12407: Explicitly skip test_capi.EmbeddingTest under Windows. + +- Issue #12400: regrtest -W doesn't rerun the tests twice anymore, but captures + the output and displays it on failure instead. regrtest -v doesn't print the + error twice anymore if there is only one error. + +- Issue #12141: Install a copy of template C module file so that test_build_ext + of test_distutils is no longer silently skipped when run outside of a build + directory. + +- Issue #8746: Add additional tests for os.chflags() and os.lchflags(). Patch + by Garrett Cooper. + +- Issue #10736: Fix test_ttk test_widgets failures with Cocoa Tk 8.5.9 on Mac + OS X. (Patch by Ronald Oussoren) + +- Issue #12057: Add tests for ISO 2022 codecs (iso2022_jp, iso2022_jp_2, + iso2022_kr). + + +What's New in Python 3.2.1 release candidate 1? +=============================================== + +*Release date: 15-May-2011* + +Core and Builtins +----------------- + +- Issue #12060: Use sig_atomic_t type and volatile keyword in the signal + module. Patch written by Charles-François Natali. + +- Issue #12044: Fixed subprocess.Popen when used as a context manager to + wait for the process to end when exiting the context to avoid unintentionally + leaving zombie processes around. + +- Issue #1195: Fix input() if it is interrupted by CTRL+d and then CTRL+c, + clear the end-of-file indicator after CTRL+d. + +Library +------- + +- Issue #11088: don't crash when using F5 to run a script in IDLE on MacOSX + with Tk 8.5. + +- Issue #9516: Issue #9516: avoid errors in sysconfig when MACOSX_DEPLOYMENT_TARGET + is set in shell. + +- Issue #12012: ssl.PROTOCOL_SSLv2 becomes optional. + +- Issue #8650: Make zlib module 64-bit clean. compress(), decompress() and + their incremental counterparts now raise OverflowError if given an input + larger than 4GB, instead of silently truncating the input and returning + an incorrect result. + +- Issue #12050: zlib.decompressobj().decompress() now clears the unconsumed_tail + attribute when called without a max_length argument. + +- Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence + on a file opened in read+write mode (namely: reading, seeking a bit forward, + writing, then seeking before the previous write but still within buffered + data, and writing again). + +- Issue #1028: Tk returns invalid Unicode null in %A: UnicodeDecodeError. + With Tk < 8.5 _tkinter.c:PythonCmd() raised UnicodeDecodeError, caused + IDLE to exit. Converted to valid Unicode null in PythonCmd(). + +- Issue #11169: compileall module uses repr() to format filenames and paths to + escape surrogate characters and show spaces. + +- Issue #10419, #6011: build_scripts command of distutils handles correctly + non-ASCII path (path to the Python executable). Open and write the script in + binary mode, but ensure that the shebang is decodable from UTF-8 and from the + encoding of the script. + +- Issue #8498: In socket.accept(), allow to specify 0 as a backlog value in + order to accept exactly one connection. Patch by Daniel Evers. + +- Issue #11164: Stop trying to use _xmlplus in the xml module. + +Build +----- + +- Issue #11347: Use --no-as-needed when linking libpython3.so. + +Tools/Demos +----------- + +- Issue #11996: libpython (gdb), replace "py-bt" command by "py-bt-full" and + add a smarter "py-bt" command printing a classic Python traceback. + +Tests +----- + +- Issue #12096: Fix a race condition in test_threading.test_waitfor(). Patch + written by Charles-François Natali. + +- Issue #11614: import __hello__ prints "Hello World!". Patch written by + Andreas Stührk. + +- Issue #5723: Improve json tests to be executed with and without accelerations. + +- Issue #11910: Fix test_heapq to skip the C tests when _heapq is missing. + + +What's New in Python 3.2.1 beta 1? +================================== + +*Release date: 08-May-2011* + +Core and Builtins +----------------- + +- Issue #1856: Avoid crashes and lockups when daemon threads run while the + interpreter is shutting down; instead, these threads are now killed when they + try to take the GIL. + +- Issue #9756: When calling a method descriptor or a slot wrapper descriptor, + the check of the object type doesn't read the __class__ attribute anymore. + Fix a crash if a class override its __class__ attribute (e.g. a proxy of the + str type). Patch written by Andreas Stührk. + +- Issue #10914: Initialize correctly the filesystem codec when creating a new + subinterpreter to fix a bootstrap issue with codecs implemented in Python, as + the ISO-8859-15 codec. + +- Issue #10517: After fork(), reinitialize the TLS used by the PyGILState_* + APIs, to avoid a crash with the pthread implementation in RHEL 5. Patch by + Charles-François Natali. + +- Issue #6780: fix starts/endswith error message to mention that tuples are + accepted too. + +- Issue #5057: fix a bug in the peepholer that led to non-portable pyc files + between narrow and wide builds while optimizing BINARY_SUBSCR on non-BMP chars + (e.g. "\U00012345"[0]). + +- Issue #11845: Fix typo in rangeobject.c that caused a crash in + compute_slice_indices. Patch by Daniel Urban. + +- Issue #11650: PyOS_StdioReadline() retries fgets() if it was interrupted + (EINTR), for example if the program is stopped with CTRL+z on Mac OS X. Patch + written by Charles-Francois Natali. + +- Issue #11395: io.FileIO().write() clamps the data length to 32,767 bytes on + Windows if the file is a TTY to workaround a Windows bug. The Windows console + returns an error (12: not enough space error) on writing into stdout if stdout + mode is binary and the length is greater than 66,000 bytes (or less, depending + on heap usage). + +- Issue #11320: fix bogus memory management in Modules/getpath.c, leading to a + possible crash when calling Py_SetPath(). + +- Issue #11510: Fixed optimizer bug which turned "a,b={1,1}" into "a,b=(1,1)". + +- Issue #11432: A bug was introduced in subprocess.Popen on posix systems with + 3.2.0 where the stdout or stderr file descriptor being the same as the stdin + file descriptor would raise an exception. webbrowser.open would fail. fixed. + +- Issue #11450: Don't truncate hg version info in Py_GetBuildInfo() when there + are many tags (e.g. when using mq). Patch by Nadeem Vawda. + +- Issue #11246: Fix PyUnicode_FromFormat("%V") to decode the byte string from + UTF-8 (with replace error handler) instead of ISO-8859-1 (in strict mode). + Patch written by Ray Allen. + +- Issue #11286: Raise a ValueError from calling PyMemoryView_FromBuffer with a + buffer struct having a NULL data pointer. + +- Issue #11272: On Windows, input() strips '\r' (and not only '\n'), and + sys.stdin uses universal newline (replace '\r\n' by '\n'). + +- issue #11828: startswith and endswith don't accept None as slice index. Patch + by Torsten Becker. + +- Issue #10830: Fix PyUnicode_FromFormatV("%c") for non-BMP characters on + narrow build. + +- Check for NULL result in PyType_FromSpec. + +- Issue #11386: bytearray.pop() now throws IndexError when the bytearray is + empty, instead of OverflowError. + +Library +------- + +- Issue #11927: SMTP_SSL now uses port 465 by default as documented. Patch by + Kasun Herath. + +- Issue #12002: ftplib's abort() method raises TypeError. + +- Issue #11999: fixed sporadic sync failure mailbox.Maildir due to its trying to + detect mtime changes by comparing to the system clock instead of to the + previous value of the mtime. + +- ntpath.samefile failed to notice that "a.txt" and "A.TXT" refer to the same + file on Windows XP. As noticed in issue #10684. + +- Issue #12000: When a SSL certificate has a subjectAltName without any dNSName + entry, ssl.match_hostname() should use the subject's commonName. Patch by + Nicolas Bareil. + +- Issue #11647: objects created using contextlib.contextmanager now support more + than one call to the function when used as a decorator. Initial patch by Ysj + Ray. + +- logging: don't define QueueListener if Python has no thread support. + +- functools.cmp_to_key() now works with collections.Hashable(). + +- Issue #11277: mmap.mmap() calls fcntl(fd, F_FULLFSYNC) on Mac OS X to get + around a mmap bug with sparse files. Patch written by Steffen Daode Nurpmeso. + +- Issue #11858: configparser.ExtendedInterpolation expected lower-case section + names. + +- Issue #11324: ConfigParser(interpolation=None) now works correctly. + +- Issue #11763: don't use difflib in TestCase.assertMultiLineEqual if the + strings are too long. + +- Issue #11236: getpass.getpass responds to ctrl-c or ctrl-z on terminal. + +- Issue #11768: The signal handler of the signal module only calls + Py_AddPendingCall() for the first signal to fix a deadlock on reentrant or + parallel calls. PyErr_SetInterrupt() writes also into the wake up file. + +- Issue #11492: fix several issues with header folding in the email package. + +- Issue #11852: Add missing imports and update tests. + +- Issue #11875: collections.OrderedDict's __reduce__ was temporarily + mutating the object instead of just working on a copy. + +- Issue #11467: Fix urlparse behavior when handling urls which contains scheme + specific part only digits. Patch by Santoso Wijaya. + +- collections.Counter().copy() now works correctly for subclasses. + +- Issue #11474: Fix the bug with url2pathname() handling of '/C|/' on Windows. + Patch by Santoso Wijaya. + +- Issue #9233: Fix json.loads('{}') to return a dict (instead of a list), when + _json is not available. + +- Issue #11830: Remove unnecessary introspection code in the decimal module. + +- Issue #11703: urllib2.geturl() does not return correct url when the original + url contains #fragment. + +- Issue #10019: Fixed regression in json module where an indent of 0 stopped + adding newlines and acted instead like 'None'. + +- Issue #5162: Treat services like frozen executables to allow child spawning + from multiprocessing.forking on Windows. + +- Issue #11814: Fix likely typo in multiprocessing.Pool._terminate(). + +- Issue #11747: Fix range formatting in difflib.context_diff() and + difflib.unified_diff(). + +- Issue #8428: Fix a race condition in multiprocessing.Pool when terminating + worker processes: new processes would be spawned while the pool is being shut + down. Patch by Charles-François Natali. + +- Issue #7311: fix html.parser to accept non-ASCII attribute values. + +- Issue #11605: email.parser.BytesFeedParser was incorrectly converting + multipart subparts with an 8-bit CTE into unicode instead of preserving the + bytes. + +- Issue #10963: Ensure that subprocess.communicate() never raises EPIPE. + +- Issue #11746: Fix SSLContext.load_cert_chain() to accept elliptic curve + private keys. + +- sys.getfilesystemencoding() raises a RuntimeError if initfsencoding() was not + called yet: detect bootstrap (startup) issues earlier. + +- Issue #11618: Fix the timeout logic in threading.Lock.acquire() under Windows. + +- Issue #11256: Fix inspect.getcallargs on functions that take only keyword + arguments. + +- Issue #11696: Fix ID generation in msilib. + +- Issue #9696: Fix exception incorrectly raised by xdrlib.Packer.pack_int when + trying to pack a negative (in-range) integer. + +- Issue #11675: multiprocessing.[Raw]Array objects created from an integer size + are now zeroed on creation. This matches the behaviour specified by the + documentation. + +- Issue #7639: Fix short file name generation in bdist_msi + +- Issue #11659: Fix ResourceWarning in test_subprocess introduced by #11459. + Patch by Ben Hayden. + +- Issue #11635: Don't use polling in worker threads and processes launched by + concurrent.futures. + +- Issue #11628: cmp_to_key generated class should use __slots__ + +- Issue #11666: let help() display named tuple attributes and methods + that start with a leading underscore. + +- Issue #11662: Make urllib and urllib2 ignore redirections if the + scheme is not HTTP, HTTPS or FTP (CVE-2011-1521). + +- Issue #5537: Fix time2isoz() and time2netscape() functions of + httplib.cookiejar for expiration year greater than 2038 on 32-bit systems. + +- Issue #11563: Connection:close header is sent by requests using URLOpener + class which helps in closing of sockets after connection is over. Patch + contributions by Jeff McNeil and Nadeem Vawda. + +- Issue #11459: A ``bufsize`` value of 0 in subprocess.Popen() really creates + unbuffered pipes, such that select() works properly on them. + +- Issue #5421: Fix misleading error message when one of socket.sendto()'s + arguments has the wrong type. Patch by Nikita Vetoshkin. + +- Issue #10979: unittest stdout buffering now works with class and module + setup and teardown. + +- Issue #11577: fix ResourceWarning triggered by improved binhex test coverage + +- Issue #11243: fix the parameter querying methods of Message to work if + the headers contain un-encoded non-ASCII data. + +- Issue #11401: fix handling of headers with no value; this fixes a regression + relative to Python2 and the result is now the same as it was in Python2. + +- Issue #9298: base64 bodies weren't being folded to line lengths less than 78, + which was a regression relative to Python2. Unlike Python2, the last line of + the folded body now ends with a carriage return. + +- Issue #11560: shutil.unpack_archive now correctly handles the format + parameter. Patch by Evan Dandrea. + +- Issue #11133: fix two cases where inspect.getattr_static can trigger code + execution. Patch by Andreas Stührk. + +- Issue #11569: use absolute path to the sysctl command in multiprocessing to + ensure that it will be found regardless of the shell PATH. This ensures that + multiprocessing.cpu_count works on default installs of MacOSX. + +- Issue #11501: disutils.archive_utils.make_zipfile no longer fails if zlib is + not installed. Instead, the zipfile.ZIP_STORED compression is used to create + the ZipFile. Patch by Natalia B. Bidart. + +- Issue #11554: Fixed support for Japanese codecs; previously the body output + encoding was not done if euc-jp or shift-jis was specified as the charset. + +- Issue #11500: Fixed a bug in the os x proxy bypass code for fully qualified IP + addresses in the proxy exception list. + +- Issue #11491: dbm.error is no longer raised when dbm.open is called with the + "n" as the flag argument and the file exists. The behavior matches the + documentation and general logic. + +- Issue #11131: Fix sign of zero in decimal.Decimal plus and minus operations + when the rounding mode is ROUND_FLOOR. + +- Issue #5622: Fix curses.wrapper to raise correct exception if curses + initialization fails. + +- Issue #11391: Writing to a mmap object created with + ``mmap.PROT_READ|mmap.PROT_EXEC`` would segfault instead of raising a + TypeError. Patch by Charles-François Natali. + +- Issue #11306: mailbox in certain cases adapts to an inability to open certain + files in read-write mode. Previously it detected this by checking for EACCES, + now it also checks for EROFS. + +- Issue #11265: asyncore now correctly handles EPIPE, EBADF and EAGAIN errors on + accept(), send() and recv(). + +- Issue #11326: Add the missing connect_ex() implementation for SSL sockets, + and make it work for non-blocking connects. + +- Issue #7322: Trying to read from a socket's file-like object after a timeout + occurred now raises an error instead of silently losing data. + +- Issue #10956: Buffered I/O classes retry reading or writing after a signal + has arrived and the handler returned successfully. + +- Issue #11224: Fixed a regression in tarfile that affected the file-like + objects returned by TarFile.extractfile() regarding performance, memory + consumption and failures with the stream interface. + +- Issue #11074: Make 'tokenize' so it can be reloaded. + +- Issue #4681: Allow mmap() to work on file sizes and offsets larger than + 4GB, even on 32-bit builds. Initial patch by Ross Lagerwall, adapted for + 32-bit Windows. + +- Issue #11089: Fix performance issue limiting the use of ConfigParser() + with large config files. + +- Issue #10276: Fix the results of zlib.crc32() and zlib.adler32() on buffers + larger than 4GB. Patch by Nadeem Vawda. + +- Issue #9348: Raise an early error if argparse nargs and metavar don't match. + +- Issue #8982: Improve the documentation for the argparse Namespace object. + +- Issue #9343: Document that argparse parent parsers must be configured before + their children. + +- Issue #9026: Fix order of argparse sub-commands in help messages. + +- Issue #9347: Fix formatting for tuples in argparse type= error messages. + +Build +----- + +- Issue #11411: Fix 'make DESTDIR=' with a relative destination. + +- Issue #11268: Prevent Mac OS X Installer failure if Documentation package had + previously been installed. + +IDLE +---- + +- Issue #11718: IDLE's open module dialog couldn't find the __init__.py file in + a package. + +Tools/Demos +----------- + +- Issue #11179: Make ccbench work under Python 3.1 and 2.7 again. + +Extension Modules +----------------- + +- Issue #12051: Fix segfault in json.dumps() while encoding highly-nested + objects using the C accelerations. + +- Issue #12017: Fix segfault in json.loads() while decoding highly-nested + objects using the C accelerations. + +- Issue #1838: Prevent segfault in ctypes, when _as_parameter_ on a class is set + to an instance of the class. + +Tests +----- + +- Issue #11873: Change regex in test_compileall to fix occasional failures when + when the randomly generated temporary path happened to match the regex. + +- Issue #10914: Add a minimal embedding test to test_capi. + +- Issue #11790: Fix sporadic failures in + test_multiprocessing.WithProcessesTestCondition. + +- Fix possible "file already exists" error when running the tests in parallel. + +- Issue #11719: Fix message about unexpected test_msilib skip on non-Windows + platforms. Patch by Nadeem Vawda. + +- Issue #11653: fix -W with -j in regrtest. + +- Issue #11577: improve test coverage of binhex.py. Patch by Arkady Koplyarov. + +- Issue #11578: added test for the timeit module. Patch by Michael Henry. + +- Issue #11503: improve test coverage of posixpath.py. Patch by Evan Dandrea. + +- Issue #11505: improves test coverage of string.py. Patch by Alicia + Arlen. + +- Issue #11548: Improve test coverage of the shutil module. Patch by + Evan Dandrea. + +- Issue #11554: Reactivated test_email_codecs. + +- Issue #11490: test_subprocess:test_leaking_fds_on_error no longer gives a + false positive if the last directory in the path is inaccessible. + +- Issue #11223: Fix test_threadsignals to fail, not hang, when the + non-semaphore implementation of locks is used under POSIX. + +- Issue #10911: Add tests on CGI with non-ASCII characters. Patch written by + Pierre Quentel. + +- Issue #9931: Fix hangs in GUI tests under Windows in certain conditions. + Patch by Hirokazu Yamamoto. + +- Issue #10826: Prevent sporadic failure in test_subprocess on Solaris due + to open door files. + +Documentation +------------- + +- Issue #11818: Fix tempfile examples for Python 3. + + +What's New in Python 3.2? +========================= + +*Release date: 20-Feb-2011* + +Core and Builtins +----------------- + +- Issue #11249: Fix potential crashes when using the limited API. + +Build +----- + +- Issue #11222: Fix non-framework shared library build on Mac OS X. + +- Issue #11184: Fix large-file support on AIX. + +- Issue #941346: Fix broken shared library build on AIX. + +Documentation +------------- + +- Issue #10709: Add updated AIX notes in Misc/README.AIX. + + +What's New in Python 3.2 Release Candidate 3? +============================================= + +*Release date: 13-Feb-2011* + +Core and Builtins +----------------- + +- Issue #11134: Add missing fields to typeslots.h. + +- Issue #11135: Remove redundant doc field from PyType_Spec. + +- Issue #11067: Add PyType_GetFlags, to support PyUnicode_Check in the limited + ABI. + +- Issue #11118: Fix bogus export of None in python3.dll. + +Library +------- + +- Issue #11116: any error during addition of a message to a mailbox now causes a + rollback, instead of leaving the mailbox partially modified. + +- Issue #11132: Fix passing of "optimize" parameter when recursing in + compileall.compile_dir(). + +- Issue #11110: Fix a potential decref of a NULL in sqlite3. + +- Issue #8275: Fix passing of callback arguments with ctypes under Win64. Patch + by Stan Mihai. + +Build +----- + +- Issue #11079: The /Applications/Python x.x folder created by the Mac OS X + installers now includes a link to the installed documentation and no longer + includes an Extras directory. The Tools directory is now installed in the + framework under share/doc. + +- Issue #11121: Fix building with --enable-shared. + +Tests +----- + +- Issue #10971: test_zipimport_support is once again compatible with the refleak + hunter feature of test.regrtest. + + +What's New in Python 3.2 Release Candidate 2? +============================================= + +*Release date: 30-Jan-2011* + +Core and Builtins +----------------- + +- Issue #10451: memoryview objects could allow to mutate a readable buffer. + Initial patch by Ross Lagerwall. + +Library +------- + +- Issue #9124: mailbox now accepts binary input and reads and writes mailbox + files in binary mode, using the email package's binary support to parse + arbitrary email messages. StringIO and text file input is deprecated, + and string input fails early if non-ASCII characters are used, where + previously it would fail when the email was processed in a later step. + +- Issue #10845: Mitigate the incompatibility between the multiprocessing + module on Windows and the use of package, zipfile or directory execution + by special casing main modules that actually *are* called __main__.py. + +- Issue #11045: Protect logging call against None argument. + +- Issue #11052: Correct IDLE menu accelerators on Mac OS X for Save + commands. + +- Issue #11053: Fix IDLE "Syntax Error" windows to behave as in 2.x, + preventing a confusing hung appearance on OS X with the windows + obscured. + +- Issue #10940: Workaround an IDLE hang on Mac OS X 10.6 when using the + menu accelerators for Open Module, Go to Line, and New Indent Width. + The accelerators still work but no longer appear in the menu items. + +- Issue #10989: Fix a crash on SSLContext.load_verify_locations(None, True). + +- Issue #11020: Command-line pyclbr was broken because of missing 2-to-3 + conversion. + +- Issue #11019: Fixed BytesGenerator so that it correctly handles a Message + with a None body. + +- Issue #11014: Make 'filter' argument in tarfile.Tarfile.add() into a + keyword-only argument. The preceding positional argument was deprecated, + so it made no sense to add filter as a positional argument. + +- Issue #11004: Repaired edge case in deque.count(). + +- Issue #10974: IDLE no longer crashes if its recent files list includes files + with non-ASCII characters in their path names. + +- Have hashlib.algorithms_available and hashlib.algorithms_guaranteed both + return sets instead of one returning a tuple and the other a frozenset. + +- Issue #10987: Fix the recursion limit handling in the _pickle module. + +- Issue #10983: Fix several bugs making tunnel requests in http.client. + +- Issue #10955: zipimport uses ASCII encoding instead of cp437 to decode + filenames, at bootstrap, if the codec registry is not ready yet. It is still + possible to have non-ASCII filenames using the Unicode flag (UTF-8 encoding) + for all file entries in the ZIP file. + +- Issue #10949: Improved robustness of rotating file handlers. + +- Issue #10955: Fix a potential crash when trying to mmap() a file past its + length. Initial patch by Ross Lagerwall. + +- Issue #10898: Allow compiling the posix module when the C library defines + a symbol named FSTAT. + +- Issue #10980: the HTTP server now encodes headers with iso-8859-1 (latin1) + encoding. This is the preferred encoding of PEP 3333 and the base encoding + of HTTP 1.1. + +- To match the behaviour of HTTP server, the HTTP client library now also + encodes headers with iso-8859-1 (latin1) encoding. It was already doing + that for incoming headers which makes this behaviour now consistent in + both incoming and outgoing direction. + +- Issue #9509: argparse now properly handles IOErrors raised by + argparse.FileType. + +- Issue #10961: The new pydoc server now better handles exceptions raised + during request handling. + +- Issue #10680: Fix mutually exclusive arguments for argument groups in + argparse. + +Build +----- + +- Issue #11054: Allow Mac OS X installer builds to again work on 10.5 with + the system-provided Python. + + +What's New in Python 3.2 Release Candidate 1 +============================================ + +*Release date: 16-Jan-2011* + +Core and Builtins +----------------- + +- Issue #10889: range indexing and slicing now works correctly on ranges with + a length that exceeds sys.maxsize. + +- Issue #10892: Don't segfault when trying to delete __abstractmethods__ from a + class. + +- Issue #8020: Avoid a crash where the small objects allocator would read + non-Python managed memory while it is being modified by another thread. Patch + by Matt Bandy. + +- Issue #10841: On Windows, set the binary mode on stdin, stdout, stderr and all + io.FileIO objects (to not translate newlines, \r\n <=> \n). The Python parser + translates newlines (\r\n => \n). + +- Remove buffer API from stable ABI for now, see #10181. + +- Issue #8651: PyArg_Parse*() functions raise an OverflowError if the file + doesn't have PY_SSIZE_T_CLEAN define and the size doesn't fit in an int + (length bigger than 2^31-1 bytes). + +- Issue #9015, #9611: FileIO.readinto(), FileIO.write(), os.write() and + stdprinter.write() clamp the length to INT_MAX on Windows. + +- Issue #8278: On Windows and with a NTFS filesystem, os.stat() and os.utime() + can now handle dates after 2038. + +- Issue #10780: PyErr_SetFromWindowsErrWithFilename() and + PyErr_SetExcFromWindowsErrWithFilename() decode the filename from the + filesystem encoding instead of UTF-8. + +- Issue #10779: PyErr_WarnExplicit() decodes the filename from the filesystem + encoding instead of UTF-8. + +- Add sys.flags attribute for the new -q command-line option. + +- Issue #11506: Trying to assign to a bytes literal should result in a + SyntaxError. + +Library +------- + +- Issue #10916: mmap should not segfault when a file is mapped using 0 as length + and a non-zero offset, and an attempt to read past the end of file is made + (IndexError is raised instead). Patch by Ross Lagerwall. + +- Issue #10154, #10090: change the normalization of UTF-8 to "UTF-8" instead + of "UTF8" in the locale module as the latter is not supported MacOSX and OpenBSD. + +- Issue #10684: shutil.move used to delete a folder on case insensitive + filesystems when the source and destination name where the same except + for the case. + +- Issue #10907: Warn OS X 10.6 IDLE users to use ActiveState Tcl/Tk 8.5, rather + than the currently problematic Apple-supplied one, when running with the + 64-/32-bit installer variant. + +- Issue #4953: cgi.FieldStorage and cgi.parse() parse the request as bytes, not + as unicode, and accept binary files. Add encoding and errors attributes to + cgi.FieldStorage. Patch written by Pierre Quentel (with many inputs by Glenn + Linderman). + +- Add encoding and errors arguments to urllib.parse_qs() and urllib.parse_qsl(). + +- Issue #10899: No function type annotations in the standard library. Removed + function type annotations from _pyio.py. + +- Issue #10875: Update Regular Expression HOWTO; patch by 'SilentGhost'. + +- Issue #10872: The repr() of TextIOWrapper objects now includes the mode + if available. + +- Issue #10869: Fixed bug where ast.increment_lineno modified the root node + twice. + +- Issue #5871: email.header.Header.encode now raises an error if any + continuation line in the formatted value has no leading white space and looks + like a header. Since Generator uses Header to format all headers, this check + is made for all headers in any serialized message at serialization time. This + provides protection against header injection attacks. + +- Issue #10859: Make ``contextlib.GeneratorContextManager`` officially + private by renaming it to ``_GeneratorContextManager``. + +- Issue #10042: Fixed the total_ordering decorator to handle cross-type + comparisons that could lead to infinite recursion. + +- Issue #10686: the email package now :rfc:`2047`\ -encodes headers with + non-ASCII bytes (parsed by a BytesParser) when doing conversion to 7bit-clean + presentation, instead of replacing them with ?s. + +- email.header.Header was incorrectly encoding folding whitespace when + rfc2047-encoding header values with embedded newlines, leaving them without + folding whitespace. It now uses the continuation_ws, as it does for + continuation lines that it creates itself. + +- Issue #1777412, #10827: Changed the rules for 2-digit years. The + time.asctime(), time.ctime() and time.strftime() functions will now format + any year when ``time.accept2dyear`` is False and will accept years >= 1000 + otherwise. ``time.mktime`` and ``time.strftime`` now accept full range + supported by the OS. With Visual Studio or on Solaris, the year is limited to + the range [1; 9999]. Conversion of 2-digit years to 4-digit is deprecated. + +- Issue #7858: Raise an error properly when os.utime() fails under Windows + on an existing file. + +- Issue #3839: wsgiref should not override a Content-Length header set by + the application. Initial patch by Clovis Fabricio. + +- Issue #10492: bdb.Bdb.run() only traces the execution of the code, not the + compilation (if the input is a string). + +- Issue #7995: When calling accept() on a socket with a timeout, the returned + socket is now always blocking, regardless of the operating system. + +- Issue #10756: atexit normalizes the exception before displaying it. Patch by + Andreas Stührk. + +- Issue #10790: email.header.Header.append's charset logic now works correctly + for charsets whose output codec is different from its input codec. + +- Issue #10819: SocketIO.name property returns -1 when its closed, instead of + raising a ValueError, to fix repr(). + +- Issue #8650: zlib.compress() and zlib.decompress() raise an OverflowError if + the input buffer length doesn't fit into an unsigned int (length bigger than + 2^32-1 bytes). + +- Issue #6643: Reinitialize locks held within the threading module after fork to + avoid a potential rare deadlock or crash on some platforms. + +- Issue #10806, issue #9905: Fix subprocess pipes when some of the standard file + descriptors (0, 1, 2) are closed in the parent process. Initial patch by Ross + Lagerwall. + +- `unittest.TestCase` can be instantiated without a method name; for simpler + exploration from the interactive interpreter. + +- Issue #10798: Reject supporting concurrent.futures if the system has too + few POSIX semaphores. + +- Issue #10807: Remove base64, bz2, hex, quopri, rot13, uu and zlib codecs from + the codec aliases. They are still accessible via codecs.lookup(). + +- Issue #10801: In zipfile, support different encodings for the header and the + filenames. + +- Issue #6285: IDLE no longer crashes on missing help file; patch by Scott + David Daniels. + +- Fix collections.OrderedDict.setdefault() so that it works in subclasses that + define __missing__(). + +- Issue #10786: unittest.TextTestRunner default stream no longer bound at import + time. `sys.stderr` now looked up at instantiation time. Fix contributed by + Mark Roddy. + +- Issue #10753: Characters ';', '=' and ',' in the PATH_INFO environment variable + won't be quoted when the URI is constructed by the wsgiref.util's request_uri + method. According to RFC 3986, these characters can be a part of params in + PATH component of URI and need not be quoted. + +- Issue #10738: Fix webbrowser.Opera.raise_opts. + +- Issue #9824: SimpleCookie now encodes , and ; in values to cater to how + browsers actually parse cookies. + +- Issue #9333: os.symlink now available regardless of user privileges. The + function now raises OSError on Windows >=6.0 when the user is unable to create + symbolic links. XP and 2003 still raise NotImplementedError. + +- Issue #10783: struct.pack() no longer implicitly encodes unicode to UTF-8. + +- Issue #10730: Add SVG mime types to mimetypes module. + +- Issue #10768: Make the Tkinter ScrolledText widget work again. + +- Issue #10777: Fix "dictionary changed size during iteration" bug in + ElementTree register_namespace(). + +- Issue #10626: test_logging now preserves logger disabled states. + +- Issue #10774: test_logging now removes temp files created during tests. + +- Issue #5258/#10642: if site.py encounters a .pth file that generates an error, + it now prints the filename, line number, and traceback to stderr and skips + the rest of that individual file, instead of stopping processing entirely. + +- Issue #10763: subprocess.communicate() closes stdout and stderr if both are + pipes (bug specific to Windows). + +- Issue #1693546: fix email.message RFC 2231 parameter encoding to be in better + compliance (no "s around encoded values). + +- Improved the diff message in the unittest module's assertCountEqual(). + +- Issue #1155362: email.utils.parsedate_tz now handles a missing space before + the '-' of a timezone field as well as before a '+'. + +- Issue #4871: The zipfile module now gives a more useful error message if + an attempt is made to use a string to specify the archive password. + +- Issue #10750: The ``raw`` attribute of buffered IO objects is now read-only. + +- Deprecated assertDictContainsSubset() in the unittest module. + +C-API +----- + +- Issue #10913: Deprecate misleading functions PyEval_AcquireLock() and + PyEval_ReleaseLock(). The thread-state aware APIs should be used instead. + +- Issue #10333: Remove ancient GC API, which has been deprecated since Python + 2.2. + +Build +----- + +- Issue #10843: Update third-party library versions used in OS X 32-bit + installer builds: bzip2 1.0.6, readline 6.1.2, SQLite 3.7.4 (with FTS3/FTS4 + and RTREE enabled), and ncursesw 5.5 (wide-char support enabled). + +- Issue #10820: Fix OS X framework installs to support version-specific + scripts (#10679). + +- Issue #7716: Under Solaris, don't assume existence of /usr/xpg4/bin/grep in + the configure script but use $GREP instead. Patch by Fabian Groffen. + +- Issue #10475: Don't hardcode compilers for LDSHARED/LDCXXSHARED on NetBSD + and DragonFly BSD. Patch by Nicolas Joly. + +- Issue #10679: The "idle", "pydoc" and "2to3" scripts are now installed with + a version-specific suffix on "make altinstall". + +- Issue #10655: Fix the build on PowerPC on Linux with GCC when building with + timestamp profiling (--with-tsc): the preprocessor test for the PowerPC + support now looks for "__powerpc__" as well as "__ppc__": the latter seems to + only be present on OS X; the former is the correct one for Linux with GCC. + +- Issue #1099: Fix the build on MacOSX when building a framework with pydebug + using GCC 4.0. + +Tools/Demos +----------- + +- Issue #10843: Install the Tools directory on OS X in the applications Extras + (/Applications/Python 3.n/Extras/) where the Demo directory had previous been + installed. + +- Issue #7962: The Demo directory is gone. Most of the old and unmaintained + demos have been removed, others integrated in documentation or a new + Tools/demo subdirectory. + +- Issue #10502: Addition of the unittestgui tool. Originally by Steve Purcell. + Updated for test discovery by Mark Roddy and Python 3 compatibility by Brian + Curtin. + +Tests +----- + +- Fix test_startfile to wait for child process to terminate before finishing. + +- Issue #10822: Fix test_posix:test_getgroups failure under Solaris. Patch + by Ross Lagerwall. + +- Make the --coverage flag work for test.regrtest. + +- Issue #1677694: Refactor and improve test_timeout. Original patch by + Björn Lindqvist. + +- Issue #5485: Add tests for the UseForeignDTD method of expat parser objects. + Patch by Jean-Paul Calderone and Sandro Tosi. + +- Issue #6293: Have regrtest.py echo back sys.flags. This is done by default in + whole runs and enabled selectively using ``--header`` when running an explicit + list of tests. Original patch by Collin Winter. + + +What's New in Python 3.2 Beta 2? +================================ + +*Release date: 19-Dec-2010* + +Core and Builtins +----------------- + +- Issue #8844: Regular and recursive lock acquisitions can now be interrupted + by signals on platforms using pthreads. Patch by Reid Kleckner. + +- Issue #4236: PyModule_Create2 now checks the import machinery directly + rather than the Py_IsInitialized flag, avoiding a Fatal Python + error in certain circumstances when an import is done in __del__. + +- Issue #5587: add a repr to dict_proxy objects. Patch by David Stanek and + Daniel Urban. + +Library +------- + +- Issue #3243: Support iterable bodies in httplib. Patch Contributions by + Xuanji Li and Chris AtLee. + +- Issue #10611: SystemExit exception will no longer kill a unittest run. + +- Issue #9857: It is now possible to skip a test in a setUp, tearDown or clean + up function. + +- Issue #10573: use actual/expected consistently in unittest methods. + The order of the args of assertCountEqual is also changed. + +- Issue #9286: email.utils.parseaddr no longer concatenates blank-separated + words in the local part of email addresses, thereby preserving the input. + +- Issue #6791: Limit header line length (to 65535 bytes) in http.client + and http.server, to avoid denial of services from the other party. + +- Issue #10404: Use ctl-button-1 on OSX for the context menu in Idle. + +- Issue #9907: Fix tab handling on OSX when using editline by calling + rl_initialize first, then setting our custom defaults, then reading .editrc. + +- Issue #4188: Avoid creating dummy thread objects when logging operations + from the threading module (with the internal verbose flag activated). + +- Issue #10711: Remove HTTP 0.9 support from http.client. The ``strict`` + parameter to HTTPConnection and friends is deprecated. + +- Issue #9721: Fix the behavior of urljoin when the relative url starts with a + ';' character. Patch by Wes Chow. + +- Issue #10714: Limit length of incoming request in http.server to 65536 bytes + for security reasons. Initial patch by Ross Lagerwall. + +- Issue #9558: Fix distutils.command.build_ext with VS 8.0. + +- Issue #10667: Fast path for collections.Counter(). + +- Issue #10695: passing the port as a string value to telnetlib no longer + causes debug mode to fail. + +- Issue #1078919: add_header now automatically RFC2231 encodes parameters + that contain non-ascii values. + +- Issue #10188 (partial resolution): tempfile.TemporaryDirectory emits + a warning on sys.stderr rather than throwing a misleading exception + if cleanup fails due to nulling out of modules during shutdown. + Also avoids an AttributeError when mkdtemp call fails and issues + a ResourceWarning on implicit cleanup via __del__. + +- Issue #10107: Warn about unsaved files in IDLE on OSX. + +- Issue #7213: subprocess.Popen's default for close_fds has been changed. + It is now True in most cases other than on Windows when input, output or + error handles are provided. + +- Issue #6559: subprocess.Popen has a new pass_fds parameter (actually + added in 3.2beta1) to allow specifying a specific list of file descriptors + to keep open in the child process. + +- Issue #1731717: Fixed the problem where subprocess.wait() could cause an + OSError exception when The OS had been told to ignore SIGCLD in our process + or otherwise not wait for exiting child processes. + +Tests +----- + +- Issue #775964: test_grp now skips YP/NIS entries instead of failing when + encountering them. + +Tools/Demos +----------- + +- Issue #6075: IDLE on Mac OS X now works with both Carbon AquaTk and + Cocoa AquaTk. + +- Issue #10710: ``Misc/setuid-prog.c`` is removed from the source tree. + +- Issue #10706: Remove outdated script runtests.sh. Either ``make test`` + or ``python -m test`` should be used instead. + +Build +----- + +- The Windows build now uses Tcl/Tk 8.5.9 and sqlite3 3.7.4. + +- Issue #9234: argparse supports alias names for subparsers. + + +What's New in Python 3.2 Beta 1? +================================ + +*Release date: 05-Dec-2010* + +Core and Builtins +----------------- + +- Issue #10630: Return dict views from the dict proxy keys()/values()/items() + methods. + +- Issue #10596: Fix float.__mod__ to have the same behaviour as float.__divmod__ + with respect to signed zeros. -4.0 % 4.0 should be 0.0, not -0.0. + +- Issue #1772833: Add the -q command-line option to suppress copyright and + version output in interactive mode. + +- Provide an *optimize* parameter in the built-in compile() function. + +- Fixed several corner case issues on Windows in os.stat/os.lstat related to + reparse points. + +- PEP 384 (Defining a Stable ABI) is implemented. + +- Issue #2690: Range objects support negative indices and slicing. + +- Issue #9915: Speed up sorting with a key. + +- Issue #8685: Speed up set difference ``a - b`` when source set ``a`` is much + larger than operand ``b``. Patch by Andrew Bennetts. + +- Issue #10518: Bring back the callable() builtin. + +- Issue #7094: Added alternate formatting (specified by '#') to ``__format__`` + method of float, complex, and Decimal. This allows more precise control over + when decimal points are displayed. + +- Issue #10474: range.count() should return integers. + +- Issue #1574217: isinstance now catches only AttributeError, rather than + masking all errors. + +Library +------- + +- logging: added "handler of last resort". See http://bit.ly/last-resort-handler + +- test.support: Added TestHandler and Matcher classes for better support of + assertions about logging. + +- Issue #4391: Use proper plural forms in argparse. + +- Issue #10601: sys.displayhook uses 'backslashreplace' error handler on + UnicodeEncodeError. + +- Add the "display" and "undisplay" pdb commands. + +- Issue #7245: Add a SIGINT handler in pdb that allows to break a program again + after a "continue" command. + +- Add the "interact" pdb command. + +- Issue #7905: Actually respect the keyencoding parameter to shelve.Shelf. + +- Issue #1569291: Speed up array.repeat(). + +- Provide an interface to set the optimization level of compilation in + py_compile, compileall and zipfile.PyZipFile. + +- Issue #7904: Changes to urllib.parse.urlsplit to handle schemes as defined by + RFC3986. Anything before :// is considered a scheme and is followed by an + authority (or netloc) and by '/' led path, which is optional. + +- Issue #6045: dbm.gnu databases now support get() and setdefault() methods. + +- Issue #10620: `python -m unittest` can accept file paths instead of module + names for running specific tests. + +- Issue #9424: Deprecate the `unittest.TestCase` methods `assertEquals`, + `assertNotEquals`, `assertAlmostEquals`, `assertNotAlmostEquals` and `assert_` + and replace them with the correct methods in the Python test suite. + +- Issue #10272: The ssl module now raises socket.timeout instead of a generic + SSLError on socket timeouts. + +- Issue #10528: Allow translators to reorder placeholders in localizable + messages from argparse. + +- Issue #10497: Fix incorrect use of gettext in argparse. + +- Issue #10478: Reentrant calls inside buffered IO objects (for example by + way of a signal handler) now raise a RuntimeError instead of freezing the + current process. + +- logging: Added getLogRecordFactory/setLogRecordFactory with docs and tests. + +- Issue #10549: Fix pydoc traceback when text-documenting certain classes. + +- Issue #2001: New HTML server with enhanced Web page features. Patch by Ron + Adam. + +- Issue #10360: In WeakSet, do not raise TypeErrors when testing for membership + of non-weakrefable objects. + +- Issue #940286: pydoc.Helper.help() ignores input/output init parameters. + +- Issue #1745035: Add a command size and data size limit to smtpd.py, to prevent + DoS attacks. Patch by Savio Sena. + +- Issue #4925: Add filename to error message when executable can't be found in + subprocess. + +- Issue #10391: Don't dereference invalid memory in error messages in the ast + module. + +- Issue #10027: st_nlink was not being set on Windows calls to os.stat or + os.lstat. Patch by Hirokazu Yamamoto. + +- Issue #9333: Expose os.symlink only when the SeCreateSymbolicLinkPrivilege is + held by the user's account, i.e., when the function can actually be used. + +- Issue #8879: Add os.link support for Windows. + +- Issue #7911: ``unittest.TestCase.longMessage`` defaults to True for improved + failure messages by default. Patch by Mark Roddy. + +- Issue #1486713: HTMLParser now has an optional tolerant mode where it tries to + guess at the correct parsing of invalid html. + +- Issue #10554: Add context manager support to subprocess.Popen objects. + +- Issue #8989: email.utils.make_msgid now has a domain parameter that can + override the domain name used in the generated msgid. + +- Issue #9299: Add exist_ok parameter to os.makedirs to suppress the 'File + exists' exception when a target directory already exists with the specified + mode. Patch by Ray Allen. + +- Issue #9573: os.fork() now works correctly when triggered as a side effect of + a module import. + +- Issue #10464: netrc now correctly handles lines with embedded '#' characters. + +- Added itertools.accumulate(). + +- Issue #4113: Added custom ``__repr__`` method to ``functools.partial``. + Original patch by Daniel Urban. + +- Issue #10273: Rename `assertRegexpMatches` and `assertRaisesRegexp` to + `assertRegex` and `assertRaisesRegex`. + +- Issue #10535: Enable silenced warnings in unittest by default. + +- Issue #9873: The URL parsing functions in urllib.parse now accept ASCII byte + sequences as input in addition to character strings. + +- Issue #10586: The statistics API for the new functools.lru_cache has been + changed to a single cache_info() method returning a named tuple. + +- Issue #10323: itertools.islice() now consumes the minimum number of inputs + before stopping. Formerly, the final state of the underlying iterator was + undefined. + +- Issue #10565: The collections.Iterator ABC now checks for both __iter__ and + __next__. + +- Issue #10242: Fixed implementation of unittest.ItemsEqual and gave it a new + more informative name, unittest.CountEqual. + +- Issue #10561: In pdb, clear the breakpoints by the breakpoint number. + +- Issue #2986: difflib.SequenceMatcher gets a new parameter, autojunk, which can + be set to False to turn off the previously undocumented 'popularity' + heuristic. Patch by Terry Reedy and Eli Bendersky. + +- Issue #10534: in difflib, expose bjunk and bpopular sets; deprecate + undocumented and now redundant isbjunk and isbpopular methods. + +- Issue #9846: zipfile is now correctly closing underlying file objects. + +- Issue #10459: Update CJK character names to Unicode 6.0. + +- Issue #4493: urllib.request adds '/' in front of path components which does not + start with '/. Common behavior exhibited by browsers and other clients. + +- Issue #6378: idle.bat now runs with the appropriate Python version rather than + the system default. Patch by Sridhar Ratnakumar. + +- Issue #10470: 'python -m unittest' will now run test discovery by default, + when no extra arguments have been provided. + +- Issue #3709: BaseHTTPRequestHandler will buffer the headers and write to + output stream only when end_headers is invoked. This is a speedup and an + internal optimization. Patch by endian. + +- Issue #10220: Added inspect.getgeneratorstate. Initial patch by Rodolpho + Eckhardt. + +- Issue #10453: compileall now uses argparse instead of getopt, and thus + provides clean output when called with '-h'. + +- Issue #8078: Add constants for higher baud rates in the termios module. Patch + by Rodolpho Eckhardt. + +- Issue #10407: Fix two NameErrors in distutils. + +- Issue #10371: Deprecated undocumented functions in the trace module. + +- Issue #10467: Fix BytesIO.readinto() after seeking into a position after the + end of the file. + +- configparser: 100% test coverage. + +- Issue #10499: configparser supports pluggable interpolation handlers. The + default classic interpolation handler is called BasicInterpolation. Another + interpolation handler added (ExtendedInterpolation) which supports the syntax + used by zc.buildout (e.g. interpolation between sections). + +- configparser: the SafeConfigParser class has been renamed to ConfigParser. + The legacy ConfigParser class has been removed but its interpolation mechanism + is still available as LegacyInterpolation. + +- configparser: Usage of RawConfigParser is now discouraged for new projects + in favor of ConfigParser(interpolation=None). + +- Issue #1682942: configparser supports alternative option/value delimiters. + +- Issue #5412: configparser supports mapping protocol access. + +- Issue #9411: configparser supports specifying encoding for read operations. + +- Issue #9421: configparser's getint(), getfloat() and getboolean() methods + accept vars and default arguments just like get() does. + +- Issue #9452: configparser supports reading from strings and dictionaries + (thanks to the mapping protocol API, the latter can be used to copy data + between parsers). + +- configparser: accepted INI file structure is now customizable, including + comment prefixes, name of the DEFAULT section, empty lines in multiline + values, and indentation. + +- Issue #10326: unittest.TestCase instances can be pickled. + +- Issue #9926: Wrapped TestSuite subclass does not get __call__ executed. + +- Issue #9920: Skip tests for cmath.atan and cmath.atanh applied to complex + zeros on systems where the log1p function fails to respect the sign of zero. + This fixes a test failure on AIX. + +- Issue #9732: Addition of getattr_static to the inspect module. + +- Issue #10446: Module documentation generated by pydoc now links to a + version-specific online reference manual. + +- Make the 'No module named' exception message from importlib consistent. + +- Issue #10443: Add the SSLContext.set_default_verify_paths() method. + +- Issue #10440: Support RUSAGE_THREAD as a constant in the resource module. + Patch by Robert Collins. + +- Issue #10429: IMAP.starttls() stored the capabilities as bytes objects, rather + than strings. + +C-API +----- + +- Issue #10557: Added a new API function, PyUnicode_TransformDecimalToASCII(), + which transforms non-ASCII decimal digits in a Unicode string to their ASCII + equivalents. + +- Issue #9518: Extend the PyModuleDef_HEAD_INIT macro to explicitly + zero-initialize all fields, fixing compiler warnings seen when building + extension modules with gcc with "-Wmissing-field-initializers" (implied by + "-W"). + +- Issue #10255: Fix reference leak in Py_InitializeEx(). Patch by Neil + Schemenauer. + +- structseq.h is now included in Python.h. + +- Loosen PyArg_ValidateKeywordArguments to allow dict subclasses. + +Tests +----- + +- regrtest.py once again ensures the test directory is removed from sys.path + when it is invoked directly as the __main__ module. + +- `python -m test` can be used to run the test suite as well as `python -m + test.regrtest`. + +- Do not fail test_socket when the IP address of the local hostname cannot be + looked up. + +- Issue #8886: Use context managers throughout test_zipfile. Patch by Eric + Carstensen. + +Build +----- + +- Issue #10325: Fix two issues in the fallback definitions for PY_ULLONG_MAX and + PY_LLONG_MAX that made them unsuitable for use in preprocessor conditionals. + +Documentation +------------- + +- Issue #10299: List the built-in functions in a table in functions.rst. + + +What's New in Python 3.2 Alpha 4? +================================= + +*Release date: 13-Nov-2010* + +Core and Builtins +----------------- + +- Issue #10372: Import the warnings module only after the IO library is + initialized, so as to avoid bootstrap issues with the '-W' option. + +- Issue #10293: Remove obsolete field in the PyMemoryView structure, unused + undocumented value PyBUF_SHADOW, and strangely-looking code in + PyMemoryView_GetContiguous. + +- Issue #6081: Add str.format_map(), similar to ``str.format(**mapping)``. + +- If FileIO.__init__ fails, close the file descriptor. + +- Issue #10221: dict.pop(k) now has a key error message that includes the + missing key (same message d[k] returns for missing keys). + +- Issue #5437: A preallocated MemoryError instance should not keep traceback + data (including local variables caught in the stack trace) alive infinitely. + +- Issue #10186: Fix the SyntaxError caret when the offset is equal to the length + of the offending line. + +- Issue #10089: Add support for arbitrary -X options on the command line. They + can be retrieved through a new attribute ``sys._xoptions``. + +- Issue #4388: On Mac OS X, decode command line arguments from UTF-8, instead of + the locale encoding. If the LANG (and LC_ALL and LC_CTYPE) environment + variable is not set, the locale encoding is ISO-8859-1, whereas most programs + (including Python) expect UTF-8. Python already uses UTF-8 for the filesystem + encoding and to encode command line arguments on this OS. + +- Issue #9713, #10114: Parser functions (e.g. PyParser_ASTFromFile) expect + filenames encoded to the filesystem encoding with the surrogateescape error + handler (to support undecodable bytes), instead of UTF-8 in strict mode. + +- Issue #9997: Don't let the name "top" have special significance in scope + resolution. + +- Issue #9862: Compensate for broken PIPE_BUF in AIX by hard coding its value as + the default 512 when compiling on AIX. + +- Use locale encoding instead of UTF-8 to encode and decode filenames if + Py_FileSystemDefaultEncoding is not set. + +- Issue #10095: fp_setreadl() doesn't reopen the file, instead reuse the file + descriptor. + +- Issue #9418: Moved private string methods ``_formatter_parser`` and + ``_formatter_field_name_split`` into a new ``_string`` module. + +- Issue #9992: Remove PYTHONFSENCODING environment variable. + +Library +------- + +- Issue #10465: fix broken delegating of attributes by gzip._PaddedFile. + +- Issue #10356: Decimal.__hash__(-1) should return -2. + +- Issue #1553375: logging: Added stack_info kwarg to display stack information. + +- Issue #5111: IPv6 Host in the Header is wrapped inside [ ]. Patch by Chandru. + +- Fix Fraction.__hash__ so that Fraction.__hash__(-1) is -2. (See also issue + #10356.) + +- Issue #4471: Add the IMAP.starttls() method to enable encryption on standard + IMAP4 connections. Original patch by Lorenzo M. Catucci. + +- Issue #1466065: Add 'validate' option to base64.b64decode to raise an error if + there are non-base64 alphabet characters in the input. + +- Issue #10386: Add __all__ to token module; this simplifies importing in + tokenize module and prevents leaking of private names through ``import *``. + +- Issue #4471: Properly shutdown socket in IMAP.shutdown(). Patch by Lorenzo + M. Catucci. + +- Fix IMAP.login() to work properly. + +- Issue #9244: multiprocessing pool worker processes could terminate + unexpectedly if the return value of a task could not be pickled. Only the + ``repr`` of such errors are now sent back, wrapped in an + ``MaybeEncodingError`` exception. + +- Issue #9244: The ``apply_async()`` and ``map_async()`` methods of + ``multiprocessing.Pool`` now accepts a ``error_callback`` argument. This can + be a callback with the signature ``callback(exc)``, which will be called if + the target raises an exception. + +- Issue #10022: The dictionary returned by the ``getpeercert()`` method of SSL + sockets now has additional items such as ``issuer`` and ``notBefore``. + +- ``usenetrc`` is now false by default for NNTP objects. + +- Issue #1926: Add support for NNTP over SSL on port 563, as well as STARTTLS. + Patch by Andrew Vant. + +- Issue #10335: Add tokenize.open(), detect the file encoding using + tokenize.detect_encoding() and open it in read only mode. + +- Issue #10321: Add support for binary data to smtplib.SMTP.sendmail, and a new + method send_message to send an email.message.Message object. + +- Issue #6011: sysconfig and distutils.sysconfig use the surrogateescape error + handler to parse the Makefile file. Avoid a UnicodeDecodeError if the source + code directory name contains a non-ASCII character and the locale encoding is + ASCII. + +- Issue #10329: The trace module writes reports using the input Python script + encoding, instead of the locale encoding. Patch written by Alexander + Belopolsky. + +- Issue #10126: Fix distutils' test_build when Python was built with + --enable-shared. + +- Issue #9281: Prevent race condition with mkdir in distutils. Patch by + Arfrever. + +- Issue #10229: Fix caching error in gettext. + +- Issue #10252: Close file objects in a timely manner in distutils code and + tests. Patch by Brian Brazil, completed by Éric Araujo. + +- Issue #10180: Pickling file objects is now explicitly forbidden, since + unpickling them produced nonsensical results. + +- Issue #10311: The signal module now restores errno before returning from its + low-level signal handler. Patch by Hallvard B Furuseth. + +- Issue #10282: Add a ``nntp_implementation`` attribute to NNTP objects. + +- Issue #10283: Add a ``group_pattern`` argument to NNTP.list(). + +- Issue #10155: Add IISCGIHandler to wsgiref.handlers to support IIS CGI + environment better, and to correct unicode environment values for WSGI 1.0.1. + +- Issue #10281: nntplib now returns None for absent fields in the OVER/XOVER + response, instead of raising an exception. + +- wsgiref now implements and validates PEP 3333, rather than an experimental + extension of PEP 333. (Note: earlier versions of Python 3.x may have + incorrectly validated some non-compliant applications as WSGI compliant; if + your app validates with Python <3.2b1+, but not on this version, it is likely + the case that your app was not compliant.) + +- Issue #10280: NNTP.nntp_version should reflect the highest version advertised + by the server. + +- Issue #10184: Touch directories only once when extracting a tarfile. + +- Issue #10199: New package, ``turtledemo`` now contains selected demo scripts + that were formerly found under Demo/turtle. + +- Issue #10265: Close file objects explicitly in sunau. Patch by Brian Brazil. + +- Issue #10266: uu.decode didn't close in_file explicitly when it was given as a + filename. Patch by Brian Brazil. + +- Issue #10110: Queue objects didn't recognize full queues when the maxsize + parameter had been reduced. + +- Issue #10160: Speed up operator.attrgetter. Patch by Christos Georgiou. + +- logging: Added style option to basicConfig() to allow %, {} or $-formatting. + +- Issue #5729: json.dumps() now supports using a string such as '\t' for + pretty-printing multilevel objects. + +- Issue #10253: FileIO leaks a file descriptor when trying to open a file for + append that isn't seekable. Patch by Brian Brazil. + +- Support context manager protocol for file-like objects returned by mailbox + ``get_file()`` methods. + +- Issue #10246: uu.encode didn't close file objects explicitly when filenames + were given to it. Patch by Brian Brazil. + +- Issue #10198: fix duplicate header written to wave files when writeframes() is + called without data. + +- Close file objects in modulefinder in a timely manner. + +- Close a io.TextIOWrapper object in email.parser in a timely manner. + +- Close a file object in distutils.sysconfig in a timely manner. + +- Close a file object in pkgutil in a timely manner. + +- Issue #10233: Close file objects in a timely manner in the tarfile module and + its test suite. + +- Issue #10093: ResourceWarnings are now issued when files and sockets are + deallocated without explicit closing. These warnings are silenced by default, + except in pydebug mode. + +- tarfile.py: Add support for all missing variants of the GNU sparse extensions + and create files with holes when extracting sparse members. + +- Issue #10218: Return timeout status from ``Condition.wait`` in threading. + +- Issue #7351: Add ``zipfile.BadZipFile`` spelling of the exception name and + deprecate the old name ``zipfile.BadZipfile``. + +- Issue #5027: The standard ``xml`` namespace is now understood by + xml.sax.saxutils.XMLGenerator as being bound to + http://www.w3.org/XML/1998/namespace. Patch by Troy J. Farrell. + +- Issue #5975: Add csv.unix_dialect class. + +- Issue #7761: telnetlib.interact failures on Windows fixed. + +- logging: Added style option to Formatter to allow %, {} or $-formatting. + +- Issue #5178: Added tempfile.TemporaryDirectory class that can be used as a + context manager. + +- Issue #1349106: Generator (and BytesGenerator) flatten method and Header + encode method now support a 'linesep' argument. + +- Issue #5639: Add a *server_hostname* argument to ``SSLContext.wrap_socket`` in + order to support the TLS SNI extension. ``HTTPSConnection`` and ``urlopen()`` + also use this argument, so that HTTPS virtual hosts are now supported. + +- Issue #10166: Avoid recursion in pstats Stats.add() for many stats items. + +- Issue #10163: Skip unreadable registry keys during mimetypes initialization. + +- logging: Made StreamHandler terminator configurable. + +- logging: Allowed filters to be just callables. + +- logging: Added tests for _logRecordClass changes. + +- Issue #10092: Properly reset locale in calendar.Locale*Calendar classes. + +- logging: Added _logRecordClass, getLogRecordClass, setLogRecordClass to + increase flexibility of LogRecord creation. + +- Issue #5117: Case normalization was needed on ntpath.relpath(). Also fixed + root directory issue on posixpath.relpath(). (Ported working fixes from + ntpath.) + +- Issue #1343: xml.sax.saxutils.XMLGenerator now has an option + short_empty_elements to direct it to use self-closing tags when appropriate. + +- Issue #9807 (part 1): Expose the ABI flags in sys.abiflags. Add --abiflags + switch to python-config for command line access. + +- Issue #6098: Don't claim DOM level 3 conformance in minidom. + +- Issue #5762: Fix AttributeError raised by ``xml.dom.minidom`` when an empty + XML namespace attribute is encountered. + +- Issue #2830: Add the ``html.escape()`` function, which quotes all problematic + characters by default. Deprecate ``cgi.escape()``. + +- Issue #9409: Fix the regex to match all kind of filenames, for interactive + debugging in doctests. + +- Issue #9183: ``datetime.timezone(datetime.timedelta(0))`` will now return the + same instance as ``datetime.timezone.utc``. + +- Issue #7523: Add SOCK_CLOEXEC and SOCK_NONBLOCK to the socket module, where + supported by the system. Patch by Nikita Vetoshkin. + +- Issue #10063: file:// scheme will stop accessing remote hosts via ftp + protocol. file:// urls had fallback to access remote hosts via ftp. This was + not correct, change is made to raise a URLError when a remote host is tried to + access via file:// scheme. + +- Issue #1710703: Write structures for an empty ZIP archive when a ZipFile is + created in modes 'a' or 'w' and then closed without adding any files. Raise + BadZipfile (rather than IOError) when opening small non-ZIP files. + +- Issue #10041: The signature of optional arguments in socket.makefile() didn't + match that of io.open(), and they also didn't get forwarded properly to + TextIOWrapper in text mode. Patch by Kai Zhu. + +- Issue #9003: http.client.HTTPSConnection, urllib.request.HTTPSHandler and + urllib.request.urlopen now take optional arguments to allow for server + certificate checking, as recommended in public uses of HTTPS. + +- Issue #6612: Fix site and sysconfig to catch os.getcwd() error, eg. if the + current directory was deleted. Patch written by W. Trevor King. + +- Issue #3873: Speed up unpickling from file objects that have a peek() method. + +- Issue #10075: Add a session_stats() method to SSLContext objects. + +- Issue #9948: Fixed problem of losing filename case information. + +Extension Modules +----------------- + +- Issue #5109: array.array constructor will now use fast code when + initial data is provided in an array object with correct type. + +- Issue #6317: Now winsound.PlaySound only accepts unicode. + +- Issue #6317: Now winsound.PlaySound can accept non ascii filename. + +- Issue #9377: Use Unicode API for gethostname on Windows. + +- Issue #10143: Update "os.pathconf" values. + +- Issue #6518: Support context manager protcol for ossaudiodev types. + +- Issue #678250: Make mmap flush a noop on ACCESS_READ and ACCESS_COPY. + +- Issue #9054: Fix a crash occurring when using the pyexpat module with expat + version 2.0.1. + +- Issue #5355: Provide mappings from Expat error numbers to string descriptions + and backwards, in order to actually make it possible to analyze error codes + provided by ExpatError. + +- The Unicode database was updated to 6.0.0. + +C-API +----- + +- Issue #10288: The deprecated family of "char"-handling macros + (ISLOWER()/ISUPPER()/etc) have now been removed: use Py_ISLOWER() etc instead. + +- Issue #9778: Hash values are now always the size of pointers. A new Py_hash_t + type has been introduced. + +Tools/Demos +----------- + +- Issue #10117: Tools/scripts/reindent.py now accepts source files that use + encoding other than ASCII or UTF-8. Source encoding is preserved when + reindented code is written to a file. + +- Issue #7287: Demo/imputil/knee.py was removed. + +Tests +----- + +- Issue #3699: Fix test_bigaddrspace and extend it to test bytestrings as well + as unicode strings. Initial patch by Sandro Tosi. + +- Issue #10294: Remove dead code form test_unicode_file. + +- Issue #10123: Don't use non-ascii filenames in test_doctest tests. Add a new + test specific to unicode (non-ascii name and filename). + +Build +----- + +- Issue #10268: Add a --enable-loadable-sqlite-extensions option to configure. + +- Issue #8852: Allow the socket module to build on OpenSolaris. + +- Drop -OPT:Olimit compiler option. + +- Issue #10094: Use versioned .so files on GNU/kfreeBSD and the GNU Hurd. + +- Accept Oracle Berkeley DB 5.0 and 5.1 as backend for the dbm extension. + +- Issue #7473: avoid link errors when building a framework with a different set + of architectures than the one that is currently installed. + + +What's New in Python 3.2 Alpha 3? +================================= + +*Release date: 09-Oct-2010* + +Core and Builtins +----------------- + +- Issue #10068: Global objects which have reference cycles with their module's + dict are now cleared again. This causes issue #7140 to appear again. + +- Issue #9738: Document PyErr_SetString() and PyErr_SetFromErrnoWithFilename() + encodings. + +- ast.literal_eval() can now handle negative numbers. It is also a little more + liberal in what it accepts without compromising the safety of the evaluation. + For example, 3j+4 and 3+4+5 are both accepted. + +- Issue #10006: type.__abstractmethods__ now raises an AttributeError. As a + result metaclasses can now be ABCs (see #9533). + +- Issue #8670: ctypes.c_wchar supports non-BMP characters with 32 bits wchar_t. + +- Issue #8670: PyUnicode_AsWideChar() and PyUnicode_AsWideCharString() replace + UTF-16 surrogate pairs by single non-BMP characters for 16 bits Py_UNICODE and + 32 bits wchar_t (eg. Linux in narrow build). + +- Issue #10003: Allow handling of SIGBREAK on Windows. Fixes a regression + introduced by issue #9324. + +- Issue #9979: Create function PyUnicode_AsWideCharString(). + +- Issue #7397: Mention that importlib.import_module() is probably what someone + really wants to be using in __import__'s docstring. + +- Issue #8521: Allow CreateKeyEx, OpenKeyEx, and DeleteKeyEx functions of winreg + to use named arguments. + +- Issue #9930: Remove bogus subtype check that was causing (e.g.) + float.__rdiv__(2.0, 3) to return NotImplemented instead of the expected 1.5. + +- Issue #9808: Implement os.getlogin for Windows. Patch by Jon Anglin. + +- Issue #9901: Destroying the GIL in Py_Finalize() can fail if some other + threads are still running. Instead, reinitialize the GIL on a second call to + Py_Initialize(). + +- All SyntaxErrors now have a column offset and therefore a caret when the error + is printed. + +- Issue #9252: PyImport_Import no longer uses a fromlist hack to return the + module that was imported, but instead gets the module from sys.modules. + +- Issue #9213: The range type_items now provides index() and count() methods, to + conform to the Sequence ABC. Patch by Daniel Urban and Daniel Stutzbach. + +- Issue #7994: Issue a PendingDeprecationWarning if object.__format__ is called + with a non-empty format string. This is an effort to future-proof user + code. If a derived class does not currently implement __format__ but later + adds its own __format__, it would most likely break user code that had + supplied a format string. This will be changed to a DeprecationWaring in + Python 3.3 and it will be an error in Python 3.4. + +- Issue #9828: Destroy the GIL in Py_Finalize(), so that it gets properly + re-created on a subsequent call to Py_Initialize(). The problem (a crash) + wouldn't appear in 3.1 or 2.7 where the GIL's structure is more trivial. + +- Issue #9210: Configure option --with-wctype-functions was removed. Using the + functions from the libc caused the methods .upper() and lower() to become + locale aware and created subtly wrong results. + +- Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on a + non-ASCII byte in the format string. + +- Issue #4617: Previously it was illegal to delete a name from the local + namespace if it occurs as a free variable in a nested block. This limitation + of the compiler has been lifted, and a new opcode introduced (DELETE_DEREF). + +- Issue #9804: ascii() now always represents unicode surrogate pairs as a single + ``\UXXXXXXXX``, regardless of whether the character is printable or not. + Also, the "backslashreplace" error handler now joins surrogate pairs into a + single character on UCS-2 builds. + +- Issue #9757: memoryview objects get a release() method to release the + underlying buffer (previously this was only done when deallocating the + memoryview), and gain support for the context management protocol. + +- Issue #9797: pystate.c wrongly assumed that zero couldn't be a valid + thread-local storage key. + +Library +------- + +- Issue #2236: distutils' mkpath ignored the mode parameter. + +- Fix typo in one sdist option (medata-check). + +- Issue #9199: Fix incorrect use of distutils.cmd.Command.announce. + +- Issue #1718574: Fix options that were supposed to accept arguments but did + not in build_clib. + +- Issue #9437: Fix building C extensions with non-default LDFLAGS. + +- Issue #4661: email can now parse bytes input and generate either converted + 7bit output or bytes output. Email version bumped to 5.1.0. + +- Issue #1589: Add ssl.match_hostname(), to help implement server identity + verification for higher-level protocols. + +- Issue #9759: GzipFile now raises ValueError when an operation is attempted + after the file is closed. Patch by Jeffrey Finkelstein. + +- Issue #9042: Fix interaction of custom translation classes and caching in + gettext. + +- Issue #6706: asyncore.dispatcher now provides a handle_accepted() method + returning a (sock, addr) pair which is called when a connection has been + established with a new remote endpoint. This is supposed to be used as a + replacement for old handle_accept() and avoids the user to call accept() + directly. + +- Issue #9065: tarfile no longer uses "root" as the default for the uname and + gname field. + +- Issue #8980: Fixed a failure in distutils.command check that was shadowed by + an environment that does not have docutils. Patch by Arfrever. + +- Issue #1050268: parseaddr now correctly quotes double quote and backslash + characters that appear inside quoted strings in email addresses. + +- Issue #10004: quoprimime no longer generates a traceback when confronted with + invalid characters after '=' in a Q-encoded word. + +- Issue #1491: BaseHTTPServer nows send a ``100 Continue`` response before + sending a 200 OK for the Expect: 100-continue request header. + +- Issue #9360: Cleanup and improvements to the nntplib module. The API now + conforms to the philosophy of bytes and unicode separation in Python 3. A + test suite has also been added. + +- Issue #9962: GzipFile now has the peek() method. + +- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or EAGAIN, + retry the select() loop instead of bailing out. This is because select() can + incorrectly report a socket as ready for reading (for example, if it received + some data with an invalid checksum). + +- Issue #3612: Added new types to ctypes.wintypes. (CHAR and pointers) + +- Issue #9950: Fix socket.sendall() crash or misbehaviour when a signal is + received. Now sendall() properly calls signal handlers if necessary, and + retries sending if these returned successfully, including on sockets with a + timeout. + +- Issue #9947: logging: Fixed locking bug in stopListening. + +- Issue #9945: logging: Fixed locking bugs in addHandler/removeHandler. + +- Issue #9936: Fixed executable lines' search in the trace module. + +- Issue #9790: Rework imports necessary for samefile and sameopenfile + in ntpath. + +- Issue #9928: Properly initialize the types exported by the bz2 module. + +- Issue #1675951: Allow GzipFile to work with unseekable file objects. Patch by + Florian Festi. + +- Logging: Added QueueListener class to facilitate logging usage for + performance-critical threads. + +- Issue #9916: Add some missing errno symbols. + +- Issue #9877: Expose sysconfig.get_makefile_filename() + +- logging: Added hasHandlers() method to Logger and LoggerAdapter. + +- Issue #9908: Fix os.stat() on bytes paths under Windows 7. + +- Issue #2643: msync() is not called anymore when deallocating an open mmap + object, only munmap(). + +- logging: Changed LoggerAdapter implementation internally, to make it easier to + subclass in a useful way. + +- logging: hasHandlers method was added to Logger, and isEnabledFor, + getEffectiveLevel, hasHandlers and setLevel were added to LoggerAdapter. + LoggerAdapter was introduced into the unit tests for logging. + +- Issue #1686: Fix string.Template when overriding the pattern attribute. + +- Issue #9854: SocketIO objects now observe the RawIOBase interface in + non-blocking mode: they return None when an operation would block (instead of + raising an exception). + +- Issue #1730136: Fix the comparison between a tk.font.Font and an object of + another kind. + +- Issue #9441: logging has better coverage for rotating file handlers. + +- Issue #9865: collections.OrderedDict now has a __sizeof__ method. + +- Issue #9854: The default read() implementation in io.RawIOBase now handles + non-blocking readinto() returning None correctly. + +- Issue #1552: socket.socketpair() now returns regular socket.socket objects + supporting the whole socket API (rather than the "raw" _socket.socket + objects). + +- Issue #9853: Fix the signature of SSLSocket.recvfrom() and SSLSocket.sendto() + to match the corresponding socket methods. + +- Issue #9840: Added a decorator to reprlib for wrapping __repr__ methods to make + them handle recursive calls within the same thread. + +- logging: Enhanced HTTPHandler with secure and credentials initializers. + +- Issue #767645: Set os.path.supports_unicode_filenames to True on Mac OS X. + +- Issue #9837: The read() method of ZipExtFile objects (as returned by + ZipFile.open()) could return more bytes than requested. + +- Issue #9826: OrderedDict.__repr__ can now handle self-referential values: + d['x'] = d. + +- Issue #9825: Using __del__ in the definition of collections.OrderedDict made + it possible for the user to create self-referencing ordered dictionaries which + become permanently uncollectable GC garbage. Reinstated the Python 3.1 + approach of using weakref proxies so that reference cycles never get created + in the first place. + +- Issue #9579, #9580: Fix os.confstr() for value longer than 255 bytes and + encode the value with filesystem encoding and surrogateescape (instead of + utf-8 in strict mode) . Patch written by David Watson. + +- Issue #9632: Remove sys.setfilesystemencoding() function: use PYTHONFSENCODING + environment variable to set the filesystem encoding at Python startup. + sys.setfilesystemencoding() creates inconsistencies because it is unable to + reencode all filenames in all objects. + +- Issue #9410: Various optimizations to the pickle module, leading to speedups + up to 4x (depending on the benchmark). Mostly ported from Unladen Swallow; + initial patch by Alexandre Vassalotti. + +- The pprint module now supports printing OrderedDicts in their given order + (formerly, it would sort the keys). + +- Logging: Added QueueHandler class to facilitate logging usage with + multiprocessing. + +- Issue #9707: Rewritten reference implementation of threading.local which is + friendlier towards reference cycles. This change is not normally visible + since an optimized C implementation (_thread._local) is used instead. + +- Issue #6394: os.getppid() is now supported on Windows. Note that it will + still return the id of the parent process after it has exited. This process + id may even have been reused by another unrelated process. + +- Issue #9792: In case of connection failure, socket.create_connection() would + swallow the exception and raise a new one, making it impossible to fetch the + original errno, or to filter timeout errors. Now the original error is + re-raised. + +- Issue #9758: When fcntl.ioctl() was called with mutable_flag set to True, and + the passed buffer was exactly 1024 bytes long, the buffer wouldn't be updated + back after the system call. Original patch by Brian Brazil. + +- Updates to the random module: + + * Document which parts of the module are guaranteed to stay the same across + versions and which parts are subject to change. + + * Update the seed() method to use all of the bits in a string instead of just + the hash value. This makes better use of the seed value and assures the + seeding is platform independent. Issue #7889. + + * Improved the random()-->integer algorithm used in choice(), shuffle(), + sample(), randrange(), and randint(). Formerly, it used int(n*random()) + which has a slight bias whenever n is not a power of two. Issue #9025. + + * Improved documentation of arguments to randrange(). Issue #9379. + +- collections.OrderedDict now supports a new method for repositioning keys to + either end. + +- Issue #9754: Similarly to assertRaises and assertRaisesRegexp, unittest test + cases now also have assertWarns and assertWarnsRegexp methods to check that a + given warning type was triggered by the code under test. + +- Issue #5506: BytesIO objects now have a getbuffer() method exporting a view of + their contents without duplicating them. The view is both readable and + writable. + +- Issue #7566: Implement os.path.sameopenfile for Windows. + +- Issue #9293: I/O streams now raise ``io.UnsupportedOperation`` when an + unsupported operation is attempted (for example, writing to a file open only + for reading). + +- hashlib has two new constant attributes: algorithms_guaranteed and + algorithms_avaiable that respectively list the names of hash algorithms + guaranteed to exist in all Python implementations and the names of hash + algorithms available in the current process. + +- A new package ``concurrent.futures`` as defined by PEP 3148. + +C-API +----- + +- Add PyErr_SyntaxLocationEx, which supports passing a column offset. + +- Issue #9834: Don't segfault in PySequence_GetSlice, PySequence_SetSlice, or + PySequence_DelSlice when the object doesn't have any mapping operations + defined. + +Tools/Demos +----------- + +- Issue #9188: The gdb extension now handles correctly narrow (UCS2) as well as + wide (UCS4) unicode builds for both the host interpreter (embedded inside gdb) + and the interpreter under test. + +Tests +----- + +- Issue #9308: Added tests for importing encoded modules that do not + depend on specific stdlib modules being encoded in a certain way. + +- Issue #1051: Add a script (Lib/test/make_ssl_certs.py) to generate the custom + certificate and private key files used by SSL-related certs. + +- Issue #9978: Wait until subprocess completes initialization. (Win32KillTests + in test_os) + +- Issue #7110: regrtest now sends test failure reports and single-failure + tracebacks to stderr rather than stdout. + +- Issue #9628: fix runtests.sh -x option so more than one test can be excluded. + +- Issue #9899: Fix test_tkinter.test_font on various platforms. Patch by Ned + Deily. + +- Issue #9894: Do not hardcode ENOENT in test_subprocess. + +- Issue #9315: Added tests for the trace module. Patch by Eli Bendersky. + +- Issue #9323: Make test.regrtest.__file__ absolute, this was not always the + case when running profile or trace, for example. + +- Issue #9568: Fix test_urllib2_localnet on OS X 10.3. + +Build +----- + +- Issue #10062: Allow building on platforms which do not have sem_timedwait. + +- Issue #10054: Some platforms provide uintptr_t in inttypes.h. Patch by Akira + Kitada. + +- Issue #10055: Make json C89-compliant in UCS4 mode. + +- Issue #9552: Avoid unnecessary rebuild of OpenSSL. (Windows) + +- Issue #1633863: Don't ignore $CC under AIX. + +- Issue #9810: Compile bzip2 source files in Python's project file directly. It + used to be built with bzip2's makefile. + +- Issue #9848: Stopping trying to build _weakref in setup.py as it is a built-in + module. + +- Issue #9806: python-config now has an ``--extension-suffix`` option that + outputs the suffix for dynamic libraries including the ABI version name + defined by PEP 3149. + +- Issue #941346: Improve the build process under AIX and allow Python to be + built as a shared library. Patch by Sébastien Sablé. + +- Issue #4026: Make the fcntl extension build under AIX. Patch by Sébastien + Sablé. + +- Issue #9701: The MacOSX installer can patch the shell profile to ensure that + the "bin" directory inside the framework is on the shell's search path. This + feature now also supports the ZSH shell. + + +What's New in Python 3.2 Alpha 2? +================================= + +*Release date: 05-Sep-2010* + +Core and Builtins +----------------- + +- Issue #9225: Remove the ROT_FOUR and DUP_TOPX opcode, the latter replaced by + the new (and simpler) DUP_TOP_TWO. Performance isn't changed, but our + bytecode is a bit simplified. Patch by Demur Rumed. + +- Issue #9766: Rename poorly named variables exposed by _warnings to prevent + confusion with the proper variables names from 'warnings' itself. + +- Issue #9212: dict_keys and dict_items now provide the isdisjoint() method, to + conform to the Set ABC. Patch by Daniel Urban. + +- Issue #9737: Fix a crash when trying to delete a slice or an item from a + memoryview object. + +- Issue #9549: sys.setdefaultencoding() and PyUnicode_SetDefaultEncoding() are + now removed, since their effect was inexistent in 3.x (the default encoding is + hardcoded to utf-8 and cannot be changed). + +- Issue #7415: PyUnicode_FromEncodedObject() now uses the new buffer API + properly. Patch by Stefan Behnel. + +- Issue #5553: The Py_LOCAL_INLINE macro now results in inlining on most + platforms. Previously, it inlined only when using Microsoft Visual C. + +- Issue #9712: Fix tokenize on identifiers that start with non-ascii names. + +- Issue #9688: __basicsize__ and __itemsize__ must be accessed as Py_ssize_t. + +- Issue #9684: Added a definition for SIZEOF_WCHAR_T to PC/pyconfig.h, to match + the pyconfig.h generated by configure on other systems. + +- Issue #9666: Only catch AttributeError in hasattr(). All other exceptions that + occur during attribute lookup are now propagated to the caller. + +- Issue #8622: Add PYTHONFSENCODING environment variable to override the + filesystem encoding. + +- Issue #5127: The C functions that access the Unicode Database now accept and + return characters from the full Unicode range, even on narrow unicode builds + (Py_UNICODE_TOLOWER, Py_UNICODE_ISDECIMAL, and others). A visible difference + in Python is that unicodedata.numeric() now returns the correct value for + large code points, and repr() may consider more characters as printable. + +- Issue #9425: Create PyModule_GetFilenameObject() function to get the filename + as a unicode object, instead of a byte string. Function needed to support + unencodable filenames. Deprecate PyModule_GetFilename() in favor on the new + function. + +- Issue #8063: Call _PyGILState_Init() earlier in Py_InitializeEx(). + +- Issue #9612: The set object is now 64-bit clean under Windows. + +- Issue #8202: sys.argv[0] is now set to '-m' instead of '-c' when searching for + the module file to be executed with the -m command line option. + +- Issue #9599: Create PySys_FormatStdout() and PySys_FormatStderr() functions to + write a message formatted by PyUnicode_FromFormatV() to sys.stdout and + sys.stderr. + +- Issue #9542: Create PyUnicode_FSDecoder() function, a ParseTuple converter: + decode bytes objects to unicode using PyUnicode_DecodeFSDefaultAndSize(); str + objects are output as-is. + +- Issue #9203: Computed gotos are now enabled by default on supported compilers + (which are detected by the configure script). They can still be disable + selectively by specifying --without-computed-gotos. + +- Issue #9425: Create PyErr_WarnFormat() function, similar to PyErr_WarnEx() but + use PyUnicode_FromFormatV() to format the warning message. + +- Issue #8530: Prevent stringlib fastsearch from reading beyond the front of an + array. + +- Issue #5319: Print an error if flushing stdout fails at interpreter shutdown. + +- Issue #9337: The str() of a float or complex number is now identical to its + repr(). + +- Issue #9416: Fix some issues with complex formatting where the output with no + type specifier failed to match the str output: + + - format(complex(-0.0, 2.0), '-') omitted the real part from the output, + - format(complex(0.0, 2.0), '-') included a sign and parentheses. + +Extension Modules +----------------- + +- Issue #8013: time.asctime and time.ctime no longer call system + asctime and ctime functions. The year range for time.asctime is now + 1900 through maxint. The range for time.ctime is the same as for + time.localtime. The string produced by these functions is longer + than 24 characters when year is greater than 9999. + +- Issue #6608: time.asctime is now checking struct tm fields its input + before passing it to the system asctime. Patch by MunSic Jeong. + +- Issue #8734: Avoid crash in msvcrt.get_osfhandle() when an invalid file + descriptor is provided. Patch by Pascal Chambon. + +- Issue #7736: Release the GIL around calls to opendir() and closedir() in the + posix module. Patch by Marcin Bachry. + +- Issue #4835: make PyLong_FromSocket_t() and PyLong_AsSocket_t() private to the + socket module, and fix the width of socket descriptors to be correctly + detected under 64-bit Windows. + +- Issue #1027206: Support IDNA in gethostbyname, gethostbyname_ex, getaddrinfo + and gethostbyaddr. getnameinfo is now restricted to numeric addresses as + input. + +- Issue #9214: Set operations on a KeysView or ItemsView in collections now + correctly return a set. Patch by Eli Bendersky. + +- Issue #5737: Add Solaris-specific mnemonics in the errno module. Patch by + Matthew Ahrens. + +- Restore GIL in nis_cat in case of error. Decode NIS data to fs encoding, using + the surrogate error handler. + +- Issue #665761: ``functools.reduce()`` will no longer mask exceptions other + than ``TypeError`` raised by the iterator argument. + +- Issue #9570: Use PEP 383 decoding in os.mknod and os.mkfifo. + +- Issue #6915: Under Windows, os.listdir() didn't release the Global Interpreter + Lock around all system calls. Original patch by Ryan Kelly. + +- Issue #8524: Add a detach() method to socket objects, so as to put the socket + into the closed state without closing the underlying file descriptor. + +- Issue #477863: Emit a ResourceWarning at shutdown if gc.garbage is not empty. + +- Issue #6869: Fix a refcount problem in the _ctypes extension. + +- Issue #5504: ctypes should now work with systems where mmap can't be + PROT_WRITE and PROT_EXEC. + +- Issue #9507: Named tuple repr will now automatically display the right name in + a tuple subclass. + +- Issue #9324: Add parameter validation to signal.signal on Windows in order to + prevent crashes. + +- Issue #9526: Remove some outdated (int) casts that were preventing the array + module from working correctly with arrays of more than 2**31 elements. + +- Fix memory leak in ssl._ssl._test_decode_cert. + +- Issue #8065: Fix memory leak in readline module (from failure to free the + result of history_get_history_state()). + +- Issue #9450: Fix memory leak in readline.replace_history_item and + readline.remove_history_item for readline version >= 5.0. + +- Issue #8105: Validate file descriptor passed to mmap.mmap on Windows. + +- Issue #8046: Add context manager protocol support and .closed property to mmap + objects. + +Library +------- + +- Issue #7451: Improve decoding performance of JSON objects, and reduce the + memory consumption of said decoded objects when they use the same strings as + keys. + +- Issue #1100562: Fix deep-copying of objects derived from the list and dict + types. Patch by Michele Orrù and Björn Lindqvist. + +- Issue #9753: Fixed socket.dup, which did not always work correctly on Windows. + +- Issue #9421: Made the get methods consistently accept the vars and + default arguments on all parser classes. + +- Issue #7005: Fixed output of None values for RawConfigParser.write and + ConfigParser.write. + +- Issue #8990: array.fromstring() and array.tostring() get renamed to + frombytes() and tobytes(), respectively, to avoid confusion. Furthermore, + array.frombytes(), array.extend() as well as the array.array() constructor now + accept bytearray objects. Patch by Thomas Jollans. + +- Issue #808164: Fixed socket.close to avoid references to globals, to avoid + issues when socket.close is called from a __del__ method. + +- Issue #9706: ssl module provides a better error handling in various + circumstances. + +- Issue #1868: Eliminate subtle timing issues in thread-local objects by getting + rid of the cached copy of thread-local attribute dictionary. + +- Issue #1512791: In setframerate() in the wave module, non-integral frame rates + are rounded to the nearest integer. + +- Issue #8797: urllib2 does a retry for Basic Authentication failure instead of + falling into recursion. + +- Issue #1194222: email.utils.parsedate now returns RFC2822 compliant four + character years even if the message contains RFC822 two character years. + +- Issue #8750: Fixed MutableSet's methods to correctly handle reflexive + operations on its self, namely x -= x and x ^= x. + +- Issue #9129: smtpd.py is vulnerable to DoS attacks deriving from missing error + handling when accepting a new connection. + +- Issue #9601: ftplib now provides a workaround for non-compliant + implementations such as IIS shipped with Windows server 2003 returning invalid + response codes for MKD and PWD commands. + +- Issue #658749: asyncore's connect() method now correctly interprets winsock + errors. + +- Issue #9501: Fixed logging regressions in cleanup code. + +- Fix functools.total_ordering() to skip methods inherited from object. + +- Issue #9572: Importlib should not raise an exception if a directory it thought + it needed to create was done concurrently by another process. + +- Issue #9617: Signals received during a low-level write operation aren't + ignored by the buffered IO layer anymore. + +- Issue #843590: Make "macintosh" an alias to the "mac_roman" encoding. + +- Create os.fsdecode(): decode from the filesystem encoding with surrogateescape + error handler, or strict error handler on Windows. + +- Issue #3488: Provide convenient shorthand functions ``gzip.compress`` and + ``gzip.decompress``. Original patch by Anand B. Pillai. + +- Issue #8807: poplib.POP3_SSL class now accepts a context parameter, which is a + ssl.SSLContext object allowing bundling SSL configuration options, + certificates and private keys into a single (potentially long-lived) + structure. + +- Issue #8866: parameters passed to socket.getaddrinfo can now be specified as + single keyword arguments. + +- Address XXX comment in dis.py by having inspect.py prefer to reuse the dis.py + compiler flag values over defining its own. + +- Issue #9147: Added dis.code_info() which is similar to show_code() but returns + formatted code information in a string rather than displaying on screen. + +- Issue #9567: functools.update_wrapper now adds a __wrapped__ attribute + pointing to the original callable. + +- Issue #3445: functools.update_wrapper now tolerates missing attributes on + wrapped callables. + +- Issue #5867: Add abc.abstractclassmethod and abc.abstractstaticmethod. + +- Issue #9605: posix.getlogin() decodes the username with file filesystem + encoding and surrogateescape error handler. Patch written by David Watson. + +- Issue #9604: posix.initgroups() encodes the username using the fileystem + encoding and surrogateescape error handler. Patch written by David Watson. + +- Issue #9603: posix.ttyname() and posix.ctermid() decode the terminal name + using the filesystem encoding and surrogateescape error handler. Patch written + by David Watson. + +- Issue #7647: The posix module now has the ST_RDONLY and ST_NOSUID constants, + for use with the statvfs() function. Patch by Adam Jackson. + +- Issue #8688: MANIFEST files created by distutils now include a magic comment + indicating they are generated. Manually maintained MANIFESTs without this + marker will not be overwritten or removed. + +- Issue #7467: when reading a file from a ZIP archive, its CRC is checked and a + BadZipfile error is raised if it doesn't match (as used to be the case in + Python 2.5 and earlier). + +- Issue #9550: a BufferedReader could issue an additional read when the original + read request had been satisfied, which could block indefinitely when the + underlying raw IO channel was e.g. a socket. Report and original patch by + Jason V. Miller. + +- Issue #3757: thread-local objects now support cyclic garbage collection. + Thread-local objects involved in reference cycles will be deallocated timely + by the cyclic GC, even if the underlying thread is still running. + +- Issue #9452: Add read_file, read_string, and read_dict to the configparser + API; new source attribute to exceptions. + +- Issue #6231: Fix xml.etree.ElementInclude to include the tail of the current + node. + +- Issue #8047: Fix the xml.etree serializer to return bytes by default. Use + ``encoding="unicode"`` to generate a Unicode string. + +- Issue #8280: urllib2's Request method will remove fragments in the url. This + is how it is supposed to work, wget and curl do the same. Previous behavior + was wrong. + +- Issue #6683: For SMTP logins we now try all authentication methods advertised + by the server. Many servers are buggy and advertise authentication methods + they do not support in reality. + +- Issue #8814: function annotations (the ``__annotations__`` attribute) are now + included in the set of attributes copied by default by functools.wraps and + functools.update_wrapper. Patch by Terrence Cole. + +- Issue #2944: asyncore doesn't handle connection refused correctly. + +- Issue #4184: Private attributes on smtpd.SMTPChannel made public and deprecate + the private attributes. Add tests for smtpd module. + +- Issue #3196: email header decoding is now forgiving if an RFC2047 encoded word + encoded in base64 is lacking padding. + +- Issue #9444: Argparse now uses the first element of prefix_chars as the option + character for the added 'h/help' option if prefix_chars does not contain a + '-', instead of raising an error. + +- Issue #7372: Fix pstats regression when stripping paths from profile data + generated with the profile module. + +- Issue #9428: Fix running scripts with the profile/cProfile modules from the + command line. + +- Issue #7781: Fix restricting stats by entry counts in the pstats interactive + browser. + +- Issue #9209: Do not crash in the pstats interactive browser on invalid regular + expressions. + +- Update collections.OrderedDict to match the implementation in Py2.7 (based on + lists instead of weakly referenced Link objects). + +- Issue #8397: Raise an error when attempting to mix iteration and regular reads + on a BZ2File object, rather than returning incorrect results. + +- Issue #9448: Fix a leak of OS resources (mutexes or semaphores) when + re-initializing a buffered IO object by calling its ``__init__`` method. + +- Issue #1713: Fix os.path.ismount(), which returned true for symbolic links + across devices. + +- Issue #8826: Properly load old-style "expires" attribute in http.cookies. + +- Issue #1690103: Fix initial namespace for code run with trace.main(). + +- Issue #7395: Fix tracebacks in pstats interactive browser. + +- Issue #8230: Fix Lib/test/sortperf.py. + +- Issue #8620: when a cmd.Cmd() is fed input that reaches EOF without a final + newline, it no longer truncates the last character of the last command line. + +- Issue #5146: Handle UID THREAD command correctly in imaplib. + +- Issue #5147: Fix the header generated for cookie files written by + http.cookiejar.MozillaCookieJar. -- Issue #4871: The zipfile module now gives a more useful error message if - an attempt is made to use a string to specify the archive password. +- Issue #8198: In pydoc, output all help text to the correct stream when + sys.stdout is reassigned. -- Issue #10750: The ``raw`` attribute of buffered IO objects is now read-only. +- Issue #7909: Do not touch paths with the special prefixes ``\\.\`` or ``\\?\`` + in ntpath.normpath(). -- Issue #6791: Limit header line length (to 65535 bytes) in http.client - and http.server, to avoid denial of services from the other party. +- Issue #1286: Allow using fileinput.FileInput as a context manager. -- Issue #10404: Use ctl-button-1 on OSX for the context menu in Idle. +- Add lru_cache() decorator to the functools module. -- Issue #4188: Avoid creating dummy thread objects when logging operations - from the threading module (with the internal verbose flag activated). +Tools/Demos +----------- -- Issue #9721: Fix the behavior of urljoin when the relative url starts with a - ';' character. Patch by Wes Chow. +- Fix ``Tools/scripts/checkpyc.py`` after PEP 3147. -- Issue #10714: Limit length of incoming request in http.server to 65536 bytes - for security reasons. Initial patch by Ross Lagerwall. +- Issue #8867: Fix ``Tools/scripts/serve.py`` to work with files containing + non-ASCII content. -- Issue #9558: Fix distutils.command.build_ext with VS 8.0. +Tests +----- -- Issue #10695: passing the port as a string value to telnetlib no longer - causes debug mode to fail. +- Issue #9601: Provide a test case for ftplib.parse257. -- Issue #1078919: add_header now automatically RFC2231 encodes parameters - that contain non-ascii values. +- Issue #8857: Provide a test case for socket.getaddrinfo. -- Issue #10107: Warn about unsaved files in IDLE on OSX. +- Issue #7564: Skip test_ioctl if another process is attached to /dev/tty. -- Issue #7904: Changes to urllib.parse.urlsplit to handle schemes as defined by - RFC3986. Anything before :// is considered a scheme and is followed by an - authority (or netloc) and by '/' led path, which is optional. +- Issue #8433: Fix test_curses failure with newer versions of ncurses. -- Issue #10478: Reentrant calls inside buffered IO objects (for example by - way of a signal handler) now raise a RuntimeError instead of freezing the - current process. +- Issue #9496: Provide a test suite for the rlcompleter module. Patch by + Michele Orrù. -- Issue #10464: netrc now correctly handles lines with embedded '#' characters. +- Issue #8687: provide a test suite for sched.py module. -- Issue #1731717: Fixed the problem where subprocess.wait() could cause an - OSError exception when The OS had been told to ignore SIGCLD in our process - or otherwise not wait for exiting child processes. +Build +----- -- Issue #11500: Fixed a bug in the os x proxy bypass code for fully qualified - IP addresses in the proxy exception list. +- Issue #1303434: Generate ZIP file containing all PDBs. -Extension Modules ------------------ +- Issue #9193: PEP 3149 is accepted. -- Issue #12051: Fix segfault in json.dumps() while encoding highly-nested - objects using the C accelerations. +- Issue #3101: Helper functions _add_one_to_index_C() and _add_one_to_index_F() + become _Py_add_one_to_index_C() and _Py_add_one_to_index_F(), respectively. -- Issue #12017: Fix segfault in json.loads() while decoding highly-nested - objects using the C accelerations. +- Issue #9700: define HAVE_BROKEN_POSIX_SEMAPHORES under AIX 6.x. Patch by + Sébastien Sablé. -- Issue #1838: Prevent segfault in ctypes, when _as_parameter_ on a class is set - to an instance of the class. +- Don't run pgen twice when using make -j. -- Issue #678250: Make mmap flush a noop on ACCESS_READ and ACCESS_COPY. -Build ------ +What's New in Python 3.2 Alpha 1? +================================= -- Issue #11411: Fix 'make DESTDIR=' with a relative destination. +*Release date: 01-Aug-2010* -- Issue #11184: Fix large-file support on AIX. +Core and Builtins +----------------- -- Issue #941346: Fix broken shared library build on AIX. +- Issue #8991: convertbuffer() rejects discontigious buffers. -- Issue #7716: Under Solaris, don't assume existence of /usr/xpg4/bin/grep in - the configure script but use $GREP instead. Patch by Fabian Groffen. +- Issue #7616: Fix copying of overlapping memoryview slices with the Intel + compiler. -- Issue #10475: Don't hardcode compilers for LDSHARED/LDCXXSHARED on NetBSD - and DragonFly BSD. Patch by Nicolas Joly. +- Issue #8413: structsequence now subclasses tuple. -- Issue #10655: Fix the build on PowerPC on Linux with GCC when building with - timestamp profiling (--with-tsc): the preprocessor test for the PowerPC - support now looks for "__powerpc__" as well as "__ppc__": the latter seems to - only be present on OS X; the former is the correct one for Linux with GCC. +- Issue #8271: during the decoding of an invalid UTF-8 byte sequence, only the + start byte and the continuation byte(s) are now considered invalid, instead of + the number of bytes specified by the start byte. E.g.: + '\xf1\x80AB'.decode('utf-8', 'replace') now returns u'\ufffdAB' and replaces + with U+FFFD only the start byte ('\xf1') and the continuation byte ('\x80') + even if '\xf1' is the start byte of a 4-bytes sequence. Previous versions + returned a single u'\ufffd'. -- Issue #1099: Fix the build on MacOSX when building a framework with pydebug - using GCC 4.0. +- Issue #9011: A negated imaginary literal (e.g., "-7j") now has real part -0.0 + rather than 0.0. So "-7j" is now exactly equivalent to "-(7j)". -Tests ------ +- Be more specific in error messages about positional arguments. -- Issue #12205: Fix test_subprocess failure due to uninstalled test data. +- Issue #8949: "z" format of PyArg_Parse*() functions doesn't accept bytes + objects, as described in the documentation. -- Issue #11614: import __hello__ prints "Hello World!". Patch written by - Andreas Stührk. +- Issue #6543: Write the traceback in the terminal encoding instead of utf-8. + Fix the encoding of the modules filename. Patch written by Amaury Forgeot + d'Arc. -- Issue #5723: Improve json tests to be executed with and without accelerations. +- Issue #9011: Remove buggy and unnecessary (in 3.x) ST->AST compilation code + dealing with unary minus applied to a constant. The removed code was mutating + the ST, causing a second compilation to fail. -- Issue #11910: Fix test_heapq to skip the C tests when _heapq is missing. +- Issue #850997: mbcs encoding (Windows only) handles errors argument: strict + mode raises unicode errors. The encoder only supports "strict" and "replace" + error handlers, the decoder only supports "strict" and "ignore" error + handlers. Patch written by Mark Hammond. -- Fix test_startfile to wait for child process to terminate before finishing. +- Issue #8850: Remove "w" and "w#" formats from PyArg_Parse*() functions, use + "w*" format instead. Add tests for "w*" format. -- Issue #11719: Fix message about unexpected test_msilib skip on non-Windows - platforms. Patch by Nadeem Vawda. +- Issue #8592: PyArg_Parse*() functions raise a TypeError for "y", "u" and "Z" + formats if the string contains a null byte/character. Write unit tests for + string formats. -- Issue #11490: test_subprocess:test_leaking_fds_on_error no longer gives a - false positive if the last directory in the path is inaccessible. +- Issue #7490: To facilitate sharing of doctests between 2.x and 3.x test + suites, the IGNORE_EXCEPTION_DETAIL directive now also ignores the module + location of the raised exception. -- Issue #10822: Fix test_posix:test_getgroups failure under Solaris. Patch - by Ross Lagerwall. +- Issue #8969: On Windows, use mbcs codec in strict mode to encode and decode + filenames and enable os.fsencode(). -- Issue #6293: Have regrtest.py echo back sys.flags. This is done by default - in whole runs and enabled selectively using ``--header`` when running an - explicit list of tests. Original patch by Collin Winter. +- Issue #9058: Remove assertions about INT_MAX in UnicodeDecodeError. -- Issue #775964: test_grp now skips YP/NIS entries instead of failing when - encountering them. +- Issue #8941: Decoding big endian UTF-32 data in UCS-2 builds could crash the + interpreter with characters outside the Basic Multilingual Plane (higher than + 0x10000). -- Issue #7110: regrtest now sends test failure reports and single-failure - tracebacks to stderr rather than stdout. +- Issue #8950: (See also issue #5080). Py_ArgParse*() functions now raise + TypeError instead of giving a DeprecationWarning when a float is parsed using + the 'L' code (for long long). (All other integer codes already raise + TypeError in this case.) +- Issue #8922: Normalize the encoding name in PyUnicode_AsEncodedString() to + enable shortcuts for upper case encoding name. Add also a shortcut for + "iso-8859-1" in PyUnicode_AsEncodedString() and PyUnicode_Decode(). -What's New in Python 3.1.3? -=========================== +- Issue #8838: Remove codecs.charbuffer_encode() function. The buffer protocol + doesn't support "char buffer" anymore in Python 3. -*Release date: 2010-11-27* +- Issue #8339: Remove "t#" format of PyArg_Parse*() functions, use "s#" or "s*" + instead. codecs.charbuffer_encode() now accepts modifiable buffer objects + like bytearray. -Core and Builtins ------------------ +- Issue #8837: Remove "O?" format of PyArg_Parse*() functions. The format is no + used anymore and it was never documented. -- Issue #10391: Don't dereference invalid memory in error messages in the ast - module. +- In str.format(), raise a ValueError when indexes to arguments are too large. -Library -------- +- Issue #2844: Make int('42', n) consistently raise ValueError for invalid + integers n (including n = -909). -- Issue #2236: distutils' mkpath ignored the mode parameter. +- Issue #8188: Introduce a new scheme for computing hashes of numbers (instances + of int, float, complex, decimal.Decimal and fractions.Fraction) that makes it + easy to maintain the invariant that hash(x) == hash(y) whenever x and y have + equal value. -- Fix typo in one sdist option (medata-check). +- Issue #8748: Fix two issues with comparisons between complex and integer + objects. (1) The comparison could incorrectly return True in some cases + (2**53+1 == complex(2**53) == 2**53), breaking transitivity of equality. + (2) The comparison raised an OverflowError for large integers, leading to + unpredictable exceptions when combining integers and complex objects in sets + or dicts. -- Issue #10323: itertools.islice() now consumes the minimum number of - inputs before stopping. Formerly, the final state of the underlying - iterator was undefined. +- Issue #8766: Initialize _warnings module before importing the first module. + Fix a crash if an empty directory called "encodings" exists in sys.path. -- Issue #10565: The collections.Iterator ABC now checks for both - __iter__ and __next__. +- Issue #8589: Decode PYTHONWARNINGS environment variable with the file system + encoding and surrogateescape error handler instead of the locale encoding to + be consistent with os.environ. Add PySys_AddWarnOptionUnicode() function. -- Issue #10561: In pdb, clear the breakpoints by the breakpoint number. +- PyObject_Dump() encodes unicode objects to utf8 with backslashreplace (instead + of strict) error handler to escape surrogates. -- Issue #10459: Update CJK character names to Unicode 5.1. +- Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode + object to Py_FileSystemDefaultEncoding with the "surrogateescape" error + handler, and return bytes. If Py_FileSystemDefaultEncoding is not set, fall + back to UTF-8. -- Issue #10092: Properly reset locale in calendar.Locale*Calendar classes. +- Enable shortcuts for common encodings in PyUnicode_AsEncodedString() for any + error handler, not only the default error handler (strict). -- Issue #6098: Don't claim DOM level 3 conformance in minidom. +- Issue #8610: Load file system codec at startup, and display a fatal error on + failure. Set the file system encoding to utf-8 (instead of None) if getting + the locale encoding failed, or if nl_langinfo(CODESET) function is missing. -- Issue #5762: Fix AttributeError raised by ``xml.dom.minidom`` when an empty - XML namespace attribute is encountered. +- PyFile_FromFd() uses PyUnicode_DecodeFSDefault() instead of + PyUnicode_FromString() to support surrogates in the filename and use the right + encoding. -- Issue #1710703: Write structures for an empty ZIP archive when a ZipFile is - created in modes 'a' or 'w' and then closed without adding any files. Raise - BadZipfile (rather than IOError) when opening small non-ZIP files. +- Issue #7507: Quote "!" in pipes.quote(); it is special to some shells. -- Issue #4493: urllib.request adds '/' in front of path components which does not - start with '/. Common behavior exhibited by browsers and other clients. +- PyUnicode_DecodeFSDefaultAndSize() uses surrogateescape error handler. -- Issue #6378: idle.bat now runs with the appropriate Python version rather than - the system default. Patch by Sridhar Ratnakumar. +- Issue #8419: Prevent the dict constructor from accepting non-string keyword + arguments. -- Issue #10407: Fix two NameErrors in distutils. +- Issue #8124: PySys_WriteStdout() and PySys_WriteStderr() don't execute + indirectly Python signal handlers anymore because mywrite() ignores exceptions + (KeyboardInterrupt). -- Issue #10198: fix duplicate header written to wave files when writeframes() - is called without data. +- Issue #8092: Fix PyUnicode_EncodeUTF8() to support error handler producing + unicode string (eg. backslashreplace). -- Issue #10467: Fix BytesIO.readinto() after seeking into a position after the - end of the file. +- Issue #8485: PyUnicode_FSConverter() doesn't accept byteearray objects + anymore, you have to convert your bytearray filenames to bytes. -- Issue #1682942: configparser supports alternative option/value delimiters. +- Issue #7332: Remove the 16KB stack-based buffer in + PyMarshal_ReadLastObjectFromFile, which doesn't bring any noticeable benefit + compared to the dynamic memory allocation fallback. Patch by Charles-François + Natali. -Build ------ +- Issue #8417: Raise an OverflowError when an integer larger than sys.maxsize is + passed to bytes or bytearray. -- Don't run pgen twice when using make -j. +- Issue #7301: Add environment variable $PYTHONWARNINGS. -- Backport r83399 to allow test_distutils to pass on installed versions. +- Issue #8329: Don't return the same lists from select.select when no fds are + changed. -- Issue #1303434: Generate ZIP file containing all PDBs (already done for rc1). +- Issue #8259: 1L << (2**31) no longer produces an 'outrageous shift error' on + 64-bit machines. The shift count for either left or right shift is permitted + to be up to sys.maxsize. -- Stop packaging versioncheck tool (already done for rc1). +- Ensure that tokenization of identifiers is not affected by locale. -- Accept Oracle Berkeley DB 4.8, 5.0 and 5.1 as backend for the dbm extension. +- Issue #1222585: Added LDCXXSHARED for C++ support. Patch by Arfrever. -Tests ------ +- Raise a TypeError when trying to delete a T_STRING_INPLACE struct member. -- Issue #9424: Replace deprecated assert* methods in the Python test suite. +- Issue #8211: Save/restore CFLAGS around AC_PROG_CC in configure.in, in case it + is set. -Documentation -------------- +- Issue #8226: sys.setfilesystemencoding() raises a LookupError if the encoding + is unknown. -- Issue #10299: List the built-in functions in a table in functions.rst. +- Issue #1583863: A str subclass can now override the __str__ method. +- Issue #8014: Setting a T_UINT or T_PYSSIZET attribute of an object with + PyMemberDefs could produce an internal error; raise TypeError instead. -What's New in Python 3.1.3 release candidate 1? -=============================================== +- Issue #7845: Rich comparison methods on the complex type now return + NotImplemented rather than raising a TypeError when comparing with an + incompatible type; this allows user-defined classes to implement their own + comparisons with complex. -*Release date: 2010-11-13* +- Issue #3137: Don't ignore errors at startup, especially a keyboard interrupt + (SIGINT). If an error occurs while importing the site module, the error is + printed and Python exits. Initialize the GIL before importing the site module. -Core and Builtins ------------------ +- Issue #7173: Generator finalization could invalidate sys.exc_info(). -- Issue #10221: dict.pop(k) now has a key error message that includes the - missing key (same message d[k] returns for missing keys). +- Issue #7544: Preallocate thread memory before creating the thread to avoid a + fatal error in low memory condition. -- Issue #5437: A preallocated MemoryError instance should not hold traceback - data (including local variables caught in the stack trace) alive infinitely. +- Issue #7820: The parser tokenizer restores all bytes in the right if the BOM + check fails. -- Issue #10077: Fix logging of site module errors at startup. +- Handle errors from looking up __prepare__ correctly. -- Issue #10186: Fix the SyntaxError caret when the offset is equal to the length - of the offending line. +- Issue #5939: Add additional runtime checking to ensure a valid capsule in + Modules/_ctypes/callproc.c. -- Issue #9713, #10114: Parser functions (eg. PyParser_ASTFromFile) expects - filenames encoded to the filesystem encoding with surrogateescape error - handler (to support undecodable bytes), instead of UTF-8 in strict mode. +- Issue #7309: Fix unchecked attribute access when converting + UnicodeEncodeError, UnicodeDecodeError, and UnicodeTranslateError to strings. -- Issue #10006: type.__abstractmethods__ now raises an AttributeError. As a - result metaclasses can now be ABCs (see #9533). +- Issue #6902: Fix problem with built-in types format incorrectly with 0 + padding. -- Issue #9997: Don't let the name "top" have special significance in scope - resolution. +- Issue #7988: Fix default alignment to be right aligned for complex.__format__. + Now it matches other numeric types. -- Issue #9930: Remove bogus subtype check that was causing (e.g.) - float.__rdiv__(2.0, 3) to return NotImplemented instead of the - expected 1.5. +- Issue #5988: Remove deprecated functions PyOS_ascii_formatd, + PyOS_ascii_strtod, and PyOS_ascii_atof. Use PyOS_double_to_string and + PyOS_string_to_double instead. See issue #5835 for the original deprecations. -- Issue #9804: ascii() now always represents unicode surrogate pairs as - a single ``\UXXXXXXXX``, regardless of whether the character is printable - or not. Also, the "backslashreplace" error handler now joins surrogate - pairs into a single character on UCS-2 builds. +- Issue #7385: Fix a crash in `MemoryView_FromObject` when `PyObject_GetBuffer` + fails. Patch by Florent Xicluna. -- Issue #9797: pystate.c wrongly assumed that zero couldn't be a valid - thread-local storage key. +- Issue #7788: Fix an interpreter crash produced by deleting a list slice with + very large step value. -- Issue #9737: Fix a crash when trying to delete a slice or an item from - a memoryview object. +- Issue #7766: Change sys.getwindowsversion() return value to a named tuple and + add the additional members returned in an OSVERSIONINFOEX structure. The new + members are service_pack_major, service_pack_minor, suite_mask, and + product_type. -- Issue #7415: PyUnicode_FromEncodedObject() now uses the new buffer API - properly. Patch by Stefan Behnel. +- Issue #7561: Operations on empty bytearrays (such as `int(bytearray())`) could + crash in many places because of the PyByteArray_AS_STRING() macro returning + NULL. The macro now returns a statically allocated empty string instead. -- Restore GIL in nis_cat in case of error. +- Issue #6690: Optimize the bytecode for expressions such as `x in {1, 2, 3}`, + where the right hand operand is a set of constants, by turning the set into a + frozenset and pre-building it as a constant. The comparison operation is made + against the constant instead of building a new set each time it is executed (a + similar optimization already existed which turned a list of constants into a + pre-built tuple). Patch and additional tests by Dave Malcolm. -- Issue #9712: Fix tokenize on identifiers that start with non-ascii names. +- Issue #7622: Improve the split(), rsplit(), splitlines() and replace() methods + of bytes, bytearray and unicode objects by using a common implementation based + on stringlib's fast search. Patch by Florent Xicluna. -- Issue #9688: __basicsize__ and __itemsize__ must be accessed as Py_ssize_t. +- Issue #7632: Fix various str -> float conversion bugs present in 2.7 alpha 2, + including: (1) a serious 'wrong output' bug that could occur for long (> 40 + digit) input strings, (2) a crash in dtoa.c that occurred in debug builds when + parsing certain long numeric strings corresponding to subnormal values, (3) a + memory leak for some values large enough to cause overflow, and (4) a number + of flaws that could lead to incorrectly rounded results. -- Issue #5319: Print an error if flushing stdout fails at interpreter - shutdown. +- The __complex__ method is now looked up on the class of instances to make it + consistent with other special methods. -- Issue #8814: function annotations (the ``__annotations__`` attribute) - are now included in the set of attributes copied by default by - functools.wraps and functools.update_wrapper. Patch by Terrence Cole. +- Issue #7462: Implement the stringlib fast search algorithm for the `rfind`, + `rindex`, `rsplit` and `rpartition` methods. Patch by Florent Xicluna. -- Issue #83755: Implicit set-to-frozenset conversion was not thread-safe. +- Issue #7604: Deleting an unset slotted attribute did not raise an + AttributeError. -- Issue #10068: Global objects which have reference cycles with their module's - dict are now cleared again. This causes issue #7140 to appear again. +- Issue #7534: Fix handling of IEEE specials (infinities, nans, negative zero) + in ** operator. The behaviour now conforms to that described in C99 Annex F. -- Issue #9416: Fix some issues with complex formatting where the - output with no type specifier failed to match the str output: +- Issue #1811: improve accuracy and cross-platform consistency for true division + of integers: the result of a/b is now correctly rounded for ints a and b (at + least on IEEE 754 platforms), and in particular does not depend on the + internal representation of an int. - - format(complex(-0.0, 2.0), '-') omitted the real part from the output, - - format(complex(0.0, 2.0), '-') included a sign and parentheses. +- Issue #6834: replace the implementation for the 'python' and 'pythonw' + executables on OSX. -- Issue #7616: Fix copying of overlapping memoryview slices with the Intel - compiler. + These executables now work properly with the arch(1) command: ``arch -ppc + python`` will start a universal binary version of python in PPC mode (unlike + previous releases). -- Issue #8271: during the decoding of an invalid UTF-8 byte sequence, only the - start byte and the continuation byte(s) are now considered invalid, instead - of the number of bytes specified by the start byte. - E.g.: '\xf1\x80AB'.decode('utf-8', 'replace') now returns u'\ufffdAB' and - replaces with U+FFFD only the start byte ('\xf1') and the continuation byte - ('\x80') even if '\xf1' is the start byte of a 4-bytes sequence. - Previous versions returned a single u'\ufffd'. +- Issue #7466: Segmentation fault when the garbage collector is called in the + middle of populating a tuple. Patch by Florent Xicluna. -- Issue #6543: Write the traceback in the terminal encoding instead of utf-8. - Fix the encoding of the modules filename. Patch written by Amaury Forgeot - d'Arc. +- Issue #7419: setlocale() could crash the interpreter on Windows when called + with invalid values. -- Issue #9058: Remove assertions about INT_MAX in UnicodeDecodeError. +- Issue #6077: On Windows, files opened with tempfile.TemporaryFile in "wt+" + mode would appear truncated on the first '0x1a' byte (aka. Ctrl+Z). -- Issue #8941: decoding big endian UTF-32 data in UCS-2 builds could crash - the interpreter with characters outside the Basic Multilingual Plane - (higher than 0x10000). +- Issue #7085: Fix crash when importing some extensions in a thread on MacOSX + 10.6. -- In the str.format(), raise a ValueError when indexes to arguments are too - large. +- Issue #1757126: Fix the cyrillic-asian alias for the ptcp154 encoding. -- Issue #8766: Initialize _warnings module before importing the first module. - Fix a crash if an empty directory called "encodings" exists in sys.path. +- Issue #6970: Remove redundant calls when comparing objects that don't + implement the relevant rich comparison methods. -- PyObject_Dump() encodes unicode objects to utf8 with backslashreplace - (instead of strict) error handler to escape surrogates +- Issue #7298: Fixes for range and reversed(range(...)). Iteration over + range(a, b, c) incorrectly gave an empty iterator when a, b and c fit in C + long but the length of the range did not. Also fix several cases where + reversed(range(a, b, c)) gave wrong results, and fix a refleak for + reversed(range(a, b, c)) with large arguments. -- Issue #8124: PySys_WriteStdout() and PySys_WriteStderr() don't execute - indirectly Python signal handlers anymore because mywrite() ignores - exceptions (KeyboardInterrupt) +- Issue #7244: itertools.izip_longest() no longer ignores exceptions raised + during the formation of an output tuple. -- Issue #8092: Fix PyUnicode_EncodeUTF8() to support error handler producing - unicode string (eg. backslashreplace) +- Issue #3297: On wide unicode builds, do not split unicode characters into + surrogates. -- Issue #8014: Setting a T_UINT or T_PYSSIZET attribute of an object with - PyMemberDefs could produce an internal error; raise TypeError instead. +- Remove length limitation when constructing a complex number from a string. -- Issue #8417: Raise an OverflowError when an integer larger than sys.maxsize is - passed to bytes or bytearray. +- Issue #1087418: Boost performance of bitwise operations for longs. -- Issue #8329: Don't return the same lists from select.select when no fds are - changed. +- Support for AtheOS has been completely removed from the code base. It was + disabled since Python 3.0. -- Raise a TypeError when trying to delete a T_STRING_INPLACE struct member. +- Support for several legacy threading libraries has been disabled. These + libraries are: Mach C threads, SunOS LWP, GNU pth, Irix threads. Support code + will be entirely removed in 3.3. -- Issue #8226: sys.setfilesystemencoding() raises a LookupError if the encoding - is unknown +- Support for OSF* has been disabled. If nobody stands up, support will be + removed in 3.3. See . -- Issue #1583863: An str subclass can now override the __str__ method +- Peephole constant folding had missed UNARY_POSITIVE. -- Issue #7072: isspace(0xa0) is true on Mac OS X +- Issue #1722344: threading._shutdown() is now called in Py_Finalize(), which + fixes the problem of some exceptions being thrown at shutdown when the + interpreter is killed. Patch by Adam Olsen. -C-API ------ +- Issue #7147: Remove support for compiling Python without complex number + support. -- Issue #9834: Don't segfault in PySequence_GetSlice, PySequence_SetSlice, or - PySequence_DelSlice when the object doesn't have any mapping operations - defined. +- Issue #7120: logging: Removed import of multiprocessing which is causing crash + in GAE. -- Issue #5753: A new C API function, :cfunc:`PySys_SetArgvEx`, allows - embedders of the interpreter to set sys.argv without also modifying - sys.path. This helps fix `CVE-2008-5983 - `_. +- Issue #1754094: Improve the stack depth calculation in the compiler. There + should be no other effect than a small decrease in memory use. Patch by + Christopher Tur Lesniewski-Laas. -Library -------- +- Issue #7065: Fix a crash in bytes.maketrans and bytearray.maketrans when using + byte values greater than 127. Patch by Derk Drukker. -- Issue #5111: IPv6 Host in the Header is wrapped inside [ ]. Patch by Chandru. +- Issue #1571184: The Unicode database contains properties for more characters. + The tables for code points representing numeric values, white spaces or line + breaks are now generated from the official Unicode Character Database files, + and include information from the Unihan.txt file. -- Issue #4471: Properly shutdown socket in IMAP.shutdown(). Patch by - Lorenzo M. Catucci. +- Issue #7019: Raise ValueError when unmarshalling bad long data, instead of + producing internally inconsistent Python longs. -- Fix IMAP.login() to work properly. +- Issue #6990: Fix threading.local subclasses leaving old state around after a + reference cycle GC which could be recycled by new locals. -- Issue #10126: Fix distutils' test_build when Python was built with - --enable-shared. +- Issue #5460: Fix an ambiguity in the grammar. -- Fix typo in one sdist option (medata-check). +- Issue #1766304: Improve performance of membership tests on range objects. -- Issue #9199: Fix incorrect use of distutils.cmd.Command.announce. +- Issue #6713: Improve performance of integer -> string conversions. -- Issue #1718574: Fix options that were supposed to accept arguments but did - not in build_clib. +- Issue #6846: Fix bug where bytearray.pop() returns negative integers. -- Issue #9281: Prevent race condition with mkdir in distutils. Patch by - Arfrever. +- Issue #6750: A text file opened with io.open() could duplicate its output when + writing from multiple threads at the same time. -- Issue #10229: Fix caching error in gettext. +- Issue #6707: dir() on an uninitialized module caused a crash. -- Issue #10252: Close file objects in a timely manner in distutils code and - tests. Patch by Brian Brazil, completed by Éric Araujo. +- Issue #6540: Fixed crash for bytearray.translate() with invalid parameters. -- Issue #10311: The signal module now restores errno before returning from - its low-level signal handler. Patch by Hallvard B Furuseth. +- Issue #6573: set.union() stopped processing inputs if an instance of self + occurred in the argument chain. -- The keyword only restriction for the places argument in - unittest.TestCase.assert[Not]AlmostEqual methods has been removed. +- Issue #6070: On posix platforms import no longer copies the execute bit from + the .py file to the .pyc file if it is set. -- Issue #6706: asyncore accept() method no longer raises EWOULDBLOCK/ECONNABORTED - on incomplete connection attempt but returns None instead. +- Issue #1616979: Added the cp720 (Arabic DOS) encoding. -- Issue #10266: uu.decode didn't close in_file explicitly when it was given - as a filename. Patch by Brian Brazil. +- Issue #6428: Since Python 3.0, the __bool__ method must return a bool object, + and not an int. Fix the corresponding error message, and the documentation. -- Issue #10246: uu.encode didn't close file objects explicitly when filenames - were given to it. Patch by Brian Brazil. +- The deprecated PyCObject has been removed. -- Issue #10253: FileIO leaks a file descriptor when trying to open a file - for append that isn't seekable. Patch by Brian Brazil. +- Issue #6347: Include inttypes.h as well as stdint.h in pyport.h. This fixes a + build failure on HP-UX: int32_t and uint32_t are defined in inttypes.h instead + of stdint.h on that platform. -- Issue #5027: The standard ``xml`` namespace is now understood by - xml.sax.saxutils.XMLGenerator as being bound to - http://www.w3.org/XML/1998/namespace. Patch by Troy J. Farrell. +- Issue #6373: Fixed a SystemError when encoding with the latin-1 codec and the + 'surrogateescape' error handler, a string which contains unpaired surrogates. -- Issue #7761: telnetlib.interact failures on Windows fixed. +- Issue #4856: Remove checks for win NT. -- Issue #5117: Case normalization was needed on ntpath.relpath(). And - fixed root directory issue on posixpath.relpath(). (Ported working fixes - from ntpath) +- Issue #6687: PyBytes_FromObject() no longer accepts an integer as its argument + to construct a null-initialized bytes object. -- Issue #10041: The signature of optional arguments in socket.makefile() - didn't match that of io.open(), and they also didn't get forwarded - properly to TextIOWrapper in text mode. Patch by Kai Zhu. +- Issue #1023290: Add from_bytes() and to_bytes() methods to integers. These + methods allow the conversion of integers to bytes, and vice-versa. -- Issue #6612: Fix site and sysconfig to catch os.getcwd() error, eg. if the - current directory was deleted. Patch written by W. Trevor King. +- Issue #7382: Fix bug in bytes.__getnewargs__ that prevented bytes instances + from being copied with copy.copy(), and bytes subclasses from being pickled + properly. -- Issue #9759: GzipFile now raises ValueError when an operation is attempted - after the file is closed. Patch by Jeffrey Finkelstein. +- Code objects now support weak references. -- Issue #9042: Fix interaction of custom translation classes and caching in - gettext. +- Issue #7072: isspace(0xa0) is true on Mac OS X. -- Issue #9065: tarfile no longer uses "root" as the default for the uname and - gname field. +- Issue #8084: PEP 370 now conforms to system conventions for framework builds + on MacOS X. That is, "python setup.py install --user" will install into + "~/Library/Python/2.7" instead of "~/.local". -- Issue #8980: Fixed a failure in distutils.command check that was shadowed - by an environment that does not have docutils. Patch by Arfrever. +C-API +----- -- Issue #1050268: parseaddr now correctly quotes double quote and backslash - characters that appear inside quoted strings in email addresses. +- Issue #2443: A new macro, `Py_VA_COPY`, copies the state of the + variable argument list. `Py_VA_COPY` is equivalent to C99 + `va_copy`, but available on all python platforms. -- Issue #10004: quoprimime no longer generates a traceback when confronted - with invalid characters after '=' in a Q-encoded word. +- PySlice_GetIndicesEx now clips the step to [-PY_SSIZE_T_MAX, PY_SSIZE_T_MAX] + instead of [-PY_SSIZE_T_MAX-1, PY_SSIZE_T_MAX]. This makes it safe to do + "step = -step" when reversing a slice. -- Issue #9950: Fix socket.sendall() crash or misbehaviour when a signal is - received. Now sendall() properly calls signal handlers if necessary, - and retries sending if these returned successfully, including on sockets - with a timeout. +- Issue #5753: A new C API function, `PySys_SetArgvEx`, allows embedders of the + interpreter to set sys.argv without also modifying sys.path. This helps fix + `CVE-2008-5983 + `_. -- Issue #9936: Fixed executable lines' search in the trace module. +- Add PyArg_ValidateKeywordArguments, which checks if all keyword arguments are + strings in an efficient manner. -- Issue #9928: Properly initialize the types exported by the bz2 module. +- Issue #8276: PyEval_CallObject() is now only available in macro form. The + function declaration, which was kept for backwards compatibility reasons, is + now removed (the macro was introduced in 1997!). -- Issue #9854: The default read() implementation in io.RawIOBase now - handles non-blocking readinto() returning None correctly. +- Issue #7767: New function PyLong_AsLongLongAndOverflow added, analogous to + PyLong_AsLongAndOverflow. -- Issue #9853: Fix the signature of SSLSocket.recvfrom() and - SSLSocket.sendto() to match the corresponding socket methods. +- Make PyUnicode_CompareWithASCIIString return not equal if the Python string + has '\0' at the end. -- Issue #9792: In case of connection failure, socket.create_connection() - would swallow the exception and raise a new one, making it impossible - to fetch the original errno, or to filter timeout errors. Now the - original error is re-raised. +- Issue #5080: The argument parsing functions PyArg_ParseTuple, + PyArg_ParseTupleAndKeywords, PyArg_VaParse, PyArg_VaParseTupleAndKeywords and + PyArg_Parse now raise a DeprecationWarning for float arguments passed with the + 'L' format code. This will become a TypeError in a future version of Python, + to match the behaviour of the other integer format codes. -- Issue #9826: OrderedDict.__repr__ can now handle self-referential - values: d['x'] = d. +- Issue #7033: Function ``PyErr_NewExceptionWithDoc()`` added. -- Issue #9758: When fcntl.ioctl() was called with mutable_flag set to True, - and the passed buffer was exactly 1024 bytes long, the buffer wouldn't - be updated back after the system call. Original patch by Brian Brazil. +- Issue #7414: 'C' code wasn't being skipped properly (for keyword arguments) in + PyArg_ParseTupleAndKeywords. -- Issue #6656: fix locale.format_string to handle escaped percents - and mappings. +- Issue #7228: Add '%lld' and '%llu' support to PyString_FromFormat(V) and + PyErr_Format, on machines with HAVE_LONG_LONG defined. -- Issue #1100562: Fix deep-copying of objects derived from the list and - dict types. Patch by Michele Orrù and Björn Lindqvist. +- Issue #6151: Made PyDescr_COMMON conform to standard C (like PyObject_HEAD in + PEP 3123). The PyDescr_TYPE and PyDescr_NAME macros should be should used for + accessing the d_type and d_name members of structures using PyDescr_COMMON. -- Issue #9753: Fixed socket.dup, which did not always work correctly - on Windows. +- Issue #6405: Remove duplicate type declarations in descrobject.h. -- Issue #1868: Eliminate subtle timing issues in thread-local objects by - getting rid of the cached copy of thread-local attribute dictionary. +- The code flags for old __future__ features are now available again. -- Issue #8797: urllib2 does a retry for Basic Authentication failure instead of - falling into recursion. +- Issue #5954: Add a PyFrame_GetLineNumber() function to replace most uses of + PyCode_Addr2Line(). -- Issue #1194222: email.utils.parsedate now returns RFC2822 compliant four - character years even if the message contains RFC822 two character years. +- Issue #5959: Add a PyCode_NewEmpty() function to create a new empty code + object at a specified file, function, and line number. -- Issue #8750: Fixed MutableSet's methods to correctly handle - reflexive operations, namely x -= x and x ^= x. +- Issue #1419652: Change the first argument to PyImport_AppendInittab() to + ``const char *`` as the string is stored beyond the call. -- Issue #9129: smtpd.py is vulnerable to DoS attacks deriving from missing - error handling when accepting a new connection. +- Issue #2422: When compiled with the ``--with-valgrind`` option, the pymalloc + allocator will be automatically disabled when running under Valgrind. This + gives improved memory leak detection when running under Valgrind, while taking + advantage of pymalloc at other times. -- Issue #658749: asyncore's connect() method now correctly interprets winsock - errors. +Library +------- -- Issue #9214: Set operations on KeysView or ItemsView in the collections - module now correctly return a set. (Patch by Eli Bendersky.) +- In pdb, when Ctrl-C is entered while defining commands for a breakpoint, the + old commands are restored. -- Issue #9617: Signals received during a low-level write operation aren't - ignored by the buffered IO layer anymore. +- For traceback debugging, the pdb listing now also shows the locations where + the exception was originally (re)raised, if it differs from the last line + executed (e.g. in case of finally clauses). -- Issue #9605: posix.getlogin() decodes the username with file filesystem - encoding and surrogateescape error handler. Patch written by David Watson. +- The pdb command "source" has been added. It displays the source code for a + given object, if possible. -- Issue #9603: posix.ttyname() and posix.ctermid() decode the terminal name - using the filesystem encoding and surrogateescape error handler. Patch - written by David Watson. +- The pdb command "longlist" has been added. It displays the whole source code + for the current function. -- Issue #8688: MANIFEST files created by distutils now include a magic - comment indicating they are generated. Manually maintained MANIFESTs - without this marker will not be overwritten or removed. +- Issue #1503502: Make pdb.Pdb easier to subclass by putting message and error + output into methods. -- Issue #7467: when reading a file from a ZIP archive, its CRC is checked - and a BadZipfile error is raised if it doesn't match (as used to be the - case in Python 2.5 and earlier). +- Issue #809887: Make the output of pdb's breakpoint deletions more consistent; + emit a message when a breakpoint is enabled or disabled. -- Issue #9550: a BufferedReader could issue an additional read when the - original read request had been satisfied, which could block indefinitely - when the underlying raw IO channel was e.g. a socket. Report and original - patch by Jason V. Miller. +- Issue #5294: Fix the behavior of pdb's "continue" command when called in the + top-level debugged frame. -- Issue #6915: Under Windows, os.listdir() didn't release the Global - Interpreter Lock around all system calls. Original patch by Ryan Kelly. +- Issue #5727: Restore the ability to use readline when calling into pdb in + doctests. -- Issue #3757: thread-local objects now support cyclic garbage collection. - Thread-local objects involved in reference cycles will be deallocated - timely by the cyclic GC, even if the underlying thread is still running. +- Issue #6719: In pdb, do not stop somewhere in the encodings machinery if the + source file to be debugged is in a non-builtin encoding. -- Fix Issue8280 - urllib2's Request method will remove fragements in the url. - This is how it is supposed to work, wget and curl do the same. Previous - behavior was wrong. +- Issue #8048: Prevent doctests from failing when sys.displayhook has been + reassigned. -- Issue #2944: asyncore doesn't handle connection refused correctly. +- Issue #8015: In pdb, do not crash when an empty line is entered as a + breakpoint command. -- Issue #3196: email header decoding is now forgiving if an RFC2047 - encoded word encoded in base64 is lacking padding. +- In pdb, allow giving a line number to the "until" command. -- Issue #8447: Make distutils.sysconfig follow symlinks in the path to - the interpreter executable. This fixes a failure of test_httpservers - on OS X. +- Issue #1437051: For pdb, allow "continue" and related commands in .pdbrc + files. Also, add a command-line option "-c" that runs a command as if given + in .pdbrc. -- Issue #7372: Fix pstats regression when stripping paths from profile - data generated with the profile module. +- Issue #4179: In pdb, allow "list ." as a command to return to the currently + debugged line. -- Issue #4108: In urllib.robotparser, if there are multiple 'User-agent: *' +- Issue #4108: In urllib.robotparser, if there are multiple ``User-agent: *`` entries, consider the first one. -- Issue #8397: Raise an error when attempting to mix iteration and regular - reads on a BZ2File object, rather than returning incorrect results. - -- Issue #7909: Do not touch paths with the special prefixes ``\\.\`` - or ``\\?\`` in ntpath.normpath(). +- Issue #6630: Allow customizing regex flags when subclassing the + string.Template class. -- Issue #5294: Fix the behavior of pdb's "continue" command when called - in the top-level debugged frame. - -- Issue #5727: Restore the ability to use readline when calling into pdb - in doctests. +- Issue #9411: Allow specifying an encoding for config files in the configparser + module. -- Issue #6719: In pdb, do not stop somewhere in the encodings machinery - if the source file to be debugged is in a non-builtin encoding. +- Issue #1682942: Improvements to configparser: support alternate delimiters, + alternate comment prefixes and empty lines in values. -- Issue #8048: Prevent doctests from failing when sys.displayhook has - been reassigned. +- Issue #9354: Provide getsockopt() in asyncore's file_wrapper. -- Issue #8015: In pdb, do not crash when an empty line is entered as - a breakpoint command. +- Issue #8966: ctypes: Remove implicit bytes-unicode conversion. -- Issue #5146: Handle UID THREAD command correctly in imaplib. +- Issue #9378: python -m pickle will now load and display the + first object in the pickle file. -- Issue #5147: Fix the header generated for cookie files written by - http.cookiejar.MozillaCookieJar. +- Issue #4770: Restrict binascii module to accept only bytes (as specified). + And fix the email package to encode to ASCII instead of ``raw-unicode-escape`` + before ASCII-to-binary decoding. -- Issue #8198: In pydoc, output all help text to the correct stream - when sys.stdout is reassigned. +- Issue #9384: ``python -m tkinter`` will now display a simple demo applet. -- Issue #8230: Fix Lib/test/sortperf.py. +- The default size of the re module's compiled regular expression cache has been + increased from 100 to 500 and the cache replacement policy has changed from + simply clearing the entire cache on overflow to forgetting the least recently + used cached compiled regular expressions. This is a performance win for + applications that use a lot of regular expressions and limits the impact of + the performance hit anytime the cache is exceeded. -- Issue #7395: Fix tracebacks in pstats interactive browser. +- Issue #7113: Speed up loading in configparser. Patch by Łukasz Langa. -- Issue #1713: Fix os.path.ismount(), which returned true for symbolic links - across devices. +- Issue #9032: XML-RPC client retries the request on EPIPE error. The EPIPE + error occurs when the server closes the socket and the client sends a big + XML-RPC request. -- Issue #8826: Properly load old-style "expires" attribute in http.cookies. +- Issue #4629: getopt raises an error if an argument ends with "=", whereas + getopt doesn't accept a value (eg. --help= is rejected if getopt uses + ['help='] long options). -- Issue #1690103: Fix initial namespace for code run with trace.main(). +- Issue #7989: Added pure python implementation of the `datetime` module. The C + module is renamed to `_datetime` and if available, overrides all classes + defined in datetime with fast C impementation. Python implementation is based + on the original python prototype for the datetime module by Tim Peters with + minor modifications by the PyPy project. The test suite now tests `datetime` + module with and without `_datetime` acceleration using the same test cases. -- Issue #9354: Provide getsockopt() in asyncore's file_wrapper. +- Issue #7895: platform.mac_ver() no longer crashes after calling os.fork(). -- Issue #9448: Fix a leak of OS resources (mutexes or semaphores) when - re-initializing a buffered IO object by calling its ``__init__`` method. +- Issue #9323: Fixed a bug in trace.py that resulted in loosing the name of the + script being traced. Patch by Eli Bendersky. -- Issue #8620: when a Cmd is fed input that reaches EOF without a final - newline, it no longer truncates the last character of the last command line. +- Issue #9282: Fixed --listfuncs option of trace.py. Thanks Eli Bendersky for + the patch. - Issue #3704: http.cookiejar was not properly handling URLs with a / in the parameters. diff --cc Modules/_ssl.c index 5419059e2998,16fbb4db42ec..751e26e39abb --- a/Modules/_ssl.c +++ b/Modules/_ssl.c @@@ -1428,443 -1551,6 +1428,444 @@@ static PyTypeObject PySSLSocket_Type = PySSLMethods, /*tp_methods*/ }; + +/* + * _SSLContext objects + */ + +static PyObject * +context_new(PyTypeObject *type, PyObject *args, PyObject *kwds) +{ + char *kwlist[] = {"protocol", NULL}; + PySSLContext *self; + int proto_version = PY_SSL_VERSION_SSL23; + SSL_CTX *ctx = NULL; + + if (!PyArg_ParseTupleAndKeywords( + args, kwds, "i:_SSLContext", kwlist, + &proto_version)) + return NULL; + + PySSL_BEGIN_ALLOW_THREADS + if (proto_version == PY_SSL_VERSION_TLS1) + ctx = SSL_CTX_new(TLSv1_method()); + else if (proto_version == PY_SSL_VERSION_SSL3) + ctx = SSL_CTX_new(SSLv3_method()); +#ifndef OPENSSL_NO_SSL2 + else if (proto_version == PY_SSL_VERSION_SSL2) + ctx = SSL_CTX_new(SSLv2_method()); +#endif + else if (proto_version == PY_SSL_VERSION_SSL23) + ctx = SSL_CTX_new(SSLv23_method()); + else + proto_version = -1; + PySSL_END_ALLOW_THREADS + + if (proto_version == -1) { + PyErr_SetString(PyExc_ValueError, + "invalid protocol version"); + return NULL; + } + if (ctx == NULL) { + PyErr_SetString(PySSLErrorObject, + "failed to allocate SSL context"); + return NULL; + } + + assert(type != NULL && type->tp_alloc != NULL); + self = (PySSLContext *) type->tp_alloc(type, 0); + if (self == NULL) { + SSL_CTX_free(ctx); + return NULL; + } + self->ctx = ctx; + /* Defaults */ + SSL_CTX_set_verify(self->ctx, SSL_VERIFY_NONE, NULL); - SSL_CTX_set_options(self->ctx, SSL_OP_ALL); ++ SSL_CTX_set_options(self->ctx, ++ SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS); + +#define SID_CTX "Python" + SSL_CTX_set_session_id_context(self->ctx, (const unsigned char *) SID_CTX, + sizeof(SID_CTX)); +#undef SID_CTX + + return (PyObject *)self; +} + +static void +context_dealloc(PySSLContext *self) +{ + SSL_CTX_free(self->ctx); + Py_TYPE(self)->tp_free(self); +} + +static PyObject * +set_ciphers(PySSLContext *self, PyObject *args) +{ + int ret; + const char *cipherlist; + + if (!PyArg_ParseTuple(args, "s:set_ciphers", &cipherlist)) + return NULL; + ret = SSL_CTX_set_cipher_list(self->ctx, cipherlist); + if (ret == 0) { + /* Clearing the error queue is necessary on some OpenSSL versions, + otherwise the error will be reported again when another SSL call + is done. */ + ERR_clear_error(); + PyErr_SetString(PySSLErrorObject, + "No cipher can be selected."); + return NULL; + } + Py_RETURN_NONE; +} + +static PyObject * +get_verify_mode(PySSLContext *self, void *c) +{ + switch (SSL_CTX_get_verify_mode(self->ctx)) { + case SSL_VERIFY_NONE: + return PyLong_FromLong(PY_SSL_CERT_NONE); + case SSL_VERIFY_PEER: + return PyLong_FromLong(PY_SSL_CERT_OPTIONAL); + case SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT: + return PyLong_FromLong(PY_SSL_CERT_REQUIRED); + } + PyErr_SetString(PySSLErrorObject, + "invalid return value from SSL_CTX_get_verify_mode"); + return NULL; +} + +static int +set_verify_mode(PySSLContext *self, PyObject *arg, void *c) +{ + int n, mode; + if (!PyArg_Parse(arg, "i", &n)) + return -1; + if (n == PY_SSL_CERT_NONE) + mode = SSL_VERIFY_NONE; + else if (n == PY_SSL_CERT_OPTIONAL) + mode = SSL_VERIFY_PEER; + else if (n == PY_SSL_CERT_REQUIRED) + mode = SSL_VERIFY_PEER | SSL_VERIFY_FAIL_IF_NO_PEER_CERT; + else { + PyErr_SetString(PyExc_ValueError, + "invalid value for verify_mode"); + return -1; + } + SSL_CTX_set_verify(self->ctx, mode, NULL); + return 0; +} + +static PyObject * +get_options(PySSLContext *self, void *c) +{ + return PyLong_FromLong(SSL_CTX_get_options(self->ctx)); +} + +static int +set_options(PySSLContext *self, PyObject *arg, void *c) +{ + long new_opts, opts, set, clear; + if (!PyArg_Parse(arg, "l", &new_opts)) + return -1; + opts = SSL_CTX_get_options(self->ctx); + clear = opts & ~new_opts; + set = ~opts & new_opts; + if (clear) { +#ifdef HAVE_SSL_CTX_CLEAR_OPTIONS + SSL_CTX_clear_options(self->ctx, clear); +#else + PyErr_SetString(PyExc_ValueError, + "can't clear options before OpenSSL 0.9.8m"); + return -1; +#endif + } + if (set) + SSL_CTX_set_options(self->ctx, set); + return 0; +} + +static PyObject * +load_cert_chain(PySSLContext *self, PyObject *args, PyObject *kwds) +{ + char *kwlist[] = {"certfile", "keyfile", NULL}; + PyObject *certfile, *keyfile = NULL; + PyObject *certfile_bytes = NULL, *keyfile_bytes = NULL; + int r; + + errno = 0; + ERR_clear_error(); + if (!PyArg_ParseTupleAndKeywords(args, kwds, + "O|O:load_cert_chain", kwlist, + &certfile, &keyfile)) + return NULL; + if (keyfile == Py_None) + keyfile = NULL; + if (!PyUnicode_FSConverter(certfile, &certfile_bytes)) { + PyErr_SetString(PyExc_TypeError, + "certfile should be a valid filesystem path"); + return NULL; + } + if (keyfile && !PyUnicode_FSConverter(keyfile, &keyfile_bytes)) { + PyErr_SetString(PyExc_TypeError, + "keyfile should be a valid filesystem path"); + goto error; + } + PySSL_BEGIN_ALLOW_THREADS + r = SSL_CTX_use_certificate_chain_file(self->ctx, + PyBytes_AS_STRING(certfile_bytes)); + PySSL_END_ALLOW_THREADS + if (r != 1) { + if (errno != 0) { + ERR_clear_error(); + PyErr_SetFromErrno(PyExc_IOError); + } + else { + _setSSLError(NULL, 0, __FILE__, __LINE__); + } + goto error; + } + PySSL_BEGIN_ALLOW_THREADS + r = SSL_CTX_use_PrivateKey_file(self->ctx, + PyBytes_AS_STRING(keyfile ? keyfile_bytes : certfile_bytes), + SSL_FILETYPE_PEM); + PySSL_END_ALLOW_THREADS + Py_XDECREF(keyfile_bytes); + Py_XDECREF(certfile_bytes); + if (r != 1) { + if (errno != 0) { + ERR_clear_error(); + PyErr_SetFromErrno(PyExc_IOError); + } + else { + _setSSLError(NULL, 0, __FILE__, __LINE__); + } + return NULL; + } + PySSL_BEGIN_ALLOW_THREADS + r = SSL_CTX_check_private_key(self->ctx); + PySSL_END_ALLOW_THREADS + if (r != 1) { + _setSSLError(NULL, 0, __FILE__, __LINE__); + return NULL; + } + Py_RETURN_NONE; + +error: + Py_XDECREF(keyfile_bytes); + Py_XDECREF(certfile_bytes); + return NULL; +} + +static PyObject * +load_verify_locations(PySSLContext *self, PyObject *args, PyObject *kwds) +{ + char *kwlist[] = {"cafile", "capath", NULL}; + PyObject *cafile = NULL, *capath = NULL; + PyObject *cafile_bytes = NULL, *capath_bytes = NULL; + const char *cafile_buf = NULL, *capath_buf = NULL; + int r; + + errno = 0; + if (!PyArg_ParseTupleAndKeywords(args, kwds, + "|OO:load_verify_locations", kwlist, + &cafile, &capath)) + return NULL; + if (cafile == Py_None) + cafile = NULL; + if (capath == Py_None) + capath = NULL; + if (cafile == NULL && capath == NULL) { + PyErr_SetString(PyExc_TypeError, + "cafile and capath cannot be both omitted"); + return NULL; + } + if (cafile && !PyUnicode_FSConverter(cafile, &cafile_bytes)) { + PyErr_SetString(PyExc_TypeError, + "cafile should be a valid filesystem path"); + return NULL; + } + if (capath && !PyUnicode_FSConverter(capath, &capath_bytes)) { + Py_XDECREF(cafile_bytes); + PyErr_SetString(PyExc_TypeError, + "capath should be a valid filesystem path"); + return NULL; + } + if (cafile) + cafile_buf = PyBytes_AS_STRING(cafile_bytes); + if (capath) + capath_buf = PyBytes_AS_STRING(capath_bytes); + PySSL_BEGIN_ALLOW_THREADS + r = SSL_CTX_load_verify_locations(self->ctx, cafile_buf, capath_buf); + PySSL_END_ALLOW_THREADS + Py_XDECREF(cafile_bytes); + Py_XDECREF(capath_bytes); + if (r != 1) { + if (errno != 0) { + ERR_clear_error(); + PyErr_SetFromErrno(PyExc_IOError); + } + else { + _setSSLError(NULL, 0, __FILE__, __LINE__); + } + return NULL; + } + Py_RETURN_NONE; +} + +static PyObject * +context_wrap_socket(PySSLContext *self, PyObject *args, PyObject *kwds) +{ + char *kwlist[] = {"sock", "server_side", "server_hostname", NULL}; + PySocketSockObject *sock; + int server_side = 0; + char *hostname = NULL; + PyObject *hostname_obj, *res; + + /* server_hostname is either None (or absent), or to be encoded + using the idna encoding. */ + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O!i|O!:_wrap_socket", kwlist, + PySocketModule.Sock_Type, + &sock, &server_side, + Py_TYPE(Py_None), &hostname_obj)) { + PyErr_Clear(); + if (!PyArg_ParseTupleAndKeywords(args, kwds, "O!iet:_wrap_socket", kwlist, + PySocketModule.Sock_Type, + &sock, &server_side, + "idna", &hostname)) + return NULL; +#ifndef SSL_CTRL_SET_TLSEXT_HOSTNAME + PyMem_Free(hostname); + PyErr_SetString(PyExc_ValueError, "server_hostname is not supported " + "by your OpenSSL library"); + return NULL; +#endif + } + + res = (PyObject *) newPySSLSocket(self->ctx, sock, server_side, + hostname); + if (hostname != NULL) + PyMem_Free(hostname); + return res; +} + +static PyObject * +session_stats(PySSLContext *self, PyObject *unused) +{ + int r; + PyObject *value, *stats = PyDict_New(); + if (!stats) + return NULL; + +#define ADD_STATS(SSL_NAME, KEY_NAME) \ + value = PyLong_FromLong(SSL_CTX_sess_ ## SSL_NAME (self->ctx)); \ + if (value == NULL) \ + goto error; \ + r = PyDict_SetItemString(stats, KEY_NAME, value); \ + Py_DECREF(value); \ + if (r < 0) \ + goto error; + + ADD_STATS(number, "number"); + ADD_STATS(connect, "connect"); + ADD_STATS(connect_good, "connect_good"); + ADD_STATS(connect_renegotiate, "connect_renegotiate"); + ADD_STATS(accept, "accept"); + ADD_STATS(accept_good, "accept_good"); + ADD_STATS(accept_renegotiate, "accept_renegotiate"); + ADD_STATS(accept, "accept"); + ADD_STATS(hits, "hits"); + ADD_STATS(misses, "misses"); + ADD_STATS(timeouts, "timeouts"); + ADD_STATS(cache_full, "cache_full"); + +#undef ADD_STATS + + return stats; + +error: + Py_DECREF(stats); + return NULL; +} + +static PyObject * +set_default_verify_paths(PySSLContext *self, PyObject *unused) +{ + if (!SSL_CTX_set_default_verify_paths(self->ctx)) { + _setSSLError(NULL, 0, __FILE__, __LINE__); + return NULL; + } + Py_RETURN_NONE; +} + +static PyGetSetDef context_getsetlist[] = { + {"options", (getter) get_options, + (setter) set_options, NULL}, + {"verify_mode", (getter) get_verify_mode, + (setter) set_verify_mode, NULL}, + {NULL}, /* sentinel */ +}; + +static struct PyMethodDef context_methods[] = { + {"_wrap_socket", (PyCFunction) context_wrap_socket, + METH_VARARGS | METH_KEYWORDS, NULL}, + {"set_ciphers", (PyCFunction) set_ciphers, + METH_VARARGS, NULL}, + {"load_cert_chain", (PyCFunction) load_cert_chain, + METH_VARARGS | METH_KEYWORDS, NULL}, + {"load_verify_locations", (PyCFunction) load_verify_locations, + METH_VARARGS | METH_KEYWORDS, NULL}, + {"session_stats", (PyCFunction) session_stats, + METH_NOARGS, NULL}, + {"set_default_verify_paths", (PyCFunction) set_default_verify_paths, + METH_NOARGS, NULL}, + {NULL, NULL} /* sentinel */ +}; + +static PyTypeObject PySSLContext_Type = { + PyVarObject_HEAD_INIT(NULL, 0) + "_ssl._SSLContext", /*tp_name*/ + sizeof(PySSLContext), /*tp_basicsize*/ + 0, /*tp_itemsize*/ + (destructor)context_dealloc, /*tp_dealloc*/ + 0, /*tp_print*/ + 0, /*tp_getattr*/ + 0, /*tp_setattr*/ + 0, /*tp_reserved*/ + 0, /*tp_repr*/ + 0, /*tp_as_number*/ + 0, /*tp_as_sequence*/ + 0, /*tp_as_mapping*/ + 0, /*tp_hash*/ + 0, /*tp_call*/ + 0, /*tp_str*/ + 0, /*tp_getattro*/ + 0, /*tp_setattro*/ + 0, /*tp_as_buffer*/ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /*tp_flags*/ + 0, /*tp_doc*/ + 0, /*tp_traverse*/ + 0, /*tp_clear*/ + 0, /*tp_richcompare*/ + 0, /*tp_weaklistoffset*/ + 0, /*tp_iter*/ + 0, /*tp_iternext*/ + context_methods, /*tp_methods*/ + 0, /*tp_members*/ + context_getsetlist, /*tp_getset*/ + 0, /*tp_base*/ + 0, /*tp_dict*/ + 0, /*tp_descr_get*/ + 0, /*tp_descr_set*/ + 0, /*tp_dictoffset*/ + 0, /*tp_init*/ + 0, /*tp_alloc*/ + context_new, /*tp_new*/ +}; + + + #ifdef HAVE_OPENSSL_RAND /* helper routines for seeding the SSL PRNG */ @@@ -2141,44 -1802,5 +2142,45 @@@ PyInit__ssl(void PY_SSL_VERSION_SSL23); PyModule_AddIntConstant(m, "PROTOCOL_TLSv1", PY_SSL_VERSION_TLS1); + + /* protocol options */ - PyModule_AddIntConstant(m, "OP_ALL", SSL_OP_ALL); ++ PyModule_AddIntConstant(m, "OP_ALL", ++ SSL_OP_ALL & ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS); + PyModule_AddIntConstant(m, "OP_NO_SSLv2", SSL_OP_NO_SSLv2); + PyModule_AddIntConstant(m, "OP_NO_SSLv3", SSL_OP_NO_SSLv3); + PyModule_AddIntConstant(m, "OP_NO_TLSv1", SSL_OP_NO_TLSv1); + +#ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME + r = Py_True; +#else + r = Py_False; +#endif + Py_INCREF(r); + PyModule_AddObject(m, "HAS_SNI", r); + + /* OpenSSL version */ + /* SSLeay() gives us the version of the library linked against, + which could be different from the headers version. + */ + libver = SSLeay(); + r = PyLong_FromUnsignedLong(libver); + if (r == NULL) + return NULL; + if (PyModule_AddObject(m, "OPENSSL_VERSION_NUMBER", r)) + return NULL; + parse_openssl_version(libver, &major, &minor, &fix, &patch, &status); + r = Py_BuildValue("IIIII", major, minor, fix, patch, status); + if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION_INFO", r)) + return NULL; + r = PyUnicode_FromString(SSLeay_version(SSLEAY_VERSION)); + if (r == NULL || PyModule_AddObject(m, "OPENSSL_VERSION", r)) + return NULL; + + libver = OPENSSL_VERSION_NUMBER; + parse_openssl_version(libver, &major, &minor, &fix, &patch, &status); + r = Py_BuildValue("IIIII", major, minor, fix, patch, status); + if (r == NULL || PyModule_AddObject(m, "_OPENSSL_API_VERSION", r)) + return NULL; + return m; }