Victor Stinner [Thu, 16 Jan 2020 14:33:30 +0000 (15:33 +0100)]
bpo-39357: Remove buffering parameter of bz2.BZ2File (GH-18028)
Remove the buffering parameter of bz2.BZ2File. Since Python 3.0, it
was ignored and using it was emitting a DeprecationWarning. Pass an
open file object to control how the file is opened.
Remove base64.encodestring() and base64.decodestring(), aliases
deprecated since Python 3.1: use base64.encodebytes() and
base64.decodebytes() instead.
Oz N Tiram [Wed, 15 Jan 2020 23:55:13 +0000 (00:55 +0100)]
bpo-39348: Fix code highlight for the SOCK_NONBLOCK example (GH-18018)
The previous double colon was wrongly place directly after Therefore.
Which produced a block without syntax highlighting. This fixes it
by separating the double colon from the text. As a result, sphinx now
properly highlights the python code.
Daniel Olshansky [Wed, 15 Jan 2020 22:51:54 +0000 (17:51 -0500)]
bpo-37958: Adding get_profile_dict to pstats (GH-15495)
pstats is really useful or profiling and printing the output of the execution of some block of code, but I've found on multiple occasions when I'd like to access this output directly in an easily usable dictionary on which I can further analyze or manipulate.
The proposal is to add a function called get_profile_dict inside of pstats that'll automatically return this data the data in an easily accessible dict.
def get_profile_dict(self, keys_filter=None):
"""
Returns a dict where the key is a function name and the value is a dict
with the following keys:
- ncalls
- tottime
- percall_tottime
- cumtime
- percall_cumtime
- file_name
- line_number
keys_filter can be optionally set to limit the key-value pairs in the
retrieved dict.
"""
pstats_dict = {}
func_list = self.fcn_list[:] if self.fcn_list else list(self.stats.keys())
if not func_list:
return pstats_dict
pstats_dict["total_tt"] = float(f8(self.total_tt))
for func in func_list:
cc, nc, tt, ct, callers = self.stats[func]
file, line, func_name = func
ncalls = str(nc) if nc == cc else (str(nc) + '/' + str(cc))
tottime = float(f8(tt))
percall_tottime = -1 if nc == 0 else float(f8(tt/nc))
cumtime = float(f8(ct))
percall_cumtime = -1 if cc == 0 else float(f8(ct/cc))
func_dict = {
"ncalls": ncalls,
"tottime": tottime, # time spent in this function alone
"percall_tottime": percall_tottime,
"cumtime": cumtime, # time spent in the function plus all functions that this function called,
"percall_cumtime": percall_cumtime,
"file_name": file,
"line_number": line
}
func_dict_filtered = func_dict if not keys_filter else { key: func_dict[key] for key in keys_filter }
pstats_dict[func_name] = func_dict_filtered
As an example, this can be used to generate a stacked column chart using various visualization tools which will assist in easily identifying program bottlenecks.
Ammar Askar [Wed, 15 Jan 2020 16:48:40 +0000 (11:48 -0500)]
Fix compiler warning on Windows (GH-18012)
Python-ast.h contains a macro named Yield that conflicts with the Yield macro
in Windows system headers. While Python-ast.h has an "undef Yield" directive
to prevent this, it means that Python-ast.h must be included before Windows
header files or we run into a re-declaration warning. In commit c96be811fa7d
an include for pycore_pystate.h was added which indirectly includes Windows
header files. In this commit we re-order the includes to fix this warning.
On Unix, subprocess.Popen.send_signal() now polls the process status.
Polling reduces the risk of sending a signal to the wrong process if
the process completed, the Popen.returncode attribute is still None,
and the pid has been reassigned (recycled) to a new different
process.
* Add test for nested async decorator patch.
* Add test for side_effect and wraps with a function.
* Add test for side_effect with an exception in the iterable.
Václav Bartoš [Tue, 14 Jan 2020 17:57:04 +0000 (18:57 +0100)]
bpo-38361: syslog: fixed making default "ident" from sys.argv[0] (GH-16557)
The default value of "ident" parameter should be sys.argv[0] with leading path
components stripped, but it contained the last slash, i.e. '/program' instead
of 'program'.
Géry Ogam [Tue, 14 Jan 2020 11:58:29 +0000 (12:58 +0100)]
bpo-39048: Look up __aenter__ before __aexit__ in async with (GH-17609)
* Reorder the __aenter__ and __aexit__ checks for async with
* Add assertions for async with body being skipped
* Swap __aexit__ and __aenter__ loading in the documentation
Mark Shannon [Tue, 14 Jan 2020 10:12:45 +0000 (10:12 +0000)]
bpo-39156: Break up COMPARE_OP into four logically distinct opcodes. (GH-17754)
Break up COMPARE_OP into four logically distinct opcodes:
* COMPARE_OP for rich comparisons
* IS_OP for 'is' and 'is not' tests
* CONTAINS_OP for 'in' and 'is not' tests
* JUMP_IF_NOT_EXC_MATCH for checking exceptions in 'try-except' statements.
Julien Danjou [Mon, 13 Jan 2020 16:30:14 +0000 (17:30 +0100)]
bpo-39164: Add private _PyErr_GetExcInfo() function (GH-17752)
This adds a new function named _PyErr_GetExcInfo() that is a variation of the
original PyErr_GetExcInfo() taking a PyThreadState as its first argument.
That function allows to retrieve the exceptions information of any Python
thread -- not only the current one.
bpo-39299: Add more tests for mimetypes and its cli. (GH-17949)
* Add tests for case insensitive check of types and extensions as fallback.
* Add tests for data url with no comma.
* Add tests for read_mime_types.
* Add tests for the mimetypes cli and refactor __main__ code to private function.
* Restore mimetypes.knownfiles value at the end of the test.
This affects both shallow and deep copying.
My idea for a fix is to add property objects to the list of "atomic" objects in the copy module.
These already include types like functions and type objects.
I also added property objects to the unit tests test_copy_atomic and test_deepcopy_atomic. This is my first PR, and it's highly likely I've made some mistake, so please be kind :)
Kyle Stanley [Sun, 12 Jan 2020 11:02:50 +0000 (06:02 -0500)]
bpo-38356: Fix ThreadedChildWatcher thread leak in test_asyncio (GH-16552)
Motivation for this PR (comment from @vstinner in bpo issue):
```
Warning seen o AMD64 Ubuntu Shared 3.x buildbot:
https://buildbot.python.org/all/#/builders/141/builds/2593
test_devnull_output (test.test_a=syncio.test_subprocess.SubprocessThreadedWatcherTests) ...
Warning -- threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)
```
The following implementation details for the new method are TBD:
1) Public vs private
2) Inclusion in `close()`
3) Name
4) Coroutine vs subroutine method
5) *timeout* parameter
If it's a private method, 3, 4, and 5 are significantly less important.
I started with the most minimal implementation that fixes the dangling threads without modifying the regression tests, which I think is particularly important. I typically try to avoid directly modifying existing tests as much as possible unless it's necessary to do so. However, I am open to changing any part of this.
Vinay Sajip [Sun, 12 Jan 2020 08:54:00 +0000 (08:54 +0000)]
bpo-16575: Disabled checks for union types being passed by value. (GH-17960)
Although the underlying libffi issue remains open, adding these
checks have caused problems in third-party projects which are in
widespread use. See the issue for examples.
Dong-hee Na [Sat, 11 Jan 2020 17:39:15 +0000 (02:39 +0900)]
bpo-39259: nntplib.NNTP/NNTP_SSL now reject timeout = 0 (GH-17936)
nntplib.NNTP and nntplib.NNTP_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
Dong-hee Na [Fri, 10 Jan 2020 14:34:05 +0000 (23:34 +0900)]
bpo-39259: poplib now rejects timeout = 0 (GH-17912)
poplib.POP3 and poplib.POP3_SSL now raise a ValueError
if the given timeout for their constructor is zero to
prevent the creation of a non-blocking socket.
Dong-hee Na [Tue, 7 Jan 2020 17:28:10 +0000 (02:28 +0900)]
bpo-38615: Add timeout parameter for IMAP4 and IMAP4_SSL constructor (GH-17203)
imaplib.IMAP4 and imaplib.IMAP4_SSL now have an
optional *timeout* parameter for their constructors.
Also, the imaplib.IMAP4.open() method now has an optional *timeout* parameter
with this change. The overridden methods of imaplib.IMAP4_SSL and
imaplib.IMAP4_stream were applied to this change.
Khalid Mammadov [Sun, 5 Jan 2020 22:39:38 +0000 (22:39 +0000)]
bpo-39130: Dict reversed was added in v3.8 so should say in the doc as well (GH-17694)
To be consistent with document layout, it should say when the feature was added.
Although it's mentioned few other places in the doc but it's not explicitly say that at that place.