gh-140727: Restructure profiling documentation for PEP 799 (#142373)
* Add profiling module documentation structure
PEP 799 introduces a new `profiling` package that reorganizes Python's
profiling tools under a unified namespace. This commit adds the documentation
structure to match: a main entry point (profiling.rst) that helps users choose
between profilers, detailed docs for the tracing profiler (profiling-tracing.rst),
and separated pstats documentation.
The tracing profiler docs note that cProfile remains as a backward-compatible
alias, so existing code continues to work. The pstats module gets its own page
since it's used by both profiler types and deserves focused documentation.
* Add profiling.sampling documentation
The sampling profiler is new in Python 3.15 and works fundamentally differently
from the tracing profiler. It observes programs from outside by periodically
capturing stack snapshots, which means zero overhead on the profiled code. This
makes it practical for production use where you can attach to live servers.
The docs explain the key concepts (statistical vs deterministic profiling),
provide quick examples upfront, document all output formats (pstats, flamegraph,
gecko, heatmap), and cover the live TUI mode. The defaults table helps users
understand what happens without any flags.
* Wire profiling docs into the documentation tree
Add the new profiling module pages to the Debugging and Profiling toctree.
The order places the main profiling.rst entry point first, followed by the
two profiler implementations, then pstats, and finally the deprecated profile
module last.
* Convert profile.rst to deprecation stub
The pure Python profile module is deprecated in 3.15 and scheduled for removal
in 3.17. Users should migrate to profiling.tracing (or use the cProfile alias
which continues to work).
The page now focuses on helping existing users migrate: it shows the old vs new
import style, keeps the shared API reference since both modules have the same
interface, and preserves the calibration docs for anyone still using the pure
Python implementation during the transition period.
* Update CLI module references for profiling restructure
Point cProfile to profiling.tracing docs and add profiling.sampling to the
list of modules with CLI interfaces. The old profile-cli label no longer
exists after the documentation restructure.
* Update whatsnew to link to profiling module docs
Enable cross-references to the new profiling module documentation and update
the CLI examples to use the current syntax with the attach subcommand. Also
reference profiling.tracing instead of cProfile since that's the new canonical
name.
Mark Shannon [Mon, 8 Dec 2025 17:57:11 +0000 (17:57 +0000)]
GH-139757: JIT: Remove redundant branches to jumps in the assembly optimizer (GH-140800)
JIT: Remove redundant branches to jump in the assembly optimizer
* Refactor JIT assembly optimizer making instructions instances not just strings
* Remove redundant jumps and branches where legal to do so
* Modifies _BINARY_OP_SUBSCR_STR_INT to avoid excessive inlining depth
gh-138122: Make sampling profiler integration tests more resilient (#142382)
The tests were flaky on slow machines because subprocesses could finish
before enough samples were collected. This adds synchronization similar
to test_external_inspection: test scripts now signal when they start
working, and the profiler waits for this signal before sampling.
Test scripts now run in infinite loops until killed rather than for
fixed iterations, ensuring the profiler always has active work to
sample regardless of machine speed.
gh-138122: Implement frame caching in RemoteUnwinder to reduce memory reads (#142137)
This PR implements frame caching in the RemoteUnwinder class to significantly reduce memory reads when profiling remote processes with deep call stacks.
When cache_frames=True, the unwinder stores the frame chain from each sample and reuses unchanged portions in subsequent samples. Since most profiling samples capture similar call stacks (especially the parent frames), this optimization avoids repeatedly reading the same frame data from the target process.
The implementation adds a last_profiled_frame field to the thread state that tracks where the previous sample stopped. On the next sample, if the current frame chain reaches this marker, the cached frames from that point onward are reused instead of being re-read from remote memory.
The sampling profiler now enables frame caching by default.
gh-142236: Fix incorrect keyword suggestions for syntax errors (#142328)
The keyword typo suggestion mechanism in traceback would incorrectly
suggest replacements when the extracted source code was merely incomplete
rather than containing an actual typo. For example, when a missing comma
caused a syntax error, the system would suggest replacing 'print' with
'not' because the incomplete code snippet happened to pass validation.
The fix adds a validation step that first checks whether the original
extracted code raises a SyntaxError. If the code compiles successfully
or is simply incomplete (compile_command returns None), the function
returns early since there is no way to verify that a keyword replacement
would actually fix the problem.
Paresh Joshi [Sat, 6 Dec 2025 20:59:35 +0000 (02:29 +0530)]
gh-142006: Fix HeaderWriteError in email.policy.default caused by extra newline (#142008)
RDM: This fixes a subtle folding error that showed up when a token exactly filled a line and was followed by whitespace and a token with no folding whitespace that was longer than a line. In this particular circumstance the whitespace after the first token got pushed on to the next line, and then stolen to go in front of the next unfoldable token...leaving a completely empty line in the line buffer. That line got turned in to a newline, which is RFC illegal, and the newish security check caught it. The fix is to just delete that empty line from the buffer.
Sanyam Khurana [Sat, 6 Dec 2025 20:47:08 +0000 (15:47 -0500)]
gh-69113: Fix doctest to report line numbers for __test__ strings (#141624)
Enhanced the _find_lineno method in doctest to correctly identify and
report line numbers for doctests defined in __test__ dictionaries when
formatted as triple-quoted strings.
Finds a non-blank line in the test string and matches it in the source
file, verifying subsequent lines also match to handle duplicate lines.
Previously, doctest would report "line None" for __test__ dictionary
strings, making it difficult to debug failing tests.
Co-authored-by: Jurjen N.E. Bos <jneb@users.sourceforge.net> Co-authored-by: R. David Murray <rdmurray@bitdance.com>
Brett Cannon [Fri, 5 Dec 2025 23:27:16 +0000 (15:27 -0800)]
GH-142234: Allow `--enable-wasm-dynamic-linking` under WASI (GH-142235)
While CPython doesn't support `--enable-wasm-dynamic-linking`, external tools like componentize-py do and they have to patch around it. Since the flag is off by default, allowing the flag so external users can add/inject dynamic linking support seems acceptable.
gh-115952: Fix a potential virtual memory allocation denial of service in pickle (GH-119204)
Loading a small data which does not even involve arbitrary code execution
could consume arbitrary large amount of memory. There were three issues:
* PUT and LONG_BINPUT with large argument (the C implementation only).
Since the memo is implemented in C as a continuous dynamic array, a single
opcode can cause its resizing to arbitrary size. Now the sparsity of
memo indices is limited.
* BINBYTES, BINBYTES8 and BYTEARRAY8 with large argument. They allocated
the bytes or bytearray object of the specified size before reading into
it. Now they read very large data by chunks.
* BINSTRING, BINUNICODE, LONG4, BINUNICODE8 and FRAME with large
argument. They read the whole data by calling the read() method of
the underlying file object, which usually allocates the bytes object of
the specified size before reading into it. Now they read very large data
by chunks.
Also add comprehensive benchmark suite to measure performance and memory
impact of chunked reading optimization in PR #119204.
Features:
- Normal mode: benchmarks legitimate pickles (time/memory metrics)
- Antagonistic mode: tests malicious pickles (DoS protection)
- Baseline comparison: side-by-side comparison of two Python builds
- Support for truncated data and sparse memo attack vectors
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com> Co-authored-by: Gregory P. Smith <greg@krypto.org>
Sanyam Khurana [Fri, 5 Dec 2025 12:18:54 +0000 (07:18 -0500)]
gh-48752: Add readline.get_pre_input_hook() function (#141586)
Add readline.get_pre_input_hook() to retrieve the current pre-input
hook. This allows applications to save and restore the hook without
overwriting user settings.
This fixes a regression introduced in gh-140558. The interpreter would
crash if we inserted a non `str` key into a split table that matches an
existing key.
Sam Gross [Wed, 3 Dec 2025 20:48:44 +0000 (15:48 -0500)]
gh-140482: Avoid changing terminal settings in test_pty (gh-142202)
The previous test_spawn_doesnt_hang test had a few problems:
* It would cause ENV CHANGED failures if other tests were running
concurrently due to stty changes
* Typing while the test was running could cause it to fail
Mark Shannon [Wed, 3 Dec 2025 17:43:35 +0000 (17:43 +0000)]
GH-141794: Limit size of generated machine code. (GH-142228)
* Factor out bodies of the largest uops, to reduce jit code size.
* Factor out common assert, also reducing jit code size.
* Limit size of jitted code for a single executor to 1MB.
Petr Viktorin [Wed, 3 Dec 2025 15:14:53 +0000 (16:14 +0100)]
gh-135676: Reword the f-string (and t-string) section (GH-137469)
Much of the information was duplicated in stdtypes.rst; this PR keeps lexical/syntactical details in Lexical Analysis and the evaluation & runtime behaviour in Standard types, with cross-references between the two.
Since the t-string section only listed differences from f-strings, and the grammar for the two is equivalent, that section was moved to Standard types almost entirely.
Co-authored-by: Blaise Pabon <blaise@gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
gh-119342: Fix a potential denial of service in plistlib (GH-119343)
Reading a specially prepared small Plist file could cause OOM because file's
read(n) preallocates a bytes object for reading the specified amount of
data. Now plistlib reads large data by chunks, therefore the upper limit of
consumed memory is proportional to the size of the input file.
gh-119451: Fix a potential denial of service in http.client (GH-119454)
Reading the whole body of the HTTP response could cause OOM if
the Content-Length value is too large even if the server does not send
a large amount of data. Now the HTTP client reads large data by chunks,
therefore the amount of consumed memory is proportional to the amount
of sent data.
projects / thirdparty / Python / cpython.git / log
