* :ref:`calendar <calendar-cli>`
* :mod:`code`
* :ref:`compileall <compileall-cli>`
-* :mod:`cProfile`: see :ref:`profile <profile-cli>`
+* ``cProfile``: see :ref:`profiling.tracing <profiling-tracing-cli>`
* :ref:`dis <dis-cli>`
* :ref:`doctest <doctest-cli>`
* :mod:`!encodings.rot_13`
* :ref:`pickletools <pickletools-cli>`
* :ref:`platform <platform-cli>`
* :mod:`poplib`
-* :ref:`profile <profile-cli>`
-* :mod:`pstats`
+* :ref:`profiling.sampling <profiling-sampling>`
+* :ref:`profiling.tracing <profiling-tracing-cli>`
+* :ref:`pstats <pstats-cli>`
* :ref:`py_compile <py_compile-cli>`
* :mod:`pyclbr`
* :mod:`pydoc`
***********************
-Debugging and Profiling
+Debugging and profiling
***********************
These libraries help you with Python development: the debugger enables you to
bdb.rst
faulthandler.rst
pdb.rst
- profile.rst
+ profiling.rst
+ pstats.rst
timeit.rst
trace.rst
tracemalloc.rst
.. _profile:
-********************
-The Python Profilers
-********************
+****************************************
+:mod:`!profile` --- Pure Python profiler
+****************************************
-**Source code:** :source:`Lib/profile.py`, :source:`Lib/pstats.py`, and :source:`Lib/profile/sample.py`
+.. module:: profile
+ :synopsis: Pure Python profiler (deprecated).
+ :deprecated:
---------------
+**Source code:** :source:`Lib/profile.py`
-.. _profiler-introduction:
+--------------
-Introduction to the profilers
-=============================
+.. deprecated-removed:: 3.15 3.17
-.. index::
- single: statistical profiling
- single: profiling, statistical
- single: deterministic profiling
- single: profiling, deterministic
+The :mod:`profile` module is deprecated and will be removed in Python 3.17.
+Use :mod:`profiling.tracing` instead.
-Python provides both :dfn:`statistical profiling` and :dfn:`deterministic profiling` of
-Python programs. A :dfn:`profile` is a set of statistics that describes how
-often and for how long various parts of the program executed. These statistics
-can be formatted into reports via the :mod:`pstats` module.
+The :mod:`profile` module provides a pure Python implementation of a
+deterministic profiler. While useful for understanding profiler internals or
+extending profiler behavior through subclassing, its pure Python implementation
+introduces significant overhead compared to the C-based :mod:`profiling.tracing`
+module.
-The Python standard library provides three different profiling implementations:
+For most profiling tasks, use:
-**Statistical Profiler:**
+- :mod:`profiling.sampling` for production debugging with zero overhead
+- :mod:`profiling.tracing` for development and testing
-1. :mod:`!profiling.sampling` provides statistical profiling of running Python processes
- using periodic stack sampling. It can attach to any running Python process without
- requiring code modification or restart, making it ideal for production debugging.
-**Deterministic Profilers:**
+Migration
+=========
-2. :mod:`cProfile` is recommended for development and testing; it's a C extension with
- reasonable overhead that makes it suitable for profiling long-running
- programs. Based on :mod:`lsprof`, contributed by Brett Rosen and Ted
- Czotter.
+Migrating from :mod:`profile` to :mod:`profiling.tracing` is straightforward.
+The APIs are compatible::
-3. :mod:`profile`, a pure Python module whose interface is imitated by
- :mod:`cProfile`, but which adds significant overhead to profiled programs.
- If you're trying to extend the profiler in some way, the task might be easier
- with this module. Originally designed and written by Jim Roskind.
+ # Old (deprecated)
+ import profile
+ profile.run('my_function()')
-.. note::
+ # New (recommended)
+ import profiling.tracing
+ profiling.tracing.run('my_function()')
- The profiler modules are designed to provide an execution profile for a given
- program, not for benchmarking purposes (for that, there is :mod:`timeit` for
- reasonably accurate results). This particularly applies to benchmarking
- Python code against C code: the profilers introduce overhead for Python code,
- but not for C-level functions, and so the C code would seem faster than any
- Python one.
-
-**Profiler Comparison:**
-
-+-------------------+--------------------------+----------------------+----------------------+
-| Feature | Statistical | Deterministic | Deterministic |
-| | (``profiling.sampling``) | (``cProfile``) | (``profile``) |
-+===================+==========================+======================+======================+
-| **Target** | Running process | Code you run | Code you run |
-+-------------------+--------------------------+----------------------+----------------------+
-| **Overhead** | Virtually none | Moderate | High |
-+-------------------+--------------------------+----------------------+----------------------+
-| **Accuracy** | Statistical approx. | Exact call counts | Exact call counts |
-+-------------------+--------------------------+----------------------+----------------------+
-| **Setup** | Attach to any PID | Instrument code | Instrument code |
-+-------------------+--------------------------+----------------------+----------------------+
-| **Use Case** | Production debugging | Development/testing | Profiler extension |
-+-------------------+--------------------------+----------------------+----------------------+
-| **Implementation**| C extension | C extension | Pure Python |
-+-------------------+--------------------------+----------------------+----------------------+
+For most code, replacing ``import profile`` with ``import profiling.tracing``
+(and using ``profiling.tracing`` instead of ``profile`` throughout) provides
+a straightforward migration path.
.. note::
- The statistical profiler (:mod:`!profiling.sampling`) is recommended for most production
- use cases due to its extremely low overhead and ability to profile running processes
- without modification. It can attach to any Python process and collect performance
- data with minimal impact on execution speed, making it ideal for debugging
- performance issues in live applications.
-
-
-.. _statistical-profiling:
-
-What Is Statistical Profiling?
-==============================
-
-:dfn:`Statistical profiling` works by periodically interrupting a running
-program to capture its current call stack. Rather than monitoring every
-function entry and exit like deterministic profilers, it takes snapshots at
-regular intervals to build a statistical picture of where the program spends
-its time.
-
-The sampling profiler uses process memory reading (via system calls like
-``process_vm_readv`` on Linux, ``vm_read`` on macOS, and ``ReadProcessMemory`` on
-Windows) to attach to a running Python process and extract stack trace
-information without requiring any code modification or restart of the target
-process. This approach provides several key advantages over traditional
-profiling methods.
-
-The fundamental principle is that if a function appears frequently in the
-collected stack samples, it is likely consuming significant CPU time. By
-analyzing thousands of samples, the profiler can accurately estimate the
-relative time spent in different parts of the program. The statistical nature
-means that while individual measurements may vary, the aggregate results
-converge to represent the true performance characteristics of the application.
-
-Since statistical profiling operates externally to the target process, it
-introduces virtually no overhead to the running program. The profiler process
-runs separately and reads the target process memory without interrupting its
-execution. This makes it suitable for profiling production systems where
-performance impact must be minimized.
-
-The accuracy of statistical profiling improves with the number of samples
-collected. Short-lived functions may be missed or underrepresented, while
-long-running functions will be captured proportionally to their execution time.
-This characteristic makes statistical profiling particularly effective for
-identifying the most significant performance bottlenecks rather than providing
-exhaustive coverage of all function calls.
-
-Statistical profiling excels at answering questions like "which functions
-consume the most CPU time?" and "where should I focus optimization efforts?"
-rather than "exactly how many times was this function called?" The trade-off
-between precision and practicality makes it an invaluable tool for performance
-analysis in real-world applications.
-
-.. _profile-instant:
-
-Instant User's Manual
-=====================
-
-This section is provided for users that "don't want to read the manual." It
-provides a very brief overview, and allows a user to rapidly perform profiling
-on an existing application.
-
-**Statistical Profiling (Recommended for Production):**
-
-To profile an existing running process::
-
- python -m profiling.sampling 1234
-
-To profile with custom settings::
-
- python -m profiling.sampling -i 50 -d 30 1234
-
-**Deterministic Profiling (Development/Testing):**
-
-To profile a function that takes a single argument, you can do::
-
- import cProfile
- import re
- cProfile.run('re.compile("foo|bar")')
-
-(Use :mod:`profile` instead of :mod:`cProfile` if the latter is not available on
-your system.)
-
-The above action would run :func:`re.compile` and print profile results like
-the following::
-
- 214 function calls (207 primitive calls) in 0.002 seconds
-
- Ordered by: cumulative time
-
- ncalls tottime percall cumtime percall filename:lineno(function)
- 1 0.000 0.000 0.002 0.002 {built-in method builtins.exec}
- 1 0.000 0.000 0.001 0.001 <string>:1(<module>)
- 1 0.000 0.000 0.001 0.001 __init__.py:250(compile)
- 1 0.000 0.000 0.001 0.001 __init__.py:289(_compile)
- 1 0.000 0.000 0.000 0.000 _compiler.py:759(compile)
- 1 0.000 0.000 0.000 0.000 _parser.py:937(parse)
- 1 0.000 0.000 0.000 0.000 _compiler.py:598(_code)
- 1 0.000 0.000 0.000 0.000 _parser.py:435(_parse_sub)
-
-The first line indicates that 214 calls were monitored. Of those calls, 207
-were :dfn:`primitive`, meaning that the call was not induced via recursion. The
-next line: ``Ordered by: cumulative time`` indicates the output is sorted
-by the ``cumtime`` values. The column headings include:
-
-ncalls
- for the number of calls.
-
-tottime
- for the total time spent in the given function (and excluding time made in
- calls to sub-functions)
-
-percall
- is the quotient of ``tottime`` divided by ``ncalls``
-
-cumtime
- is the cumulative time spent in this and all subfunctions (from invocation
- till exit). This figure is accurate *even* for recursive functions.
-
-percall
- is the quotient of ``cumtime`` divided by primitive calls
-
-filename:lineno(function)
- provides the respective data of each function
-
-When there are two numbers in the first column (for example ``3/1``), it means
-that the function recursed. The second value is the number of primitive calls
-and the former is the total number of calls. Note that when the function does
-not recurse, these two values are the same, and only the single figure is
-printed.
-
-Instead of printing the output at the end of the profile run, you can save the
-results to a file by specifying a filename to the :func:`run` function::
-
- import cProfile
- import re
- cProfile.run('re.compile("foo|bar")', 'restats')
-
-The :class:`pstats.Stats` class reads profile results from a file and formats
-them in various ways.
-
-.. _sampling-profiler-cli:
-
-Statistical Profiler Command Line Interface
-===========================================
-
-.. program:: profiling.sampling
-
-The :mod:`!profiling.sampling` module can be invoked as a script to profile running processes::
-
- python -m profiling.sampling [options] PID
-
-**Basic Usage Examples:**
-
-Profile process 1234 for 10 seconds with default settings::
-
- python -m profiling.sampling 1234
-
-Profile with custom interval and duration, save to file::
-
- python -m profiling.sampling -i 50 -d 30 -o profile.stats 1234
-
-Generate collapsed stacks to use with tools like `flamegraph.pl
-<https://github.com/brendangregg/FlameGraph>`_::
-
- python -m profiling.sampling --collapsed 1234
-
-Profile all threads, sort by total time::
-
- python -m profiling.sampling -a --sort-tottime 1234
-
-Profile with real-time sampling statistics::
-
- python -m profiling.sampling --realtime-stats 1234
-
-**Command Line Options:**
-
-.. option:: PID
-
- Process ID of the Python process to profile (required)
-
-.. option:: -i, --interval INTERVAL
-
- Sampling interval in microseconds (default: 100)
-
-.. option:: -d, --duration DURATION
-
- Sampling duration in seconds (default: 10)
-
-.. option:: -a, --all-threads
-
- Sample all threads in the process instead of just the main thread
-
-.. option:: --native
-
- Include artificial ``<native>`` frames to denote calls to non-Python code.
-
-.. option:: --no-gc
-
- Don't include artificial ``<GC>`` frames to denote active garbage collection.
-
-.. option:: --realtime-stats
-
- Print real-time sampling statistics during profiling
-
-.. option:: --pstats
-
- Generate pstats output (default)
-
-.. option:: --collapsed
-
- Generate collapsed stack traces for flamegraphs
-
-.. option:: -o, --outfile OUTFILE
-
- Save output to a file
-
-**Sorting Options (pstats format only):**
+ The ``cProfile`` module remains available as a backward-compatible alias
+ to :mod:`profiling.tracing`. Existing code using ``import cProfile`` will
+ continue to work without modification.
-.. option:: --sort-nsamples
- Sort by number of direct samples
+:mod:`!profile` and :mod:`!profiling.tracing` module reference
+==============================================================
-.. option:: --sort-tottime
-
- Sort by total time
-
-.. option:: --sort-cumtime
-
- Sort by cumulative time (default)
-
-.. option:: --sort-sample-pct
-
- Sort by sample percentage
-
-.. option:: --sort-cumul-pct
-
- Sort by cumulative sample percentage
-
-.. option:: --sort-nsamples-cumul
-
- Sort by cumulative samples
-
-.. option:: --sort-name
-
- Sort by function name
-
-.. option:: -l, --limit LIMIT
-
- Limit the number of rows in the output (default: 15)
-
-.. option:: --no-summary
-
- Disable the summary section in the output
-
-**Understanding Statistical Profile Output:**
-
-The statistical profiler produces output similar to deterministic profilers but with different column meanings::
-
- Profile Stats:
- nsamples sample% tottime (ms) cumul% cumtime (ms) filename:lineno(function)
- 45/67 12.5 23.450 18.6 56.780 mymodule.py:42(process_data)
- 23/23 6.4 15.230 6.4 15.230 <built-in>:0(len)
-
-**Column Meanings:**
-
-- **nsamples**: ``direct/cumulative`` - Times function was directly executing / on call stack
-- **sample%**: Percentage of total samples where function was directly executing
-- **tottime**: Estimated time spent directly in this function
-- **cumul%**: Percentage of samples where function was anywhere on call stack
-- **cumtime**: Estimated cumulative time including called functions
-- **filename:lineno(function)**: Location and name of the function
-
-.. _profile-cli:
-
-Deterministic Profiler Command Line Interface
-=============================================
-
-.. program:: cProfile
-
-The files :mod:`cProfile` and :mod:`profile` can also be invoked as a script to
-profile another script. For example::
-
- python -m cProfile [-o output_file] [-s sort_order] (-m module | myscript.py)
-
-.. option:: -o <output_file>
-
- Writes the profile results to a file instead of to stdout.
-
-.. option:: -s <sort_order>
-
- Specifies one of the :func:`~pstats.Stats.sort_stats` sort values
- to sort the output by.
- This only applies when :option:`-o <cProfile -o>` is not supplied.
-
-.. option:: -m <module>
-
- Specifies that a module is being profiled instead of a script.
-
- .. versionadded:: 3.7
- Added the ``-m`` option to :mod:`cProfile`.
-
- .. versionadded:: 3.8
- Added the ``-m`` option to :mod:`profile`.
-
-The :mod:`pstats` module's :class:`~pstats.Stats` class has a variety of methods
-for manipulating and printing the data saved into a profile results file::
-
- import pstats
- from pstats import SortKey
- p = pstats.Stats('restats')
- p.strip_dirs().sort_stats(-1).print_stats()
-
-The :meth:`~pstats.Stats.strip_dirs` method removed the extraneous path from all
-the module names. The :meth:`~pstats.Stats.sort_stats` method sorted all the
-entries according to the standard module/line/name string that is printed. The
-:meth:`~pstats.Stats.print_stats` method printed out all the statistics. You
-might try the following sort calls::
-
- p.sort_stats(SortKey.NAME)
- p.print_stats()
-
-The first call will actually sort the list by function name, and the second call
-will print out the statistics. The following are some interesting calls to
-experiment with::
-
- p.sort_stats(SortKey.CUMULATIVE).print_stats(10)
-
-This sorts the profile by cumulative time in a function, and then only prints
-the ten most significant lines. If you want to understand what algorithms are
-taking time, the above line is what you would use.
-
-If you were looking to see what functions were looping a lot, and taking a lot
-of time, you would do::
-
- p.sort_stats(SortKey.TIME).print_stats(10)
-
-to sort according to time spent within each function, and then print the
-statistics for the top ten functions.
-
-You might also try::
-
- p.sort_stats(SortKey.FILENAME).print_stats('__init__')
-
-This will sort all the statistics by file name, and then print out statistics
-for only the class init methods (since they are spelled with ``__init__`` in
-them). As one final example, you could try::
-
- p.sort_stats(SortKey.TIME, SortKey.CUMULATIVE).print_stats(.5, 'init')
-
-This line sorts statistics with a primary key of time, and a secondary key of
-cumulative time, and then prints out some of the statistics. To be specific, the
-list is first culled down to 50% (re: ``.5``) of its original size, then only
-lines containing ``init`` are maintained, and that sub-sub-list is printed.
-
-If you wondered what functions called the above functions, you could now (``p``
-is still sorted according to the last criteria) do::
-
- p.print_callers(.5, 'init')
-
-and you would get a list of callers for each of the listed functions.
-
-If you want more functionality, you're going to have to read the manual, or
-guess what the following functions do::
-
- p.print_callees()
- p.add('restats')
-
-Invoked as a script, the :mod:`pstats` module is a statistics browser for
-reading and examining profile dumps. It has a simple line-oriented interface
-(implemented using :mod:`cmd`) and interactive help.
-
-:mod:`profile` and :mod:`cProfile` Module Reference
-=======================================================
-
-.. module:: cProfile
-.. module:: profile
- :synopsis: Python source profiler.
-
-Both the :mod:`profile` and :mod:`cProfile` modules provide the following
-functions:
+Both the :mod:`profile` and :mod:`profiling.tracing` modules provide the
+following functions:
.. function:: run(command, filename=None, sort=-1)
.. class:: Profile(timer=None, timeunit=0.0, subcalls=True, builtins=True)
This class is normally only used if more precise control over profiling is
- needed than what the :func:`cProfile.run` function provides.
+ needed than what the :func:`profiling.tracing.run` function provides.
A custom timer can be supplied for measuring how long code takes to run via
the *timer* argument. This must be a function that returns a single number
Directly using the :class:`Profile` class allows formatting profile results
without writing the profile data to a file::
- import cProfile, pstats, io
+ import profiling.tracing
+ import pstats
+ import io
from pstats import SortKey
- pr = cProfile.Profile()
+
+ pr = profiling.tracing.Profile()
pr.enable()
# ... do something ...
pr.disable()
print(s.getvalue())
The :class:`Profile` class can also be used as a context manager (supported
- only in :mod:`cProfile` module. see :ref:`typecontextmanager`)::
+ only in :mod:`profiling.tracing`, not in the deprecated :mod:`profile`
+ module; see :ref:`typecontextmanager`)::
- import cProfile
+ import profiling.tracing
- with cProfile.Profile() as pr:
+ with profiling.tracing.Profile() as pr:
# ... do something ...
pr.print_stats()
.. method:: enable()
- Start collecting profiling data. Only in :mod:`cProfile`.
+ Start collecting profiling data. Only in :mod:`profiling.tracing`.
.. method:: disable()
- Stop collecting profiling data. Only in :mod:`cProfile`.
+ Stop collecting profiling data. Only in :mod:`profiling.tracing`.
.. method:: create_stats()
The *sort* parameter specifies the sorting order of the displayed
statistics. It accepts a single key or a tuple of keys to enable
- multi-level sorting, as in :func:`Stats.sort_stats <pstats.Stats.sort_stats>`.
+ multi-level sorting, as in :meth:`pstats.Stats.sort_stats`.
.. versionadded:: 3.13
:meth:`~Profile.print_stats` now accepts a tuple of keys.
during the called command/function execution) no profiling results will be
printed.
-.. _profile-stats:
-
-The :class:`Stats` Class
-========================
-Analysis of the profiler data is done using the :class:`~pstats.Stats` class.
+Differences from :mod:`!profiling.tracing`
+==========================================
-.. module:: pstats
- :synopsis: Statistics object for use with the profiler.
+The :mod:`profile` module differs from :mod:`profiling.tracing` in several
+ways:
-.. class:: Stats(*filenames or profile, stream=sys.stdout)
+**Higher overhead.** The pure Python implementation is significantly slower
+than the C implementation, making it unsuitable for profiling long-running
+programs or performance-sensitive code.
- This class constructor creates an instance of a "statistics object" from a
- *filename* (or list of filenames) or from a :class:`Profile` instance. Output
- will be printed to the stream specified by *stream*.
+**Calibration support.** The :mod:`profile` module supports calibration to
+compensate for profiling overhead. This is not needed in :mod:`profiling.tracing`
+because the C implementation has negligible overhead.
- The file selected by the above constructor must have been created by the
- corresponding version of :mod:`profile` or :mod:`cProfile`. To be specific,
- there is *no* file compatibility guaranteed with future versions of this
- profiler, and there is no compatibility with files produced by other
- profilers, or the same profiler run on a different operating system. If
- several files are provided, all the statistics for identical functions will
- be coalesced, so that an overall view of several processes can be considered
- in a single report. If additional files need to be combined with data in an
- existing :class:`~pstats.Stats` object, the :meth:`~pstats.Stats.add` method
- can be used.
+**Custom timers.** Both modules support custom timers, but :mod:`profile`
+accepts timer functions that return tuples (like :func:`os.times`), while
+:mod:`profiling.tracing` requires a function returning a single number.
- Instead of reading the profile data from a file, a :class:`cProfile.Profile`
- or :class:`profile.Profile` object can be used as the profile data source.
+**Subclassing.** The pure Python implementation is easier to subclass and
+extend for custom profiling behavior.
- :class:`Stats` objects have the following methods:
-
- .. method:: strip_dirs()
-
- This method for the :class:`Stats` class removes all leading path
- information from file names. It is very useful in reducing the size of
- the printout to fit within (close to) 80 columns. This method modifies
- the object, and the stripped information is lost. After performing a
- strip operation, the object is considered to have its entries in a
- "random" order, as it was just after object initialization and loading.
- If :meth:`~pstats.Stats.strip_dirs` causes two function names to be
- indistinguishable (they are on the same line of the same filename, and
- have the same function name), then the statistics for these two entries
- are accumulated into a single entry.
-
-
- .. method:: add(*filenames)
-
- This method of the :class:`Stats` class accumulates additional profiling
- information into the current profiling object. Its arguments should refer
- to filenames created by the corresponding version of :func:`profile.run`
- or :func:`cProfile.run`. Statistics for identically named (re: file, line,
- name) functions are automatically accumulated into single function
- statistics.
-
-
- .. method:: dump_stats(filename)
-
- Save the data loaded into the :class:`Stats` object to a file named
- *filename*. The file is created if it does not exist, and is overwritten
- if it already exists. This is equivalent to the method of the same name
- on the :class:`profile.Profile` and :class:`cProfile.Profile` classes.
-
-
- .. method:: sort_stats(*keys)
-
- This method modifies the :class:`Stats` object by sorting it according to
- the supplied criteria. The argument can be either a string or a SortKey
- enum identifying the basis of a sort (example: ``'time'``, ``'name'``,
- ``SortKey.TIME`` or ``SortKey.NAME``). The SortKey enums argument have
- advantage over the string argument in that it is more robust and less
- error prone.
-
- When more than one key is provided, then additional keys are used as
- secondary criteria when there is equality in all keys selected before
- them. For example, ``sort_stats(SortKey.NAME, SortKey.FILE)`` will sort
- all the entries according to their function name, and resolve all ties
- (identical function names) by sorting by file name.
-
- For the string argument, abbreviations can be used for any key names, as
- long as the abbreviation is unambiguous.
-
- The following are the valid string and SortKey:
-
- +------------------+---------------------+----------------------+
- | Valid String Arg | Valid enum Arg | Meaning |
- +==================+=====================+======================+
- | ``'calls'`` | SortKey.CALLS | call count |
- +------------------+---------------------+----------------------+
- | ``'cumulative'`` | SortKey.CUMULATIVE | cumulative time |
- +------------------+---------------------+----------------------+
- | ``'cumtime'`` | N/A | cumulative time |
- +------------------+---------------------+----------------------+
- | ``'file'`` | N/A | file name |
- +------------------+---------------------+----------------------+
- | ``'filename'`` | SortKey.FILENAME | file name |
- +------------------+---------------------+----------------------+
- | ``'module'`` | N/A | file name |
- +------------------+---------------------+----------------------+
- | ``'ncalls'`` | N/A | call count |
- +------------------+---------------------+----------------------+
- | ``'pcalls'`` | SortKey.PCALLS | primitive call count |
- +------------------+---------------------+----------------------+
- | ``'line'`` | SortKey.LINE | line number |
- +------------------+---------------------+----------------------+
- | ``'name'`` | SortKey.NAME | function name |
- +------------------+---------------------+----------------------+
- | ``'nfl'`` | SortKey.NFL | name/file/line |
- +------------------+---------------------+----------------------+
- | ``'stdname'`` | SortKey.STDNAME | standard name |
- +------------------+---------------------+----------------------+
- | ``'time'`` | SortKey.TIME | internal time |
- +------------------+---------------------+----------------------+
- | ``'tottime'`` | N/A | internal time |
- +------------------+---------------------+----------------------+
-
- Note that all sorts on statistics are in descending order (placing most
- time consuming items first), where as name, file, and line number searches
- are in ascending order (alphabetical). The subtle distinction between
- ``SortKey.NFL`` and ``SortKey.STDNAME`` is that the standard name is a
- sort of the name as printed, which means that the embedded line numbers
- get compared in an odd way. For example, lines 3, 20, and 40 would (if
- the file names were the same) appear in the string order 20, 3 and 40.
- In contrast, ``SortKey.NFL`` does a numeric compare of the line numbers.
- In fact, ``sort_stats(SortKey.NFL)`` is the same as
- ``sort_stats(SortKey.NAME, SortKey.FILENAME, SortKey.LINE)``.
-
- For backward-compatibility reasons, the numeric arguments ``-1``, ``0``,
- ``1``, and ``2`` are permitted. They are interpreted as ``'stdname'``,
- ``'calls'``, ``'time'``, and ``'cumulative'`` respectively. If this old
- style format (numeric) is used, only one sort key (the numeric key) will
- be used, and additional arguments will be silently ignored.
-
- .. For compatibility with the old profiler.
-
- .. versionadded:: 3.7
- Added the SortKey enum.
-
- .. method:: reverse_order()
-
- This method for the :class:`Stats` class reverses the ordering of the
- basic list within the object. Note that by default ascending vs
- descending order is properly selected based on the sort key of choice.
-
- .. This method is provided primarily for compatibility with the old
- profiler.
-
-
- .. method:: print_stats(*restrictions)
-
- This method for the :class:`Stats` class prints out a report as described
- in the :func:`profile.run` definition.
-
- The order of the printing is based on the last
- :meth:`~pstats.Stats.sort_stats` operation done on the object (subject to
- caveats in :meth:`~pstats.Stats.add` and
- :meth:`~pstats.Stats.strip_dirs`).
-
- The arguments provided (if any) can be used to limit the list down to the
- significant entries. Initially, the list is taken to be the complete set
- of profiled functions. Each restriction is either an integer (to select a
- count of lines), or a decimal fraction between 0.0 and 1.0 inclusive (to
- select a percentage of lines), or a string that will be interpreted as a
- regular expression (to pattern match the standard name that is printed).
- If several restrictions are provided, then they are applied sequentially.
- For example::
-
- print_stats(.1, 'foo:')
-
- would first limit the printing to first 10% of list, and then only print
- functions that were part of filename :file:`.\*foo:`. In contrast, the
- command::
-
- print_stats('foo:', .1)
-
- would limit the list to all functions having file names :file:`.\*foo:`,
- and then proceed to only print the first 10% of them.
-
-
- .. method:: print_callers(*restrictions)
-
- This method for the :class:`Stats` class prints a list of all functions
- that called each function in the profiled database. The ordering is
- identical to that provided by :meth:`~pstats.Stats.print_stats`, and the
- definition of the restricting argument is also identical. Each caller is
- reported on its own line. The format differs slightly depending on the
- profiler that produced the stats:
-
- * With :mod:`profile`, a number is shown in parentheses after each caller
- to show how many times this specific call was made. For convenience, a
- second non-parenthesized number repeats the cumulative time spent in the
- function at the right.
-
- * With :mod:`cProfile`, each caller is preceded by three numbers: the
- number of times this specific call was made, and the total and
- cumulative times spent in the current function while it was invoked by
- this specific caller.
-
-
- .. method:: print_callees(*restrictions)
-
- This method for the :class:`Stats` class prints a list of all function
- that were called by the indicated function. Aside from this reversal of
- direction of calls (re: called vs was called by), the arguments and
- ordering are identical to the :meth:`~pstats.Stats.print_callers` method.
-
-
- .. method:: get_stats_profile()
-
- This method returns an instance of StatsProfile, which contains a mapping
- of function names to instances of FunctionProfile. Each FunctionProfile
- instance holds information related to the function's profile such as how
- long the function took to run, how many times it was called, etc...
-
- .. versionadded:: 3.9
- Added the following dataclasses: StatsProfile, FunctionProfile.
- Added the following function: get_stats_profile.
.. _deterministic-profiling:
-What Is Deterministic Profiling?
+What is deterministic profiling?
================================
:dfn:`Deterministic profiling` is meant to reflect the fact that all *function
call*, *function return*, and *exception* events are monitored, and precise
timings are made for the intervals between these events (during which time the
user's code is executing). In contrast, :dfn:`statistical profiling` (which is
-provided by the :mod:`!profiling.sampling` module) periodically samples the effective instruction pointer, and
-deduces where time is being spent. The latter technique traditionally involves
-less overhead (as the code does not need to be instrumented), but provides only
-relative indications of where time is being spent.
+provided by the :mod:`profiling.sampling` module) periodically samples the
+effective instruction pointer, and deduces where time is being spent. The
+latter technique traditionally involves less overhead (as the code does not
+need to be instrumented), but provides only relative indications of where time
+is being spent.
In Python, since there is an interpreter active during execution, the presence
of instrumented code is not required in order to do deterministic profiling.
the accuracy of the clock (less than one clock tick), but it *can* accumulate
and become very significant.
-The problem is more important with :mod:`profile` than with the lower-overhead
-:mod:`cProfile`. For this reason, :mod:`profile` provides a means of
-calibrating itself for a given platform so that this error can be
-probabilistically (on the average) removed. After the profiler is calibrated, it
-will be more accurate (in a least square sense), but it will sometimes produce
-negative numbers (when call counts are exceptionally low, and the gods of
-probability work against you :-). ) Do *not* be alarmed by negative numbers in
-the profile. They should *only* appear if you have calibrated your profiler,
-and the results are actually better than without calibration.
+The problem is more important with the deprecated :mod:`profile` module than
+with the lower-overhead :mod:`profiling.tracing`. For this reason,
+:mod:`profile` provides a means of calibrating itself for a given platform so
+that this error can be probabilistically (on the average) removed. After the
+profiler is calibrated, it will be more accurate (in a least square sense), but
+it will sometimes produce negative numbers (when call counts are exceptionally
+low, and the gods of probability work against you :-). ) Do *not* be alarmed
+by negative numbers in the profile. They should *only* appear if you have
+calibrated your profiler, and the results are actually better than without
+calibration.
.. _profile-calibration:
If you have a choice, you are better off choosing a smaller constant, and then
your results will "less often" show up as negative in profile statistics.
+
.. _profile-timers:
Using a custom timer
pr = profile.Profile(your_time_func)
The resulting profiler will then call ``your_time_func``. Depending on whether
-you are using :class:`profile.Profile` or :class:`cProfile.Profile`,
+you are using :class:`profile.Profile` or :class:`profiling.tracing.Profile`,
``your_time_func``'s return value will be interpreted differently:
:class:`profile.Profile`
replacement dispatch method that best handles your timer call, along with the
appropriate calibration constant.
-:class:`cProfile.Profile`
+:class:`profiling.tracing.Profile`
``your_time_func`` should return a single number. If it returns integers,
you can also invoke the class constructor with a second argument specifying
the real duration of one unit of time. For example, if
``your_integer_time_func`` returns times measured in thousands of seconds,
you would construct the :class:`Profile` instance as follows::
- pr = cProfile.Profile(your_integer_time_func, 0.001)
+ pr = profiling.tracing.Profile(your_integer_time_func, 0.001)
- As the :class:`cProfile.Profile` class cannot be calibrated, custom timer
- functions should be used with care and should be as fast as possible. For
- the best results with a custom timer, it might be necessary to hard-code it
- in the C source of the internal :mod:`!_lsprof` module.
+ As the :class:`profiling.tracing.Profile` class cannot be calibrated, custom
+ timer functions should be used with care and should be as fast as possible.
+ For the best results with a custom timer, it might be necessary to hard-code
+ it in the C source of the internal :mod:`!_lsprof` module.
Python 3.3 adds several new functions in :mod:`time` that can be used to make
precise measurements of process or wall-clock time. For example, see
:func:`time.perf_counter`.
+
+
+.. seealso::
+
+ :mod:`profiling`
+ Overview of Python profiling tools.
+
+ :mod:`profiling.tracing`
+ Recommended replacement for this module.
+
+ :mod:`pstats`
+ Statistical analysis and formatting for profile data.
--- /dev/null
+.. highlight:: shell-session
+
+.. _profiling-module:
+
+***************************************
+:mod:`profiling` --- Python profilers
+***************************************
+
+.. module:: profiling
+ :synopsis: Python profiling tools for performance analysis.
+
+.. versionadded:: 3.15
+
+**Source code:** :source:`Lib/profiling/`
+
+--------------
+
+.. index::
+ single: statistical profiling
+ single: profiling, statistical
+ single: deterministic profiling
+ single: profiling, deterministic
+
+
+Introduction to profiling
+=========================
+
+A :dfn:`profile` is a set of statistics that describes how often and for how
+long various parts of a program execute. These statistics help identify
+performance bottlenecks and guide optimization efforts. Python provides two
+fundamentally different approaches to collecting this information: statistical
+sampling and deterministic tracing.
+
+The :mod:`profiling` package organizes Python's built-in profiling tools under
+a single namespace. It contains two submodules, each implementing a different
+profiling methodology:
+
+:mod:`profiling.sampling`
+ A statistical profiler that periodically samples the call stack. Run scripts
+ directly or attach to running processes by PID. Provides multiple output
+ formats (flame graphs, heatmaps, Firefox Profiler), GIL analysis, GC tracking,
+ and multiple profiling modes (wall-clock, CPU, GIL) with virtually no overhead.
+
+:mod:`profiling.tracing`
+ A deterministic profiler that traces every function call, return, and
+ exception event. Provides exact call counts and precise timing information,
+ capturing every invocation including very fast functions.
+
+.. note::
+
+ The profiler modules are designed to provide an execution profile for a
+ given program, not for benchmarking purposes. For benchmarking, use the
+ :mod:`timeit` module, which provides reasonably accurate timing
+ measurements. This distinction is particularly important when comparing
+ Python code against C code: deterministic profilers introduce overhead for
+ Python code but not for C-level functions, which can skew comparisons.
+
+
+.. _choosing-a-profiler:
+
+Choosing a profiler
+===================
+
+For most performance analysis, use the statistical profiler
+(:mod:`profiling.sampling`). It has minimal overhead, works for both development
+and production, and provides rich visualization options including flamegraphs,
+heatmaps, GIL analysis, and more.
+
+Use the deterministic profiler (:mod:`profiling.tracing`) when you need **exact
+call counts** and cannot afford to miss any function calls. Since it instruments
+every function call and return, it will capture even very fast functions that
+complete between sampling intervals. The tradeoff is higher overhead.
+
+The following table summarizes the key differences:
+
++--------------------+------------------------------+------------------------------+
+| Feature | Statistical sampling | Deterministic |
+| | (:mod:`profiling.sampling`) | (:mod:`profiling.tracing`) |
++====================+==============================+==============================+
+| **Overhead** | Virtually none | Moderate |
++--------------------+------------------------------+------------------------------+
+| **Accuracy** | Statistical estimate | Exact call counts |
++--------------------+------------------------------+------------------------------+
+| **Output formats** | pstats, flamegraph, heatmap, | pstats |
+| | gecko, collapsed | |
++--------------------+------------------------------+------------------------------+
+| **Profiling modes**| Wall-clock, CPU, GIL | Wall-clock |
++--------------------+------------------------------+------------------------------+
+| **Special frames** | GC, native (C extensions) | N/A |
++--------------------+------------------------------+------------------------------+
+| **Attach to PID** | Yes | No |
++--------------------+------------------------------+------------------------------+
+
+
+When to use statistical sampling
+--------------------------------
+
+The statistical profiler (:mod:`profiling.sampling`) is recommended for most
+performance analysis tasks. Use it the same way you would use
+:mod:`profiling.tracing`::
+
+ python -m profiling.sampling run script.py
+
+One of the main strengths of the sampling profiler is its variety of output
+formats. Beyond traditional pstats tables, it can generate interactive
+flamegraphs that visualize call hierarchies, line-level source heatmaps that
+show exactly where time is spent in your code, and Firefox Profiler output for
+timeline-based analysis.
+
+The profiler also provides insight into Python interpreter behavior that
+deterministic profiling cannot capture. Use ``--mode gil`` to identify GIL
+contention in multi-threaded code, ``--mode cpu`` to measure actual CPU time
+excluding I/O waits, or inspect ``<GC>`` frames to understand garbage collection
+overhead. The ``--native`` option reveals time spent in C extensions, helping
+distinguish Python overhead from library performance.
+
+For multi-threaded applications, the ``-a`` option samples all threads
+simultaneously, showing how work is distributed. And for production debugging,
+the ``attach`` command connects to any running Python process by PID without
+requiring a restart or code changes.
+
+
+When to use deterministic tracing
+---------------------------------
+
+The deterministic profiler (:mod:`profiling.tracing`) instruments every function
+call and return. This approach has higher overhead than sampling, but guarantees
+complete coverage of program execution.
+
+The primary reason to choose deterministic tracing is when you need exact call
+counts. Statistical profiling estimates frequency based on sampling, which may
+undercount short-lived functions that complete between samples. If you need to
+verify that an optimization actually reduced the number of function calls, or
+if you want to trace the complete call graph to understand caller-callee
+relationships, deterministic tracing is the right choice.
+
+Deterministic tracing also excels at capturing functions that execute in
+microseconds. Such functions may not appear frequently enough in statistical
+samples, but deterministic tracing records every invocation regardless of
+duration.
+
+
+Quick start
+===========
+
+This section provides the minimal steps needed to start profiling. For complete
+documentation, see the dedicated pages for each profiler.
+
+
+Statistical profiling
+---------------------
+
+To profile a script, use the :mod:`profiling.sampling` module with the ``run``
+command::
+
+ python -m profiling.sampling run script.py
+ python -m profiling.sampling run -m mypackage.module
+
+This runs the script under the profiler and prints a summary of where time was
+spent. For an interactive flamegraph::
+
+ python -m profiling.sampling run --flamegraph script.py
+
+To profile an already-running process, use the ``attach`` command with the
+process ID::
+
+ python -m profiling.sampling attach 1234
+
+For custom settings, specify the sampling interval (in microseconds) and
+duration (in seconds)::
+
+ python -m profiling.sampling run -i 50 -d 30 script.py
+
+
+Deterministic profiling
+-----------------------
+
+To profile a script from the command line::
+
+ python -m profiling.tracing script.py
+
+To profile a piece of code programmatically:
+
+.. code-block:: python
+
+ import profiling.tracing
+ profiling.tracing.run('my_function()')
+
+This executes the given code under the profiler and prints a summary showing
+exact function call counts and timing.
+
+
+.. _profile-output:
+
+Understanding profile output
+============================
+
+Both profilers collect function-level statistics, though they present them in
+different formats. The sampling profiler offers multiple visualizations
+(flamegraphs, heatmaps, Firefox Profiler, pstats tables), while the
+deterministic profiler produces pstats-compatible output. Regardless of format,
+the underlying concepts are the same.
+
+Key profiling concepts:
+
+**Direct time** (also called *self time* or *tottime*)
+ Time spent executing code in the function itself, excluding time spent in
+ functions it called. High direct time indicates the function contains
+ expensive operations.
+
+**Cumulative time** (also called *total time* or *cumtime*)
+ Time spent in the function and all functions it called. This measures the
+ total cost of calling a function, including its entire call subtree.
+
+**Call count** (also called *ncalls* or *samples*)
+ How many times the function was called (deterministic) or sampled
+ (statistical). In deterministic profiling, this is exact. In statistical
+ profiling, it represents the number of times the function appeared in a
+ stack sample.
+
+**Primitive calls**
+ Calls that are not induced by recursion. When a function recurses, the total
+ call count includes recursive invocations, but primitive calls counts only
+ the initial entry. Displayed as ``total/primitive`` (for example, ``3/1``
+ means three total calls, one primitive).
+
+**Caller/Callee relationships**
+ Which functions called a given function (callers) and which functions it
+ called (callees). Flamegraphs visualize this as nested rectangles; pstats
+ can display it via the :meth:`~pstats.Stats.print_callers` and
+ :meth:`~pstats.Stats.print_callees` methods.
+
+
+Legacy compatibility
+====================
+
+For backward compatibility, the ``cProfile`` module remains available as an
+alias to :mod:`profiling.tracing`. Existing code using ``import cProfile`` will
+continue to work without modification in all future Python versions.
+
+.. deprecated:: 3.15
+
+ The pure Python :mod:`profile` module is deprecated and will be removed in
+ Python 3.17. Use :mod:`profiling.tracing` (or its alias ``cProfile``)
+ instead. See :mod:`profile` for migration guidance.
+
+
+.. seealso::
+
+ :mod:`profiling.sampling`
+ Statistical sampling profiler with flamegraphs, heatmaps, and GIL analysis.
+ Recommended for most users.
+
+ :mod:`profiling.tracing`
+ Deterministic tracing profiler for exact call counts.
+
+ :mod:`pstats`
+ Statistics analysis and formatting for profile data.
+
+ :mod:`timeit`
+ Module for measuring execution time of small code snippets.
+
+
+.. rubric:: Submodules
+
+.. toctree::
+ :maxdepth: 1
+
+ profiling.tracing.rst
+ profiling.sampling.rst
--- /dev/null
+.. highlight:: shell-session
+
+.. _profiling-sampling:
+
+***************************************************
+:mod:`profiling.sampling` --- Statistical profiler
+***************************************************
+
+.. module:: profiling.sampling
+ :synopsis: Statistical sampling profiler for Python processes.
+
+.. versionadded:: 3.15
+
+**Source code:** :source:`Lib/profiling/sampling/`
+
+--------------
+
+.. image:: tachyon-logo.png
+ :alt: Tachyon logo
+ :align: center
+ :width: 300px
+
+The :mod:`profiling.sampling` module, named **Tachyon**, provides statistical
+profiling of Python programs through periodic stack sampling. Tachyon can
+run scripts directly or attach to any running Python process without requiring
+code changes or restarts. Because sampling occurs externally to the target
+process, overhead is virtually zero, making Tachyon suitable for both
+development and production environments.
+
+
+What is statistical profiling?
+==============================
+
+Statistical profiling builds a picture of program behavior by periodically
+capturing snapshots of the call stack. Rather than instrumenting every function
+call and return as deterministic profilers do, Tachyon reads the call stack at
+regular intervals to record what code is currently running.
+
+This approach rests on a simple principle: functions that consume significant
+CPU time will appear frequently in the collected samples. By gathering thousands
+of samples over a profiling session, Tachyon constructs an accurate statistical
+estimate of where time is spent. The more samples collected, the
+more precise this estimate becomes.
+
+
+How time is estimated
+---------------------
+
+The time values shown in Tachyon's output are **estimates derived from sample
+counts**, not direct measurements. Tachyon counts how many times each function
+appears in the collected samples, then multiplies by the sampling interval to
+estimate time.
+
+For example, with a 100 microsecond sampling interval over a 10-second profile,
+Tachyon collects approximately 100,000 samples. If a function appears in 5,000
+samples (5% of total), Tachyon estimates it consumed 5% of the 10-second
+duration, or about 500 milliseconds. This is a statistical estimate, not a
+precise measurement.
+
+The accuracy of these estimates depends on sample count. With 100,000 samples,
+a function showing 5% has a margin of error of roughly ±0.5%. With only 1,000
+samples, the same 5% measurement could actually represent anywhere from 3% to
+7% of real time.
+
+This is why longer profiling durations and shorter sampling intervals produce
+more reliable results---they collect more samples. For most performance
+analysis, the default settings provide sufficient accuracy to identify
+bottlenecks and guide optimization efforts.
+
+Because sampling is statistical, results will vary slightly between runs. A
+function showing 12% in one run might show 11% or 13% in the next. This is
+normal and expected. Focus on the overall pattern rather than exact percentages,
+and don't worry about small variations between runs.
+
+
+When to use a different approach
+--------------------------------
+
+Statistical sampling is not ideal for every situation.
+
+For very short scripts that complete in under one second, the profiler may not
+collect enough samples for reliable results. Use :mod:`profiling.tracing`
+instead, or run the script in a loop to extend profiling time.
+
+When you need exact call counts, sampling cannot provide them. Sampling
+estimates frequency from snapshots, so if you need to know precisely how many
+times a function was called, use :mod:`profiling.tracing`.
+
+When comparing two implementations where the difference might be only 1-2%,
+sampling noise can obscure real differences. Use :mod:`timeit` for
+micro-benchmarks or :mod:`profiling.tracing` for precise measurements.
+
+
+The key difference from :mod:`profiling.tracing` is how measurement happens.
+A tracing profiler instruments your code, recording every function call and
+return. This provides exact call counts and precise timing but adds overhead
+to every function call. A sampling profiler, by contrast, observes the program
+from outside at fixed intervals without modifying its execution. Think of the
+difference like this: tracing is like having someone follow you and write down
+every step you take, while sampling is like taking photographs every second
+and inferring your path from those snapshots.
+
+This external observation model is what makes sampling profiling practical for
+production use. The profiled program runs at full speed because there is no
+instrumentation code running inside it, and the target process is never stopped
+or paused during sampling---Tachyon reads the call stack directly from the
+process's memory while it continues to run. You can attach to a live server,
+collect data, and detach without the application ever knowing it was observed.
+The trade-off is that very short-lived functions may be missed if they happen
+to complete between samples.
+
+Statistical profiling excels at answering the question, "Where is my program
+spending time?" It reveals hotspots and bottlenecks in production code where
+deterministic profiling overhead would be unacceptable. For exact call counts
+and complete call graphs, use :mod:`profiling.tracing` instead.
+
+
+Quick examples
+==============
+
+Profile a script and see the results immediately::
+
+ python -m profiling.sampling run script.py
+
+Profile a module with arguments::
+
+ python -m profiling.sampling run -m mypackage.module arg1 arg2
+
+Generate an interactive flame graph::
+
+ python -m profiling.sampling run --flamegraph -o profile.html script.py
+
+Attach to a running process by PID::
+
+ python -m profiling.sampling attach 12345
+
+Use live mode for real-time monitoring (press ``q`` to quit)::
+
+ python -m profiling.sampling run --live script.py
+
+Profile for 60 seconds with a faster sampling rate::
+
+ python -m profiling.sampling run -d 60 -i 50 script.py
+
+Generate a line-by-line heatmap::
+
+ python -m profiling.sampling run --heatmap script.py
+
+
+Commands
+========
+
+Tachyon operates through two subcommands that determine how to obtain the
+target process.
+
+
+The ``run`` command
+-------------------
+
+The ``run`` command launches a Python script or module and profiles it from
+startup::
+
+ python -m profiling.sampling run script.py
+ python -m profiling.sampling run -m mypackage.module
+
+When profiling a script, the profiler starts the target in a subprocess, waits
+for it to initialize, then begins collecting samples. The ``-m`` flag
+indicates that the target should be run as a module (equivalent to
+``python -m``). Arguments after the target are passed through to the
+profiled program::
+
+ python -m profiling.sampling run script.py --config settings.yaml
+
+
+The ``attach`` command
+----------------------
+
+The ``attach`` command connects to an already-running Python process by its
+process ID::
+
+ python -m profiling.sampling attach 12345
+
+This command is particularly valuable for investigating performance issues in
+production systems. The target process requires no modification and need not
+be restarted. The profiler attaches, collects samples for the specified
+duration, then detaches and produces output.
+
+On most systems, attaching to another process requires appropriate permissions.
+See :ref:`profiling-permissions` for platform-specific requirements.
+
+
+Profiling in production
+-----------------------
+
+The sampling profiler is designed for production use. It imposes no measurable
+overhead on the target process because it reads memory externally rather than
+instrumenting code. The target application continues running at full speed and
+is unaware it is being profiled.
+
+When profiling production systems, keep these guidelines in mind:
+
+Start with shorter durations (10-30 seconds) to get quick results, then extend
+if you need more statistical accuracy. The default 10-second duration is usually
+sufficient to identify major hotspots.
+
+If possible, profile during representative load rather than peak traffic.
+Profiles collected during normal operation are easier to interpret than those
+collected during unusual spikes.
+
+The profiler itself consumes some CPU on the machine where it runs (not on the
+target process). On the same machine, this is typically negligible. When
+profiling remote processes, network latency does not affect the target.
+
+Results from production may differ from development due to different data
+sizes, concurrent load, or caching effects. This is expected and is often
+exactly what you want to capture.
+
+
+.. _profiling-permissions:
+
+Platform requirements
+---------------------
+
+The profiler reads the target process's memory to capture stack traces. This
+requires elevated permissions on most operating systems.
+
+**Linux**
+
+On Linux, the profiler uses ``ptrace`` or ``process_vm_readv`` to read the
+target process's memory. This typically requires one of:
+
+- Running as root
+- Having the ``CAP_SYS_PTRACE`` capability
+- Adjusting the Yama ptrace scope: ``/proc/sys/kernel/yama/ptrace_scope``
+
+The default ptrace_scope of 1 restricts ptrace to parent processes only. To
+allow attaching to any process owned by the same user, set it to 0::
+
+ echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
+
+**macOS**
+
+On macOS, the profiler uses ``task_for_pid()`` to access the target process.
+This requires one of:
+
+- Running as root
+- The profiler binary having the ``com.apple.security.cs.debugger`` entitlement
+- System Integrity Protection (SIP) being disabled (not recommended)
+
+**Windows**
+
+On Windows, the profiler requires administrative privileges or the
+``SeDebugPrivilege`` privilege to read another process's memory.
+
+
+Version compatibility
+---------------------
+
+The profiler and target process must run the same Python minor version (for
+example, both Python 3.15). Attaching from Python 3.14 to a Python 3.15 process
+is not supported.
+
+Additional restrictions apply to pre-release Python versions: if either the
+profiler or target is running a pre-release (alpha, beta, or release candidate),
+both must run the exact same version.
+
+On free-threaded Python builds, the profiler cannot attach from a free-threaded
+build to a standard build, or vice versa.
+
+
+Sampling configuration
+======================
+
+Before exploring the various output formats and visualization options, it is
+important to understand how to configure the sampling process itself. The
+profiler offers several options that control how frequently samples are
+collected, how long profiling runs, which threads are observed, and what
+additional context is captured in each sample.
+
+The default configuration works well for most use cases:
+
+.. list-table::
+ :header-rows: 1
+ :widths: 25 75
+
+ * - Option
+ - Default behavior
+ * - ``--interval`` / ``-i``
+ - 100 µs between samples (~10,000 samples/sec)
+ * - ``--duration`` / ``-d``
+ - Profile for 10 seconds
+ * - ``--all-threads`` / ``-a``
+ - Sample main thread only
+ * - ``--native``
+ - No ``<native>`` frames (C code time attributed to caller)
+ * - ``--no-gc``
+ - Include ``<GC>`` frames when garbage collection is active
+ * - ``--mode``
+ - Wall-clock mode (all samples recorded)
+ * - ``--realtime-stats``
+ - No live statistics display during profiling
+
+
+Sampling interval and duration
+------------------------------
+
+The two most fundamental parameters are the sampling interval and duration.
+Together, these determine how many samples will be collected during a profiling
+session.
+
+The ``--interval`` option (``-i``) sets the time between samples in
+microseconds. The default is 100 microseconds, which produces approximately
+10,000 samples per second::
+
+ python -m profiling.sampling run -i 50 script.py
+
+Lower intervals capture more samples and provide finer-grained data at the
+cost of slightly higher profiler CPU usage. Higher intervals reduce profiler
+overhead but may miss short-lived functions. For most applications, the
+default interval provides a good balance between accuracy and overhead.
+
+The ``--duration`` option (``-d``) sets how long to profile in seconds. The
+default is 10 seconds::
+
+ python -m profiling.sampling run -d 60 script.py
+
+Longer durations collect more samples and produce more statistically reliable
+results, especially for code paths that execute infrequently. When profiling
+a program that runs for a fixed time, you may want to set the duration to
+match or exceed the expected runtime.
+
+
+Thread selection
+----------------
+
+Python programs often use multiple threads, whether explicitly through the
+:mod:`threading` module or implicitly through libraries that manage thread
+pools.
+
+By default, the profiler samples only the main thread. The ``--all-threads``
+option (``-a``) enables sampling of all threads in the process::
+
+ python -m profiling.sampling run -a script.py
+
+Multi-thread profiling reveals how work is distributed across threads and can
+identify threads that are blocked or starved. Each thread's samples are
+combined in the output, with the ability to filter by thread in some formats.
+This option is particularly useful when investigating concurrency issues or
+when work is distributed across a thread pool.
+
+
+Special frames
+--------------
+
+The profiler can inject artificial frames into the captured stacks to provide
+additional context about what the interpreter is doing at the moment each
+sample is taken. These synthetic frames help distinguish different types of
+execution that would otherwise be invisible.
+
+The ``--native`` option adds ``<native>`` frames to indicate when Python has
+called into C code (extension modules, built-in functions, or the interpreter
+itself)::
+
+ python -m profiling.sampling run --native script.py
+
+These frames help distinguish time spent in Python code versus time spent in
+native libraries. Without this option, native code execution appears as time
+in the Python function that made the call. This is useful when optimizing
+code that makes heavy use of C extensions like NumPy or database drivers.
+
+By default, the profiler includes ``<GC>`` frames when garbage collection is
+active. The ``--no-gc`` option suppresses these frames::
+
+ python -m profiling.sampling run --no-gc script.py
+
+GC frames help identify programs where garbage collection consumes significant
+time, which may indicate memory allocation patterns worth optimizing. If you
+see substantial time in ``<GC>`` frames, consider investigating object
+allocation rates or using object pooling.
+
+
+Real-time statistics
+--------------------
+
+The ``--realtime-stats`` option displays sampling rate statistics during
+profiling::
+
+ python -m profiling.sampling run --realtime-stats script.py
+
+This shows the actual achieved sampling rate, which may be lower than requested
+if the profiler cannot keep up. The statistics help verify that profiling is
+working correctly and that sufficient samples are being collected. See
+:ref:`sampling-efficiency` for details on interpreting these metrics.
+
+
+.. _sampling-efficiency:
+
+Sampling efficiency
+-------------------
+
+Sampling efficiency metrics help assess the quality of the collected data.
+These metrics appear in the profiler's terminal output and in the flame graph
+sidebar.
+
+**Sampling efficiency** is the percentage of sample attempts that succeeded.
+Each sample attempt reads the target process's call stack from memory. An
+attempt can fail if the process is in an inconsistent state at the moment of
+reading, such as during a context switch or while the interpreter is updating
+its internal structures. A low efficiency may indicate that the profiler could
+not keep up with the requested sampling rate, often due to system load or an
+overly aggressive interval setting.
+
+**Missed samples** is the percentage of expected samples that were not
+collected. Based on the configured interval and duration, the profiler expects
+to collect a certain number of samples. Some samples may be missed if the
+profiler falls behind schedule, for example when the system is under heavy
+load. A small percentage of missed samples is normal and does not significantly
+affect the statistical accuracy of the profile.
+
+Both metrics are informational. Even with some failed attempts or missed
+samples, the profile remains statistically valid as long as enough samples
+were collected. The profiler reports the actual number of samples captured,
+which you can use to judge whether the data is sufficient for your analysis.
+
+
+Profiling modes
+===============
+
+The sampling profiler supports three modes that control which samples are
+recorded. The mode determines what the profile measures: total elapsed time,
+CPU execution time, or time spent holding the global interpreter lock.
+
+
+Wall-clock mode
+---------------
+
+Wall-clock mode (``--mode=wall``) captures all samples regardless of what the
+thread is doing. This is the default mode and provides a complete picture of
+where time passes during program execution::
+
+ python -m profiling.sampling run --mode=wall script.py
+
+In wall-clock mode, samples are recorded whether the thread is actively
+executing Python code, waiting for I/O, blocked on a lock, or sleeping.
+This makes wall-clock profiling ideal for understanding the overall time
+distribution in your program, including time spent waiting.
+
+If your program spends significant time in I/O operations, network calls, or
+sleep, wall-clock mode will show these waits as time attributed to the calling
+function. This is often exactly what you want when optimizing end-to-end
+latency.
+
+
+CPU mode
+--------
+
+CPU mode (``--mode=cpu``) records samples only when the thread is actually
+executing on a CPU core::
+
+ python -m profiling.sampling run --mode=cpu script.py
+
+Samples taken while the thread is sleeping, blocked on I/O, or waiting for
+a lock are discarded. The resulting profile shows where CPU cycles are consumed,
+filtering out idle time.
+
+CPU mode is useful when you want to focus on computational hotspots without
+being distracted by I/O waits. If your program alternates between computation
+and network calls, CPU mode reveals which computational sections are most
+expensive.
+
+
+Comparing wall-clock and CPU profiles
+-------------------------------------
+
+Running both wall-clock and CPU mode profiles can reveal whether a function's
+time is spent computing or waiting.
+
+If a function appears prominently in both profiles, it is a true computational
+hotspot---actively using the CPU. Optimization should focus on algorithmic
+improvements or more efficient code.
+
+If a function is high in wall-clock mode but low or absent in CPU mode, it is
+I/O-bound or waiting. The function spends most of its time waiting for network,
+disk, locks, or sleep. CPU optimization won't help here; consider async I/O,
+connection pooling, or reducing wait time instead.
+
+
+GIL mode
+--------
+
+GIL mode (``--mode=gil``) records samples only when the thread holds Python's
+global interpreter lock::
+
+ python -m profiling.sampling run --mode=gil script.py
+
+The GIL is held only while executing Python bytecode. When Python calls into
+C extensions, performs I/O operations, or executes native code, the GIL is
+typically released. This means GIL mode effectively measures time spent
+running Python code specifically, filtering out time in native libraries.
+
+In multi-threaded programs, GIL mode reveals which code is preventing other
+threads from running Python bytecode. Since only one thread can hold the GIL
+at a time, functions that appear frequently in GIL mode profiles are
+monopolizing the interpreter.
+
+GIL mode helps answer questions like "which functions are monopolizing the
+GIL?" and "why are my other threads starving?" It can also be useful in
+single-threaded programs to distinguish Python execution time from time spent
+in C extensions or I/O.
+
+
+Output formats
+==============
+
+The profiler produces output in several formats, each suited to different
+analysis workflows. The format is selected with a command-line flag, and
+output goes to stdout, a file, or a directory depending on the format.
+
+
+pstats format
+-------------
+
+The pstats format (``--pstats``) produces a text table similar to what
+deterministic profilers generate. This is the default output format::
+
+ python -m profiling.sampling run script.py
+ python -m profiling.sampling run --pstats script.py
+
+Output appears on stdout by default::
+
+ Profile Stats (Mode: wall):
+ nsamples sample% tottime (ms) cumul% cumtime (ms) filename:lineno(function)
+ 234/892 11.7% 234.00 44.6% 892.00 server.py:145(handle_request)
+ 156/156 7.8% 156.00 7.8% 156.00 <built-in>:0(socket.recv)
+ 98/421 4.9% 98.00 21.1% 421.00 parser.py:67(parse_message)
+
+The columns show sampling counts and estimated times:
+
+- **nsamples**: Displayed as ``direct/cumulative`` (for example, ``10/50``).
+ Direct samples are when the function was at the top of the stack, actively
+ executing. Cumulative samples are when the function appeared anywhere on the
+ stack, including when it was waiting for functions it called. If a function
+ shows ``10/50``, it was directly executing in 10 samples and was on the call
+ stack in 50 samples total.
+
+- **sample%** and **cumul%**: Percentages of total samples for direct and
+ cumulative counts respectively.
+
+- **tottime** and **cumtime**: Estimated wall-clock time based on sample counts
+ and the profiling duration. Time units are selected automatically based on
+ the magnitude: seconds for large values, milliseconds for moderate values,
+ or microseconds for small values.
+
+The output includes a legend explaining each column and a summary of
+interesting functions that highlights:
+
+- **Hot spots**: Functions with high direct/cumulative sample ratio (ratio
+ close to 1.0). These functions spend most of their time executing their own
+ code rather than waiting for callees. High ratios indicate where CPU time
+ is actually consumed.
+
+- **Indirect calls**: Functions with large differences between cumulative and
+ direct samples. These are orchestration functions that delegate work to
+ other functions. They appear frequently on the stack but rarely at the top.
+
+- **Call magnification**: Functions where cumulative samples far exceed direct
+ samples (high cumulative/direct multiplier). These are frequently-nested
+ functions that appear deep in many call chains.
+
+Use ``--no-summary`` to suppress both the legend and summary sections.
+
+To save pstats output to a file instead of stdout::
+
+ python -m profiling.sampling run -o profile.txt script.py
+
+The pstats format supports several options for controlling the display.
+The ``--sort`` option determines the column used for ordering results::
+
+ python -m profiling.sampling run --sort=tottime script.py
+ python -m profiling.sampling run --sort=cumtime script.py
+ python -m profiling.sampling run --sort=nsamples script.py
+
+The ``--limit`` option restricts output to the top N entries::
+
+ python -m profiling.sampling run --limit=30 script.py
+
+The ``--no-summary`` option suppresses the header summary that precedes the
+statistics table.
+
+
+Collapsed stacks format
+-----------------------
+
+Collapsed stacks format (``--collapsed``) produces one line per unique call
+stack, with a count of how many times that stack was sampled::
+
+ python -m profiling.sampling run --collapsed script.py
+
+The output looks like:
+
+.. code-block:: text
+
+ main;process_data;parse_json;decode_utf8 42
+ main;process_data;parse_json 156
+ main;handle_request;send_response 89
+
+Each line contains semicolon-separated function names representing the call
+stack from bottom to top, followed by a space and the sample count. This
+format is designed for compatibility with external flame graph tools,
+particularly Brendan Gregg's ``flamegraph.pl`` script.
+
+To generate a flame graph from collapsed stacks::
+
+ python -m profiling.sampling run --collapsed script.py > stacks.txt
+ flamegraph.pl stacks.txt > profile.svg
+
+The resulting SVG can be viewed in any web browser and provides an interactive
+visualization where you can click to zoom into specific call paths.
+
+
+Flame graph format
+------------------
+
+Flame graph format (``--flamegraph``) produces a self-contained HTML file with
+an interactive flame graph visualization::
+
+ python -m profiling.sampling run --flamegraph script.py
+ python -m profiling.sampling run --flamegraph -o profile.html script.py
+
+If no output file is specified, the profiler generates a filename based on
+the process ID (for example, ``flamegraph.12345.html``).
+
+The generated HTML file requires no external dependencies and can be opened
+directly in a web browser. The visualization displays call stacks as nested
+rectangles, with width proportional to time spent. Hovering over a rectangle
+shows details about that function including source code context, and clicking
+zooms into that portion of the call tree.
+
+The flame graph interface includes:
+
+- A sidebar showing profile summary, thread statistics, sampling efficiency
+ metrics (see :ref:`sampling-efficiency`), and top hotspot functions
+- Search functionality supporting both function name matching and
+ ``file.py:42`` line patterns
+- Per-thread filtering via dropdown
+- Dark/light theme toggle (preference saved across sessions)
+- SVG export for saving the current view
+
+The thread statistics section shows runtime behavior metrics:
+
+- **GIL Held**: percentage of samples where a thread held the global interpreter
+ lock (actively running Python code)
+- **GIL Released**: percentage of samples where no thread held the GIL
+- **Waiting GIL**: percentage of samples where a thread was waiting to acquire
+ the GIL
+- **GC**: percentage of samples during garbage collection
+
+These statistics help identify GIL contention and understand how time is
+distributed between Python execution, native code, and waiting.
+
+Flame graphs are particularly effective for identifying deep call stacks and
+understanding the hierarchical structure of time consumption. Wide rectangles
+at the top indicate functions that consume significant time either directly
+or through their callees.
+
+
+Gecko format
+------------
+
+Gecko format (``--gecko``) produces JSON output compatible with the Firefox
+Profiler::
+
+ python -m profiling.sampling run --gecko script.py
+ python -m profiling.sampling run --gecko -o profile.json script.py
+
+The `Firefox Profiler <https://profiler.firefox.com>`__ is a sophisticated
+web-based tool originally built for profiling Firefox itself. It provides
+features beyond basic flame graphs, including a timeline view, call tree
+exploration, and marker visualization. See the
+`Firefox Profiler documentation <https://profiler.firefox.com/docs/#/>`__ for
+detailed usage instructions.
+
+To use the output, open the Firefox Profiler in your browser and load the
+JSON file. The profiler runs entirely client-side, so your profiling data
+never leaves your machine.
+
+Gecko format automatically collects additional metadata about GIL state and
+CPU activity, enabling analysis features specific to Python's threading model.
+The profiler emits interval markers that appear as colored bands in the
+Firefox Profiler timeline:
+
+- **GIL markers**: show when threads hold or release the global interpreter lock
+- **CPU markers**: show when threads are executing on CPU versus idle
+- **Code type markers**: distinguish Python code from native (C extension) code
+- **GC markers**: indicate garbage collection activity
+
+For this reason, the ``--mode`` option is not available with Gecko format;
+all relevant data is captured automatically.
+
+
+Heatmap format
+--------------
+
+Heatmap format (``--heatmap``) generates an interactive HTML visualization
+showing sample counts at the source line level::
+
+ python -m profiling.sampling run --heatmap script.py
+ python -m profiling.sampling run --heatmap -o my_heatmap script.py
+
+Unlike other formats that produce a single file, heatmap output creates a
+directory containing HTML files for each profiled source file. If no output
+path is specified, the directory is named ``heatmap_PID``.
+
+The heatmap visualization displays your source code with a color gradient
+indicating how many samples were collected at each line. Hot lines (many
+samples) appear in warm colors, while cold lines (few or no samples) appear
+in cool colors. This view helps pinpoint exactly which lines of code are
+responsible for time consumption.
+
+The heatmap interface provides several interactive features:
+
+- **Coloring modes**: toggle between "Self Time" (direct execution) and
+ "Total Time" (cumulative, including time in called functions)
+- **Cold code filtering**: show all lines or only lines with samples
+- **Call graph navigation**: each line shows navigation buttons (▲ for callers,
+ ▼ for callees) that let you trace execution paths through your code. When
+ multiple functions called or were called from a line, a menu appears showing
+ all options with their sample counts.
+- **Scroll minimap**: a vertical overview showing the heat distribution across
+ the entire file
+- **Hierarchical index**: files organized by type (stdlib, site-packages,
+ project) with aggregate sample counts per folder
+- **Dark/light theme**: toggle with preference saved across sessions
+- **Line linking**: click line numbers to create shareable URLs
+
+Heatmaps are especially useful when you know which file contains a performance
+issue but need to identify the specific lines. Many developers prefer this
+format because it maps directly to their source code, making it easy to read
+and navigate. For smaller scripts and focused analysis, heatmaps provide an
+intuitive view that shows exactly where time is spent without requiring
+interpretation of hierarchical visualizations.
+
+
+Live mode
+=========
+
+Live mode (``--live``) provides a terminal-based real-time view of profiling
+data, similar to the ``top`` command for system processes::
+
+ python -m profiling.sampling run --live script.py
+ python -m profiling.sampling attach --live 12345
+
+The display updates continuously as new samples arrive, showing the current
+hottest functions. This mode requires the :mod:`curses` module, which is
+available on Unix-like systems but not on Windows. The terminal must be at
+least 60 columns wide and 12 lines tall; larger terminals display more columns.
+
+The header displays the top 3 hottest functions, sampling efficiency metrics,
+and thread status statistics (GIL held percentage, CPU usage, GC time). The
+main table shows function statistics with the currently sorted column indicated
+by an arrow (▼).
+
+
+Keyboard commands
+-----------------
+
+Within live mode, keyboard commands control the display:
+
+:kbd:`q`
+ Quit the profiler and return to the shell.
+
+:kbd:`s` / :kbd:`S`
+ Cycle through sort orders forward/backward (sample count, percentage,
+ total time, cumulative percentage, cumulative time).
+
+:kbd:`p`
+ Pause or resume display updates. Sampling continues in the background
+ while the display is paused, so you can freeze the view to examine results
+ without stopping data collection.
+
+:kbd:`r`
+ Reset all statistics and start fresh. This is disabled after profiling
+ finishes to prevent accidental data loss.
+
+:kbd:`/`
+ Enter filter mode to search for functions by name. The filter uses
+ case-insensitive substring matching against the filename and function name.
+ Type a pattern and press Enter to apply, or Escape to cancel. Glob patterns
+ and regular expressions are not supported.
+
+:kbd:`c`
+ Clear the current filter and show all functions again.
+
+:kbd:`t`
+ Toggle between viewing all threads combined or per-thread statistics.
+ In per-thread mode, a thread counter (for example, ``1/4``) appears showing
+ your position among the available threads.
+
+:kbd:`←` :kbd:`→` or :kbd:`↑` :kbd:`↓`
+ In per-thread view, navigate between threads. Navigation wraps around
+ from the last thread to the first and vice versa.
+
+:kbd:`+` / :kbd:`-`
+ Increase or decrease the display refresh rate. The range is 0.05 seconds
+ (20 Hz, very responsive) to 1.0 second (1 Hz, lower overhead). Faster refresh
+ rates use more CPU. The default is 0.1 seconds (10 Hz).
+
+:kbd:`x`
+ Toggle trend indicators that show whether functions are becoming hotter
+ or cooler over time. When enabled, increasing metrics appear in green and
+ decreasing metrics appear in red, comparing each update to the previous one.
+
+:kbd:`h` or :kbd:`?`
+ Show the help screen with all available commands.
+
+When profiling finishes (duration expires or target process exits), the display
+shows a "PROFILING COMPLETE" banner and freezes the final results. You can
+still navigate, sort, and filter the results before pressing :kbd:`q` to exit.
+
+Live mode is incompatible with output format options (``--collapsed``,
+``--flamegraph``, and so on) because it uses an interactive terminal
+interface rather than producing file output.
+
+
+Async-aware profiling
+=====================
+
+For programs using :mod:`asyncio`, the profiler offers async-aware mode
+(``--async-aware``) that reconstructs call stacks based on the task structure
+rather than the raw Python frames::
+
+ python -m profiling.sampling run --async-aware async_script.py
+
+Standard profiling of async code can be confusing because the physical call
+stack often shows event loop internals rather than the logical flow of your
+coroutines. Async-aware mode addresses this by tracking which task is running
+and presenting stacks that reflect the ``await`` chain.
+
+.. note::
+
+ Async-aware profiling requires the target process to have the :mod:`asyncio`
+ module loaded. If you profile a script before it imports asyncio, async-aware
+ mode will not be able to capture task information.
+
+
+Async modes
+-----------
+
+The ``--async-mode`` option controls which tasks appear in the profile::
+
+ python -m profiling.sampling run --async-aware --async-mode=running async_script.py
+ python -m profiling.sampling run --async-aware --async-mode=all async_script.py
+
+With ``--async-mode=running`` (the default), only the task currently executing
+on the CPU is profiled. This shows where your program is actively spending time
+and is the typical choice for performance analysis.
+
+With ``--async-mode=all``, tasks that are suspended (awaiting I/O, locks, or
+other tasks) are also included. This mode is useful for understanding what your
+program is waiting on, but produces larger profiles since every suspended task
+appears in each sample.
+
+
+Task markers and stack reconstruction
+-------------------------------------
+
+In async-aware profiles, you will see ``<task>`` frames that mark boundaries
+between asyncio tasks. These are synthetic frames inserted by the profiler to
+show the task structure. The task name appears as the function name in these
+frames.
+
+When a task awaits another task, the profiler reconstructs the logical call
+chain by following the ``await`` relationships. Only "leaf" tasks (tasks that
+no other task is currently awaiting) generate their own stack entries. Tasks
+being awaited by other tasks appear as part of their awaiter's stack instead.
+
+If a task has multiple awaiters (a diamond pattern in the task graph), the
+profiler deterministically selects one parent and annotates the task marker
+with the number of parents, for example ``MyTask (2 parents)``. This indicates
+that alternate execution paths exist but are not shown in this particular stack.
+
+
+Option restrictions
+-------------------
+
+Async-aware mode uses a different stack reconstruction mechanism and is
+incompatible with: ``--native``, ``--no-gc``, ``--all-threads``, and
+``--mode=cpu`` or ``--mode=gil``.
+
+
+Command-line interface
+======================
+
+.. program:: profiling.sampling
+
+The complete command-line interface for reference.
+
+
+Global options
+--------------
+
+.. option:: run
+
+ Run and profile a Python script or module.
+
+.. option:: attach
+
+ Attach to and profile a running process by PID.
+
+
+Sampling options
+----------------
+
+.. option:: -i <microseconds>, --interval <microseconds>
+
+ Sampling interval in microseconds. Default: 100.
+
+.. option:: -d <seconds>, --duration <seconds>
+
+ Profiling duration in seconds. Default: 10.
+
+.. option:: -a, --all-threads
+
+ Sample all threads, not just the main thread.
+
+.. option:: --realtime-stats
+
+ Display sampling statistics during profiling.
+
+.. option:: --native
+
+ Include ``<native>`` frames for non-Python code.
+
+.. option:: --no-gc
+
+ Exclude ``<GC>`` frames for garbage collection.
+
+.. option:: --async-aware
+
+ Enable async-aware profiling for asyncio programs.
+
+
+Mode options
+------------
+
+.. option:: --mode <mode>
+
+ Sampling mode: ``wall`` (default), ``cpu``, or ``gil``.
+ The ``cpu`` and ``gil`` modes are incompatible with ``--async-aware``.
+
+.. option:: --async-mode <mode>
+
+ Async profiling mode: ``running`` (default) or ``all``.
+ Requires ``--async-aware``.
+
+
+Output options
+--------------
+
+.. option:: --pstats
+
+ Generate text statistics output. This is the default.
+
+.. option:: --collapsed
+
+ Generate collapsed stack format for external flame graph tools.
+
+.. option:: --flamegraph
+
+ Generate self-contained HTML flame graph.
+
+.. option:: --gecko
+
+ Generate Gecko JSON format for Firefox Profiler.
+
+.. option:: --heatmap
+
+ Generate HTML heatmap with line-level sample counts.
+
+.. option:: -o <path>, --output <path>
+
+ Output file or directory path. Default behavior varies by format:
+ ``--pstats`` writes to stdout, ``--flamegraph`` and ``--gecko`` generate
+ files like ``flamegraph.PID.html``, and ``--heatmap`` creates a directory
+ named ``heatmap_PID``.
+
+
+pstats display options
+----------------------
+
+These options apply only to pstats format output.
+
+.. option:: --sort <key>
+
+ Sort order: ``nsamples``, ``tottime``, ``cumtime``, ``sample-pct``,
+ ``cumul-pct``, ``nsamples-cumul``, or ``name``. Default: ``nsamples``.
+
+.. option:: -l <count>, --limit <count>
+
+ Maximum number of entries to display. Default: 15.
+
+.. option:: --no-summary
+
+ Omit the Legend and Summary of Interesting Functions sections from output.
+
+
+Run command options
+-------------------
+
+.. option:: -m, --module
+
+ Treat the target as a module name rather than a script path.
+
+.. option:: --live
+
+ Start interactive terminal interface instead of batch profiling.
+
+
+.. seealso::
+
+ :mod:`profiling`
+ Overview of Python profiling tools and guidance on choosing a profiler.
+
+ :mod:`profiling.tracing`
+ Deterministic tracing profiler for exact call counts and timing.
+
+ :mod:`pstats`
+ Statistics analysis for profile data.
+
+ `Firefox Profiler <https://profiler.firefox.com>`__
+ Web-based profiler that accepts Gecko format output. See the
+ `documentation <https://profiler.firefox.com/docs/#/>`__ for usage details.
+
+ `FlameGraph <https://github.com/brendangregg/FlameGraph>`__
+ Tools for generating flame graphs from collapsed stack format.
--- /dev/null
+.. _profiling-tracing:
+
+****************************************************
+:mod:`profiling.tracing` --- Deterministic profiler
+****************************************************
+
+.. module:: profiling.tracing
+ :synopsis: Deterministic tracing profiler for Python programs.
+
+.. module:: cProfile
+ :synopsis: Alias for profiling.tracing (backward compatibility).
+ :noindex:
+
+.. versionadded:: 3.15
+
+**Source code:** :source:`Lib/profiling/tracing/`
+
+--------------
+
+The :mod:`profiling.tracing` module provides deterministic profiling of Python
+programs. It monitors every function call, function return, and exception event,
+recording precise timing for each. This approach provides exact call counts and
+complete visibility into program execution, making it ideal for development and
+testing scenarios.
+
+.. note::
+
+ This module is also available as ``cProfile`` for backward compatibility.
+ The ``cProfile`` name will continue to work in all future Python versions.
+ Use whichever import style suits your codebase::
+
+ # Preferred (new style)
+ import profiling.tracing
+ profiling.tracing.run('my_function()')
+
+ # Also works (backward compatible)
+ import cProfile
+ cProfile.run('my_function()')
+
+
+What is deterministic profiling?
+================================
+
+:dfn:`Deterministic profiling` captures every function call, function return,
+and exception event during program execution. The profiler measures the precise
+time intervals between these events, providing exact statistics about how the
+program behaves.
+
+In contrast to :ref:`statistical profiling <profiling-sampling>`, which samples
+the call stack periodically to estimate where time is spent, deterministic
+profiling records every event. This means you get exact call counts rather than
+statistical approximations. The trade-off is that instrumenting every event
+introduces overhead that can slow down program execution.
+
+Python's interpreted nature makes deterministic profiling practical. The
+interpreter already dispatches events for function calls and returns, so the
+profiler can hook into this mechanism without requiring code modification. The
+overhead tends to be moderate relative to the inherent cost of interpretation,
+making deterministic profiling suitable for most development workflows.
+
+Deterministic profiling helps answer questions like:
+
+- How many times was this function called?
+- What is the complete call graph of my program?
+- Which functions are called by a particular function?
+- Are there unexpected function calls happening?
+
+Call count statistics can identify bugs (surprising counts) and inline
+expansion opportunities (high call counts). Internal time statistics reveal
+"hot loops" that warrant optimization. Cumulative time statistics help identify
+algorithmic inefficiencies. The handling of cumulative times in this profiler
+allows direct comparison of recursive and iterative implementations.
+
+
+.. _profiling-tracing-cli:
+
+Command-line interface
+======================
+
+.. program:: profiling.tracing
+
+The :mod:`profiling.tracing` module can be invoked as a script to profile
+another script or module:
+
+.. code-block:: shell-session
+
+ python -m profiling.tracing [-o output_file] [-s sort_order] (-m module | script.py)
+
+This runs the specified script or module under the profiler and prints the
+results to standard output (or saves them to a file).
+
+.. option:: -o <output_file>
+
+ Write the profile results to a file instead of standard output. The output
+ file can be read by the :mod:`pstats` module for later analysis.
+
+.. option:: -s <sort_order>
+
+ Sort the output by the specified key. This accepts any of the sort keys
+ recognized by :meth:`pstats.Stats.sort_stats`, such as ``cumulative``,
+ ``time``, ``calls``, or ``name``. This option only applies when
+ :option:`-o <profiling.tracing -o>` is not specified.
+
+.. option:: -m <module>
+
+ Profile a module instead of a script. The module is located using the
+ standard import mechanism.
+
+ .. versionadded:: 3.7
+ The ``-m`` option for ``cProfile``.
+
+ .. versionadded:: 3.8
+ The ``-m`` option for :mod:`profile`.
+
+
+Programmatic usage examples
+===========================
+
+For more control over profiling, use the module's functions and classes
+directly.
+
+
+Basic profiling
+---------------
+
+The simplest approach uses the :func:`!run` function::
+
+ import profiling.tracing
+ profiling.tracing.run('my_function()')
+
+This profiles the given code string and prints a summary to standard output.
+To save results for later analysis::
+
+ profiling.tracing.run('my_function()', 'output.prof')
+
+
+Using the :class:`!Profile` class
+---------------------------------
+
+The :class:`!Profile` class provides fine-grained control::
+
+ import profiling.tracing
+ import pstats
+ from io import StringIO
+
+ pr = profiling.tracing.Profile()
+ pr.enable()
+ # ... code to profile ...
+ pr.disable()
+
+ # Print results
+ s = StringIO()
+ ps = pstats.Stats(pr, stream=s).sort_stats(pstats.SortKey.CUMULATIVE)
+ ps.print_stats()
+ print(s.getvalue())
+
+The :class:`!Profile` class also works as a context manager::
+
+ import profiling.tracing
+
+ with profiling.tracing.Profile() as pr:
+ # ... code to profile ...
+
+ pr.print_stats()
+
+
+Module reference
+================
+
+.. currentmodule:: profiling.tracing
+
+.. function:: run(command, filename=None, sort=-1)
+
+ Profile execution of a command and print or save the results.
+
+ This function executes the *command* string using :func:`exec` in the
+ ``__main__`` module's namespace::
+
+ exec(command, __main__.__dict__, __main__.__dict__)
+
+ If *filename* is not provided, the function creates a :class:`pstats.Stats`
+ instance and prints a summary to standard output. If *filename* is
+ provided, the raw profile data is saved to that file for later analysis
+ with :mod:`pstats`.
+
+ The *sort* argument specifies the sort order for printed output, accepting
+ any value recognized by :meth:`pstats.Stats.sort_stats`.
+
+
+.. function:: runctx(command, globals, locals, filename=None, sort=-1)
+
+ Profile execution of a command with explicit namespaces.
+
+ Like :func:`run`, but executes the command with the specified *globals*
+ and *locals* mappings instead of using the ``__main__`` module's namespace::
+
+ exec(command, globals, locals)
+
+
+.. class:: Profile(timer=None, timeunit=0.0, subcalls=True, builtins=True)
+
+ A profiler object that collects execution statistics.
+
+ The optional *timer* argument specifies a custom timing function. If not
+ provided, the profiler uses a platform-appropriate default timer. When
+ supplying a custom timer, it must return a single number representing the
+ current time. If the timer returns integers, use *timeunit* to specify the
+ duration of one time unit (for example, ``0.001`` for milliseconds).
+
+ The *subcalls* argument controls whether the profiler tracks call
+ relationships between functions. The *builtins* argument controls whether
+ built-in functions are profiled.
+
+ .. versionchanged:: 3.8
+ Added context manager support.
+
+ .. method:: enable()
+
+ Start collecting profiling data.
+
+ .. method:: disable()
+
+ Stop collecting profiling data.
+
+ .. method:: create_stats()
+
+ Stop collecting data and record the results internally as the current
+ profile.
+
+ .. method:: print_stats(sort=-1)
+
+ Create a :class:`pstats.Stats` object from the current profile and print
+ the results to standard output.
+
+ The *sort* argument specifies the sorting order. It accepts a single
+ key or a tuple of keys for multi-level sorting, using the same values
+ as :meth:`pstats.Stats.sort_stats`.
+
+ .. versionadded:: 3.13
+ Support for a tuple of sort keys.
+
+ .. method:: dump_stats(filename)
+
+ Write the current profile data to *filename*. The file can be read by
+ :class:`pstats.Stats` for later analysis.
+
+ .. method:: run(cmd)
+
+ Profile the command string via :func:`exec`.
+
+ .. method:: runctx(cmd, globals, locals)
+
+ Profile the command string via :func:`exec` with the specified
+ namespaces.
+
+ .. method:: runcall(func, /, *args, **kwargs)
+
+ Profile a function call. Returns whatever *func* returns::
+
+ result = pr.runcall(my_function, arg1, arg2, keyword=value)
+
+.. note::
+
+ Profiling requires that the profiled code returns normally. If the
+ interpreter terminates (for example, via :func:`sys.exit`) during
+ profiling, no results will be available.
+
+
+Using a custom timer
+====================
+
+The :class:`Profile` class accepts a custom timing function, allowing you to
+measure different aspects of execution such as wall-clock time or CPU time.
+Pass the timing function to the constructor::
+
+ pr = profiling.tracing.Profile(my_timer_function)
+
+The timer function must return a single number representing the current time.
+If it returns integers, also specify *timeunit* to indicate the duration of
+one unit::
+
+ # Timer returns time in milliseconds
+ pr = profiling.tracing.Profile(my_ms_timer, 0.001)
+
+For best performance, the timer function should be as fast as possible. The
+profiler calls it frequently, so timer overhead directly affects profiling
+overhead.
+
+The :mod:`time` module provides several functions suitable for use as custom
+timers:
+
+- :func:`time.perf_counter` for high-resolution wall-clock time
+- :func:`time.process_time` for CPU time (excluding sleep)
+- :func:`time.monotonic` for monotonic clock time
+
+
+Limitations
+===========
+
+Deterministic profiling has inherent limitations related to timing accuracy.
+
+The underlying timer typically has a resolution of about one millisecond.
+Measurements cannot be more accurate than this resolution. With enough
+measurements, timing errors tend to average out, but individual measurements
+may be imprecise.
+
+There is also latency between when an event occurs and when the profiler
+captures the timestamp. Similarly, there is latency after reading the
+timestamp before user code resumes. Functions called frequently accumulate
+this latency, which can make them appear slower than they actually are. This
+error is typically less than one clock tick per call but can become
+significant for functions called many times.
+
+The :mod:`profiling.tracing` module (and its ``cProfile`` alias) is
+implemented as a C extension with low overhead, so these timing issues are
+less pronounced than with the deprecated pure Python :mod:`profile` module.
+
+
+.. seealso::
+
+ :mod:`profiling`
+ Overview of Python profiling tools and guidance on choosing a profiler.
+
+ :mod:`profiling.sampling`
+ Statistical sampling profiler for production use.
+
+ :mod:`pstats`
+ Statistics analysis and formatting for profile data.
+
+ :mod:`profile`
+ Deprecated pure Python profiler (includes calibration documentation).
--- /dev/null
+.. _pstats-module:
+
+********************************************
+:mod:`pstats` --- Statistics for profilers
+********************************************
+
+.. module:: pstats
+ :synopsis: Statistics object for analyzing profiler output.
+
+**Source code:** :source:`Lib/pstats.py`
+
+--------------
+
+The :mod:`pstats` module provides tools for reading, manipulating, and
+displaying profiling statistics generated by Python's profilers. It reads
+output from both :mod:`profiling.tracing` (deterministic profiler) and
+:mod:`profiling.sampling` (statistical profiler).
+
+
+Reading and displaying profile data
+===================================
+
+The :class:`Stats` class is the primary interface for working with profile
+data. It can read statistics from files or directly from a
+:class:`~profiling.tracing.Profile` object.
+
+Load statistics from a file and print a basic report::
+
+ import pstats
+
+ p = pstats.Stats('profile_output.prof')
+ p.print_stats()
+
+The :class:`Stats` object provides methods for sorting and filtering the
+data before printing. For example, to see the ten functions with the highest
+cumulative time::
+
+ from pstats import SortKey
+
+ p = pstats.Stats('profile_output.prof')
+ p.sort_stats(SortKey.CUMULATIVE).print_stats(10)
+
+
+Working with statistics
+-----------------------
+
+The :class:`Stats` class supports method chaining, making it convenient to
+perform multiple operations::
+
+ p = pstats.Stats('restats')
+ p.strip_dirs().sort_stats(-1).print_stats()
+
+The :meth:`~Stats.strip_dirs` method removes directory paths from filenames,
+making the output more compact. The :meth:`~Stats.sort_stats` method accepts
+various keys to control the sort order.
+
+Different sort keys highlight different aspects of performance::
+
+ from pstats import SortKey
+
+ # Functions that consume the most cumulative time
+ p.sort_stats(SortKey.CUMULATIVE).print_stats(10)
+
+ # Functions that consume the most time in their own code
+ p.sort_stats(SortKey.TIME).print_stats(10)
+
+ # Functions sorted by name
+ p.sort_stats(SortKey.NAME).print_stats()
+
+
+Filtering output
+----------------
+
+The :meth:`~Stats.print_stats` method accepts restrictions that filter
+which functions are displayed. Restrictions can be integers (limiting the
+count), floats between 0 and 1 (selecting a percentage), or strings (matching
+function names via regular expression).
+
+Print only the top 10%::
+
+ p.print_stats(.1)
+
+Print only functions whose names contain "init"::
+
+ p.print_stats('init')
+
+Combine restrictions (they apply sequentially)::
+
+ # Top 10%, then only those containing "init"
+ p.print_stats(.1, 'init')
+
+ # Functions in files matching "foo:", limited to top 50%
+ p.sort_stats(SortKey.FILENAME).print_stats('foo:', .5)
+
+
+Analyzing call relationships
+----------------------------
+
+The :meth:`~Stats.print_callers` method shows which functions called each
+displayed function::
+
+ p.print_callers()
+
+The :meth:`~Stats.print_callees` method shows the opposite relationship,
+listing which functions each displayed function called::
+
+ p.print_callees()
+
+Both methods accept the same restriction arguments as :meth:`~Stats.print_stats`.
+
+
+Combining multiple profiles
+---------------------------
+
+Statistics from multiple profiling runs can be combined into a single
+:class:`Stats` object::
+
+ # Load multiple files at once
+ p = pstats.Stats('run1.prof', 'run2.prof', 'run3.prof')
+
+ # Or add files incrementally
+ p = pstats.Stats('run1.prof')
+ p.add('run2.prof')
+ p.add('run3.prof')
+
+When files are combined, statistics for identical functions (same file, line,
+and name) are accumulated, giving an aggregate view across all profiling runs.
+
+
+The :class:`!Stats` class
+=========================
+
+.. class:: Stats(*filenames_or_profile, stream=sys.stdout)
+
+ Create a statistics object from profile data.
+
+ The arguments can be filenames (strings or path-like objects) or
+ :class:`~profiling.tracing.Profile` objects. If multiple sources are
+ provided, their statistics are combined.
+
+ The *stream* argument specifies where output from :meth:`print_stats` and
+ related methods is written. It defaults to :data:`sys.stdout`.
+
+ The profile data format is specific to the Python version that created it.
+ There is no compatibility guarantee between Python versions or between
+ different profilers.
+
+ .. method:: strip_dirs()
+
+ Remove leading path information from all filenames.
+
+ This method modifies the object in place and returns it for method
+ chaining. After stripping, the statistics are considered to be in
+ random order.
+
+ If stripping causes two functions to become indistinguishable (same
+ filename, line number, and function name), their statistics are
+ combined into a single entry.
+
+ .. method:: add(*filenames)
+
+ Add profiling data from additional files.
+
+ The files must have been created by the same profiler type. Statistics
+ for identical functions are accumulated.
+
+ .. method:: dump_stats(filename)
+
+ Save the current statistics to a file.
+
+ The file is created if it does not exist and overwritten if it does.
+ The saved data can be loaded by creating a new :class:`Stats` object.
+
+ .. method:: sort_stats(*keys)
+
+ Sort the statistics according to the specified criteria.
+
+ Each key can be a string or a :class:`SortKey` enum member. When
+ multiple keys are provided, later keys break ties in earlier keys.
+
+ Using :class:`SortKey` enum members is preferred over strings as it
+ provides better error checking::
+
+ from pstats import SortKey
+ p.sort_stats(SortKey.CUMULATIVE)
+
+ Valid sort keys:
+
+ +------------------+------------------------+----------------------+
+ | String | Enum | Meaning |
+ +==================+========================+======================+
+ | ``'calls'`` | ``SortKey.CALLS`` | call count |
+ +------------------+------------------------+----------------------+
+ | ``'cumulative'`` | ``SortKey.CUMULATIVE`` | cumulative time |
+ +------------------+------------------------+----------------------+
+ | ``'cumtime'`` | N/A | cumulative time |
+ +------------------+------------------------+----------------------+
+ | ``'file'`` | N/A | file name |
+ +------------------+------------------------+----------------------+
+ | ``'filename'`` | ``SortKey.FILENAME`` | file name |
+ +------------------+------------------------+----------------------+
+ | ``'module'`` | N/A | file name |
+ +------------------+------------------------+----------------------+
+ | ``'ncalls'`` | N/A | call count |
+ +------------------+------------------------+----------------------+
+ | ``'pcalls'`` | ``SortKey.PCALLS`` | primitive call count |
+ +------------------+------------------------+----------------------+
+ | ``'line'`` | ``SortKey.LINE`` | line number |
+ +------------------+------------------------+----------------------+
+ | ``'name'`` | ``SortKey.NAME`` | function name |
+ +------------------+------------------------+----------------------+
+ | ``'nfl'`` | ``SortKey.NFL`` | name/file/line |
+ +------------------+------------------------+----------------------+
+ | ``'stdname'`` | ``SortKey.STDNAME`` | standard name |
+ +------------------+------------------------+----------------------+
+ | ``'time'`` | ``SortKey.TIME`` | internal time |
+ +------------------+------------------------+----------------------+
+ | ``'tottime'`` | N/A | internal time |
+ +------------------+------------------------+----------------------+
+
+ All sorts on statistics are in descending order (most time consuming
+ first), while name, file, and line number sorts are ascending
+ (alphabetical).
+
+ The difference between ``SortKey.NFL`` and ``SortKey.STDNAME`` is that
+ NFL sorts line numbers numerically while STDNAME sorts them as strings.
+ ``sort_stats(SortKey.NFL)`` is equivalent to
+ ``sort_stats(SortKey.NAME, SortKey.FILENAME, SortKey.LINE)``.
+
+ For backward compatibility, the numeric arguments ``-1``, ``0``, ``1``,
+ and ``2`` are also accepted, meaning ``'stdname'``, ``'calls'``,
+ ``'time'``, and ``'cumulative'`` respectively.
+
+ .. versionadded:: 3.7
+ The :class:`SortKey` enum.
+
+ .. method:: reverse_order()
+
+ Reverse the current sort order.
+
+ By default, the sort direction is chosen appropriately for the sort key
+ (descending for time-based keys, ascending for name-based keys). This
+ method inverts that choice.
+
+ .. method:: print_stats(*restrictions)
+
+ Print a report of the profiling statistics.
+
+ The output includes a header line summarizing the data, followed by a
+ table of function statistics sorted according to the last
+ :meth:`sort_stats` call.
+
+ Restrictions filter the output. Each restriction is either:
+
+ - An integer: limits output to that many entries
+ - A float between 0.0 and 1.0: selects that fraction of entries
+ - A string: matches function names via regular expression
+
+ Restrictions are applied sequentially. For example::
+
+ print_stats(.1, 'foo:')
+
+ First limits to the top 10%, then filters to functions matching 'foo:'.
+
+ .. method:: print_callers(*restrictions)
+
+ Print the callers of each function in the statistics.
+
+ For each function in the filtered results, shows which functions called
+ it and how often.
+
+ With :mod:`profiling.tracing` (or ``cProfile``), each caller line
+ shows three numbers: the number of calls from that caller, and the
+ total and cumulative times for those specific calls.
+
+ Accepts the same restriction arguments as :meth:`print_stats`.
+
+ .. method:: print_callees(*restrictions)
+
+ Print the functions called by each function in the statistics.
+
+ This is the inverse of :meth:`print_callers`, showing which functions
+ each listed function called.
+
+ Accepts the same restriction arguments as :meth:`print_stats`.
+
+ .. method:: get_stats_profile()
+
+ Return a ``StatsProfile`` object containing the statistics.
+
+ The returned object provides programmatic access to the profile data,
+ with function names mapped to ``FunctionProfile`` objects
+ containing timing and call count information.
+
+ .. versionadded:: 3.9
+
+
+.. class:: SortKey
+
+ An enumeration of valid sort keys for :meth:`Stats.sort_stats`.
+
+ .. attribute:: CALLS
+
+ Sort by call count.
+
+ .. attribute:: CUMULATIVE
+
+ Sort by cumulative time.
+
+ .. attribute:: FILENAME
+
+ Sort by file name.
+
+ .. attribute:: LINE
+
+ Sort by line number.
+
+ .. attribute:: NAME
+
+ Sort by function name.
+
+ .. attribute:: NFL
+
+ Sort by name, then file, then line number (numeric line sort).
+
+ .. attribute:: PCALLS
+
+ Sort by primitive (non-recursive) call count.
+
+ .. attribute:: STDNAME
+
+ Sort by standard name (string-based line sort).
+
+ .. attribute:: TIME
+
+ Sort by internal time (time in function excluding subcalls).
+
+
+.. _pstats-cli:
+
+Command-line interface
+======================
+
+The :mod:`pstats` module can be invoked as a script to interactively browse
+profile data::
+
+ python -m pstats profile_output.prof
+
+This opens a line-oriented interface (built on :mod:`cmd`) for examining the
+statistics. Type ``help`` at the prompt for available commands.
+
+
+.. seealso::
+
+ :mod:`profiling`
+ Overview of Python profiling tools.
+
+ :mod:`profiling.tracing`
+ Deterministic tracing profiler.
+
+ :mod:`profiling.sampling`
+ Statistical sampling profiler.
.. _superseded:
******************
-Superseded Modules
+Superseded modules
******************
The modules described in this chapter have been superseded by other modules
:maxdepth: 1
getopt.rst
+ profile.rst
Doc/library/optparse.rst
Doc/library/os.rst
Doc/library/pickletools.rst
-Doc/library/profile.rst
Doc/library/pyexpat.rst
Doc/library/select.rst
Doc/library/socket.rst
with the new ':keyword:`with`' statement. See section :ref:`contextlibmod`
for more about this module.
-* New module: The :mod:`cProfile` module is a C implementation of the existing
- :mod:`profile` module that has much lower overhead. The module's interface is
- the same as :mod:`profile`: you run ``cProfile.run('main()')`` to profile a
+* New module: The :mod:`!cProfile` module is a C implementation of the existing
+ :mod:`!profile` module that has much lower overhead. The module's interface is
+ the same as :mod:`!profile`: you run ``cProfile.run('main()')`` to profile a
function, can save profile data to a file, etc. It's not yet known if the
Hotshot profiler, which is also written in C but doesn't match the
- :mod:`profile` module's interface, will continue to be maintained in future
+ :mod:`!profile` module's interface, will continue to be maintained in future
versions of Python. (Contributed by Armin Rigo.)
Also, the :mod:`pstats` module for analyzing the data measured by the profiler
* :pep:`799`: :ref:`A dedicated profiling package for organizing Python
profiling tools <whatsnew315-profiling-package>`
+* :pep:`799`: :ref:`Tachyon: High frequency statistical sampling profiler
+ profiling tools <whatsnew315-sampling-profiler>`
* :pep:`686`: :ref:`Python now uses UTF-8 as the default encoding
<whatsnew315-utf8-default>`
* :pep:`782`: :ref:`A new PyBytesWriter C API to create a Python bytes object
:pep:`799`: A dedicated profiling package
-----------------------------------------
-A new :mod:`!profiling` module has been added to organize Python's built-in
+A new :mod:`profiling` module has been added to organize Python's built-in
profiling tools under a single, coherent namespace. This module contains:
-* :mod:`!profiling.tracing`: deterministic function-call tracing (relocated from
- :mod:`cProfile`).
-* :mod:`!profiling.sampling`: a new statistical sampling profiler (named Tachyon).
+* :mod:`profiling.tracing`: deterministic function-call tracing (relocated from
+ ``cProfile``).
+* :mod:`profiling.sampling`: a new statistical sampling profiler (named Tachyon).
-The :mod:`cProfile` module remains as an alias for backwards compatibility.
+The ``cProfile`` module remains as an alias for backwards compatibility.
The :mod:`profile` module is deprecated and will be removed in Python 3.17.
.. seealso:: :pep:`799` for further details.
Tachyon: High frequency statistical sampling profiler
-----------------------------------------------------
+.. image:: ../library/tachyon-logo.png
+ :alt: Tachyon profiler logo
+ :align: center
+ :width: 200px
+
A new statistical sampling profiler (Tachyon) has been added as
-:mod:`!profiling.sampling`. This profiler enables low-overhead performance analysis of
+:mod:`profiling.sampling`. This profiler enables low-overhead performance analysis of
running Python processes without requiring code modification or process restart.
-Unlike deterministic profilers (:mod:`cProfile` and :mod:`profile`) that instrument
+Unlike deterministic profilers (such as :mod:`profiling.tracing`) that instrument
every function call, the sampling profiler periodically captures stack traces from
running processes. This approach provides virtually zero overhead while achieving
sampling rates of **up to 1,000,000 Hz**, making it the fastest sampling profiler
(``--async-aware``). See which coroutines are consuming time, with options to show only
running tasks or all tasks including those waiting.
+See :mod:`profiling.sampling` for the complete documentation, including all
+available output formats, profiling modes, and configuration options.
+
(Contributed by Pablo Galindo and László Kiss Kollár in :gh:`135953` and :gh:`138122`.)
cProfile
--------
-The :mod:`cProfile` command line now accepts ``-m module_name`` as an
+The :mod:`!cProfile` command line now accepts ``-m module_name`` as an
alternative to script path. (Contributed by Sanyam Khurana in :issue:`21862`.)
}
.sidebar-logo-img {
- width: 90px;
- height: 90px;
+ width: 220px;
+ height: 180px;
display: flex;
align-items: center;
justify-content: center;
<!-- Top Bar -->
<header class="top-bar">
<div class="brand">
+ <div class="brand-logo"><!-- PYTHON_LOGO --></div>
<span class="brand-text">Tachyon</span>
<span class="brand-divider"></span>
<span class="brand-subtitle">Heatmap Report</span>
<!-- Top Bar (Code Header) -->
<header class="top-bar">
<div class="brand">
+ <div class="brand-logo"><!-- PYTHON_LOGO --></div>
<span class="brand-text">Tachyon</span>
<span class="brand-divider"></span>
<span class="brand-subtitle" style="font-family: var(--font-mono); font-size: 13px;"><!-- FILENAME --></span>
display: flex;
align-items: center;
justify-content: center;
- width: 28px;
- height: 28px;
+ width: 48px;
+ height: 40px;
flex-shrink: 0;
}
/* Style the inlined SVG/img inside brand-logo */
.brand-logo svg,
.brand-logo img {
- width: 28px;
- height: 28px;
+ width: 48px;
+ height: 40px;
display: block;
object-fit: contain;
filter: drop-shadow(0 1px 2px rgba(0, 0, 0, 0.2));
self.index_js = f"{shared_js}\n{(assets_dir / 'heatmap_index.js').read_text(encoding='utf-8')}"
self.file_js = f"{shared_js}\n{(assets_dir / 'heatmap.js').read_text(encoding='utf-8')}"
- # Load Python logo
+ # Load Tachyon logo
logo_dir = template_dir / "_assets"
try:
- png_path = logo_dir / "python-logo-only.png"
+ png_path = logo_dir / "tachyon-logo.png"
b64_logo = base64.b64encode(png_path.read_bytes()).decode("ascii")
- self.logo_html = f'<img src="data:image/png;base64,{b64_logo}" alt="Python logo" class="python-logo"/>'
+ self.logo_html = f'<img src="data:image/png;base64,{b64_logo}" alt="Tachyon logo" class="python-logo"/>'
except (FileNotFoundError, IOError) as e:
self.logo_html = '<div class="python-logo-placeholder"></div>'
- print(f"Warning: Could not load Python logo: {e}")
+ print(f"Warning: Could not load Tachyon logo: {e}")
except (FileNotFoundError, IOError) as e:
raise RuntimeError(f"Failed to load heatmap template files: {e}") from e
"<!-- CODE_LINES -->": ''.join(code_lines_html),
"<!-- INLINE_CSS -->": f"<style>\n{self._template_loader.file_css}\n</style>",
"<!-- INLINE_JS -->": f"<script>\n{self._template_loader.file_js}\n</script>",
+ "<!-- PYTHON_LOGO -->": self._template_loader.logo_html,
}
html_content = self._template_loader.file_template
"<!-- INLINE_JS -->", f"<script>\n{js_content}\n</script>"
)
- png_path = assets_dir / "python-logo-only.png"
+ png_path = assets_dir / "tachyon-logo.png"
b64_logo = base64.b64encode(png_path.read_bytes()).decode("ascii")
# Let CSS control size; keep markup simple
- logo_html = f'<img src="data:image/png;base64,{b64_logo}" alt="Python logo"/>'
+ logo_html = f'<img src="data:image/png;base64,{b64_logo}" alt="Tachyon logo"/>'
html_template = html_template.replace("<!-- INLINE_LOGO -->", logo_html)
d3_js = d3_path.read_text(encoding="utf-8")
.. nonce: ONk9Na
.. section: Library
-Fix ``--outfile`` for :mod:`cProfile` / :mod:`profile` not writing the
+Fix ``--outfile`` for :mod:`!cProfile` / :mod:`!profile` not writing the
output file in the original directory when the program being profiled
changes the working directory. PR by Anthony Sottile.
.. nonce: Jq6Az-
.. section: Library
-Fix CLI of :mod:`cProfile` and :mod:`profile` to catch
+Fix CLI of :mod:`!cProfile` and :mod:`!profile` to catch
:exc:`BrokenPipeError`.
..
.. nonce: n_AfcS
.. section: Library
-Update :mod:`cProfile` to use PEP 669 API
+Update :mod:`!cProfile` to use PEP 669 API
..
.. nonce: ya5jBT
.. section: Library
-Added PY_THROW event hook for :mod:`cProfile` for generators
+Added PY_THROW event hook for :mod:`!cProfile` for generators
..
.. nonce: s4HXR0
.. section: Library
-Fixed the use-after-free issue in :mod:`cProfile` by disallowing
+Fixed the use-after-free issue in :mod:`!cProfile` by disallowing
``disable()`` and ``clear()`` in external timers.
..
.. nonce: APBFCw
.. section: Library
-Fixed the :exc:`SystemError` in :mod:`cProfile` when locating the actual C
+Fixed the :exc:`SystemError` in :mod:`!cProfile` when locating the actual C
function of a method raises an exception.
..
.. nonce: yLZJpV
.. section: Core and Builtins
-Make :mod:`cProfile` thread-safe on the :term:`free threaded <free
+Make :mod:`!cProfile` thread-safe on the :term:`free threaded <free
threading>` build.
..
Remove the ``PyEval_GetCallStats()`` function and deprecate the untested and
undocumented ``sys.callstats()`` function. Remove the ``CALL_PROFILE``
-special build: use the :func:`sys.setprofile` function, :mod:`cProfile` or
-:mod:`profile` to profile function calls.
+special build: use the :func:`sys.setprofile` function, :mod:`!cProfile` or
+:mod:`!profile` to profile function calls.
..
projects / thirdparty / Python / cpython.git / commitdiff
