.. include:: /using/venv-create.inc
+.. _venv-explanation:
-.. _venv-def:
+How venvs work
+--------------
-.. note:: A virtual environment is a Python environment such that the Python
- interpreter, libraries and scripts installed into it are isolated from those
- installed in other virtual environments, and (by default) any libraries
- installed in a "system" Python, i.e., one which is installed as part of your
- operating system.
-
- A virtual environment is a directory tree which contains Python executable
- files and other files which indicate that it is a virtual environment.
-
- Common installation tools such as setuptools_ and pip_ work as
- expected with virtual environments. In other words, when a virtual
- environment is active, they install Python packages into the virtual
- environment without needing to be told to do so explicitly.
-
- When a virtual environment is active (i.e., the virtual environment's Python
- interpreter is running), the attributes :attr:`sys.prefix` and
- :attr:`sys.exec_prefix` point to the base directory of the virtual
- environment, whereas :attr:`sys.base_prefix` and
- :attr:`sys.base_exec_prefix` point to the non-virtual environment Python
- installation which was used to create the virtual environment. If a virtual
- environment is not active, then :attr:`sys.prefix` is the same as
- :attr:`sys.base_prefix` and :attr:`sys.exec_prefix` is the same as
- :attr:`sys.base_exec_prefix` (they all point to a non-virtual environment
- Python installation).
-
- When a virtual environment is active, any options that change the
- installation path will be ignored from all :mod:`distutils` configuration
- files to prevent projects being inadvertently installed outside of the
- virtual environment.
-
- When working in a command shell, users can make a virtual environment active
- by running an ``activate`` script in the virtual environment's executables
- directory (the precise filename and command to use the file is
- shell-dependent), which prepends the virtual environment's directory for
- executables to the ``PATH`` environment variable for the running shell. There
- should be no need in other circumstances to activate a virtual
- environment; scripts installed into virtual environments have a "shebang"
- line which points to the virtual environment's Python interpreter. This means
- that the script will run with that interpreter regardless of the value of
- ``PATH``. On Windows, "shebang" line processing is supported if you have the
- Python Launcher for Windows installed (this was added to Python in 3.3 - see
- :pep:`397` for more details). Thus, double-clicking an installed script in a
- Windows Explorer window should run the script with the correct interpreter
- without there needing to be any reference to its virtual environment in
- ``PATH``.
+When a Python interpreter is running from a virtual environment,
+:data:`sys.prefix` and :data:`sys.exec_prefix`
+point to the directories of the virtual environment,
+whereas :data:`sys.base_prefix` and :data:`sys.base_exec_prefix`
+point to those of the base Python used to create the environment.
+It is sufficient to check
+``sys.prefix == sys.base_prefix`` to determine if the current interpreter is
+running from a virtual environment.
+
+A virtual environment may be "activated" using a script in its binary directory
+(``bin`` on POSIX; ``Scripts`` on Windows).
+This will prepend that directory to your :envvar:`!PATH`, so that running
+:program:`!python` will invoke the environment's Python interpreter
+and you can run installed scripts without having to use their full path.
+The invocation of the activation script is platform-specific
+(:samp:`{<venv>}` must be replaced by the path to the directory
+containing the virtual environment):
+
++-------------+------------+--------------------------------------------------+
+| Platform | Shell | Command to activate virtual environment |
++=============+============+==================================================+
+| POSIX | bash/zsh | :samp:`$ source {<venv>}/bin/activate` |
+| +------------+--------------------------------------------------+
+| | fish | :samp:`$ source {<venv>}/bin/activate.fish` |
+| +------------+--------------------------------------------------+
+| | csh/tcsh | :samp:`$ source {<venv>}/bin/activate.csh` |
+| +------------+--------------------------------------------------+
+| | PowerShell | :samp:`$ {<venv>}/bin/Activate.ps1` |
++-------------+------------+--------------------------------------------------+
+| Windows | cmd.exe | :samp:`C:\\> {<venv>}\\Scripts\\activate.bat` |
+| +------------+--------------------------------------------------+
+| | PowerShell | :samp:`PS C:\\> {<venv>}\\Scripts\\Activate.ps1` |
++-------------+------------+--------------------------------------------------+
+
+.. versionadded:: 3.4
+ :program:`!fish` and :program:`!csh` activation scripts.
+
+.. versionadded:: 3.8
+ PowerShell activation scripts installed under POSIX for PowerShell Core
+ support.
+
+You don't specifically *need* to activate a virtual environment,
+as you can just specify the full path to that environment's
+Python interpreter when invoking Python.
+Furthermore, all scripts installed in the environment
+should be runnable without activating it.
+
+In order to achieve this, scripts installed into virtual environments have
+a "shebang" line which points to the environment's Python interpreter,
+i.e. :samp:`#!/{<path-to-venv>}/bin/python`.
+This means that the script will run with that interpreter regardless of the
+value of :envvar:`!PATH`. On Windows, "shebang" line processing is supported if
+you have the :ref:`launcher` installed. Thus, double-clicking an installed
+script in a Windows Explorer window should run it with the correct interpreter
+without the environment needing to be activated or on the :envvar:`!PATH`.
+
+When a virtual environment has been activated, the :envvar:`!VIRTUAL_ENV`
+environment variable is set to the path of the environment.
+Since explicitly activating a virtual environment is not required to use it,
+:envvar:`!VIRTUAL_ENV` cannot be relied upon to determine
+whether a virtual environment is being used.
+
+.. warning:: Because scripts installed in environments should not expect the
+ environment to be activated, their shebang lines contain the absolute paths
+ to their environment's interpreters. Because of this, environments are
+ inherently non-portable, in the general case. You should always have a
+ simple means of recreating an environment (for example, if you have a
+ requirements file ``requirements.txt``, you can invoke ``pip install -r
+ requirements.txt`` using the environment's ``pip`` to install all of the
+ packages needed by the environment). If for any reason you need to move the
+ environment to a new location, you should recreate it at the desired
+ location and delete the one at the old location. If you move an environment
+ because you moved a parent directory of it, you should recreate the
+ environment in its new location. Otherwise, software installed into the
+ environment may not work as expected.
+
+You can deactivate a virtual environment by typing ``deactivate`` in your shell.
+The exact mechanism is platform-specific and is an internal implementation
+detail (typically, a script or shell function will be used).
+ .. warning:: Because scripts installed in environments should not expect the
+ environment to be activated, their shebang lines contain the absolute paths
+ to their environment's interpreters. Because of this, environments are
+ inherently non-portable, in the general case. You should always have a
+ simple means of recreating an environment (for example, if you have a
+ requirements file ``requirements.txt``, you can invoke ``pip install -r
+ requirements.txt`` using the environment's ``pip`` to install all of the
+ packages needed by the environment). If for any reason you need to move the
+ environment to a new location, you should recreate it at the desired
+ location and delete the one at the old location. If you move an environment
+ because you moved a parent directory of it, you should recreate the
+ environment in its new location. Otherwise, software installed into the
+ environment may not work as expected.
.. _venv-api:
CPython bytecode changes
========================
-* The bytecode now contains inline cache entries, which take the form of
- :opcode:`CACHE` instructions. Many opcodes expect to be followed by an exact
- number of caches, and instruct the interpreter to skip over them at runtime.
- Populated caches can look like arbitrary instructions, so great care should be
- taken when reading or modifying raw, adaptive bytecode containing quickened
- data.
+The bytecode now contains inline cache entries,
+which take the form of the newly-added :opcode:`CACHE` instructions.
+Many opcodes expect to be followed by an exact number of caches,
+and instruct the interpreter to skip over them at runtime.
+Populated caches can look like arbitrary instructions,
+so great care should be taken when reading or modifying
+raw, adaptive bytecode containing quickened data.
-* Replaced all numeric ``BINARY_*`` and ``INPLACE_*`` instructions with a single
- :opcode:`BINARY_OP` implementation.
-* Replaced the three call instructions: :opcode:`CALL_FUNCTION`,
- :opcode:`CALL_FUNCTION_KW` and :opcode:`CALL_METHOD` with
- :opcode:`PUSH_NULL`, :opcode:`PRECALL`, :opcode:`CALL`,
- and :opcode:`KW_NAMES`.
- This decouples the argument shifting for methods from the handling of
- keyword arguments and allows better specialization of calls.
+.. _whatsnew311-added-opcodes:
-* Removed ``COPY_DICT_WITHOUT_KEYS`` and ``GEN_START``.
+New opcodes
+-----------
+
+* :opcode:`ASYNC_GEN_WRAP`, :opcode:`RETURN_GENERATOR` and :opcode:`SEND`,
+ used in generators and co-routines.
-* :opcode:`MATCH_CLASS` and :opcode:`MATCH_KEYS` no longer push an additional
- boolean value indicating whether the match succeeded or failed. Instead, they
- indicate failure with :const:`None` (where a tuple of extracted values would
- otherwise be).
+* :opcode:`COPY_FREE_VARS`,
+ which avoids needing special caller-side code for closures.
-* Replace several stack manipulation instructions (``DUP_TOP``, ``DUP_TOP_TWO``,
- ``ROT_TWO``, ``ROT_THREE``, ``ROT_FOUR``, and ``ROT_N``) with new
- :opcode:`COPY` and :opcode:`SWAP` instructions.
+* :opcode:`JUMP_BACKWARD_NO_INTERRUPT`,
+ for use in certain loops where handling interrupts is undesirable.
-* Replaced :opcode:`JUMP_IF_NOT_EXC_MATCH` by :opcode:`CHECK_EXC_MATCH` which
- performs the check but does not jump.
+* :opcode:`MAKE_CELL`, to create :ref:`cell-objects`.
-* Replaced :opcode:`JUMP_IF_NOT_EG_MATCH` by :opcode:`CHECK_EG_MATCH` which
- performs the check but does not jump.
+* :opcode:`CHECK_EG_MATCH` and :opcode:`PREP_RERAISE_STAR`,
+ to handle the :ref:`new exception groups and except* <whatsnew311-pep654>`
+ added in :pep:`654`.
-* Replaced :opcode:`JUMP_ABSOLUTE` by the relative :opcode:`JUMP_BACKWARD`.
+* :opcode:`PUSH_EXC_INFO`, for use in exception handlers.
-* Added :opcode:`JUMP_BACKWARD_NO_INTERRUPT`, which is used in certain loops where it
- is undesirable to handle interrupts.
+* :opcode:`RESUME`, a no-op,
+ for internal tracing, debugging and optimization checks.
-* Replaced :opcode:`POP_JUMP_IF_TRUE` and :opcode:`POP_JUMP_IF_FALSE` by
- the relative :opcode:`POP_JUMP_FORWARD_IF_TRUE`, :opcode:`POP_JUMP_BACKWARD_IF_TRUE`,
- :opcode:`POP_JUMP_FORWARD_IF_FALSE` and :opcode:`POP_JUMP_BACKWARD_IF_FALSE`.
-* Added :opcode:`POP_JUMP_FORWARD_IF_NOT_NONE`, :opcode:`POP_JUMP_BACKWARD_IF_NOT_NONE`,
- :opcode:`POP_JUMP_FORWARD_IF_NONE` and :opcode:`POP_JUMP_BACKWARD_IF_NONE`
- opcodes to speed up conditional jumps.
+.. _whatsnew311-replaced-opcodes:
-* :opcode:`JUMP_IF_TRUE_OR_POP` and :opcode:`JUMP_IF_FALSE_OR_POP` are now
- relative rather than absolute.
+Replaced opcodes
+----------------
-* :opcode:`RESUME` has been added. It is a no-op. Performs internal tracing,
- debugging and optimization checks.
++------------------------------------+-----------------------------------+-----------------------------------------+
+| Replaced Opcode(s) | New Opcode(s) | Notes |
++====================================+===================================+=========================================+
+| | :opcode:`!BINARY_*` | :opcode:`BINARY_OP` | Replaced all numeric binary/in-place |
+| | :opcode:`!INPLACE_*` | | opcodes with a single opcode |
++------------------------------------+-----------------------------------+-----------------------------------------+
+| | :opcode:`!CALL_FUNCTION` | | :opcode:`CALL` | Decouples argument shifting for methods |
+| | :opcode:`!CALL_FUNCTION_KW` | | :opcode:`KW_NAMES` | from handling of keyword arguments; |
+| | :opcode:`!CALL_METHOD` | | :opcode:`PRECALL` | allows better specialization of calls |
+| | | :opcode:`PUSH_NULL` | |
++------------------------------------+-----------------------------------+-----------------------------------------+
+| | :opcode:`!DUP_TOP` | | :opcode:`COPY` | Stack manipulation instructions |
+| | :opcode:`!DUP_TOP_TWO` | | :opcode:`SWAP` | |
+| | :opcode:`!ROT_TWO` | | |
+| | :opcode:`!ROT_THREE` | | |
+| | :opcode:`!ROT_FOUR` | | |
+| | :opcode:`!ROT_N` | | |
++------------------------------------+-----------------------------------+-----------------------------------------+
+| | :opcode:`!JUMP_IF_NOT_EXC_MATCH` | | :opcode:`CHECK_EXC_MATCH` | Now performs check but doesn't jump |
++------------------------------------+-----------------------------------+-----------------------------------------+
+| | :opcode:`!JUMP_ABSOLUTE` | | :opcode:`JUMP_BACKWARD` | See [#bytecode-jump]_; |
+| | :opcode:`!POP_JUMP_IF_FALSE` | | :opcode:`POP_JUMP_BACKWARD_IF_* | ``TRUE``, ``FALSE``, |
+| | :opcode:`!POP_JUMP_IF_TRUE` | <POP_JUMP_BACKWARD_IF_TRUE>` | ``NONE`` and ``NOT_NONE`` variants |
+| | | :opcode:`POP_JUMP_FORWARD_IF_* | for each direction |
+| | <POP_JUMP_FORWARD_IF_TRUE>` | |
++------------------------------------+-----------------------------------+-----------------------------------------+
+| | :opcode:`!SETUP_WITH` | :opcode:`BEFORE_WITH` | :keyword:`with` block setup |
+| | :opcode:`!SETUP_ASYNC_WITH` | | |
++------------------------------------+-----------------------------------+-----------------------------------------+
+
+.. [#bytecode-jump] All jump opcodes are now relative, including the
+ existing :opcode:`JUMP_IF_TRUE_OR_POP` and :opcode:`JUMP_IF_FALSE_OR_POP`.
+ The argument is now an offset from the current instruction
+ rather than an absolute location.
+
+
+.. _whatsnew311-changed-opcodes:
+.. _whatsnew311-removed-opcodes:
+.. _whatsnew311-changed-removed-opcodes:
+
+Changed/removed opcodes
+-----------------------
+
+* Changed :opcode:`MATCH_CLASS` and :opcode:`MATCH_KEYS`
+ to no longer push an additional boolean value to indicate success/failure.
+ Instead, ``None`` is pushed on failure
+ in place of the tuple of extracted values.
+
+* Changed opcodes that work with exceptions to reflect them
+ now being represented as one item on the stack instead of three
+ (see :gh:`89874`).
+
+* Removed :opcode:`!COPY_DICT_WITHOUT_KEYS`, :opcode:`!GEN_START`,
+ :opcode:`!POP_BLOCK`, :opcode:`!SETUP_FINALLY` and :opcode:`!YIELD_FROM`.
+
+
++.. _whatsnew311-deprecated:
++.. _whatsnew311-python-api-deprecated:
+
+
.. _whatsnew311-deprecated:
.. _whatsnew311-python-api-deprecated:
is now deprecated. Support will be removed in Python 3.13. (Contributed by
Jingchen Ye in :gh:`90224`.)
-* The :func:`re.template` function and the corresponding :const:`re.TEMPLATE`
- and :const:`re.T` flags are deprecated, as they were undocumented and
- lacked an obvious purpose. They will be removed in Python 3.13.
- (Contributed by Serhiy Storchaka and Miro Hrončok in :gh:`92728`.)
+* :class:`!webbrowser.MacOSX` is deprecated and will be removed in Python 3.13.
+ It is untested, undocumented, and not used by :mod:`webbrowser` itself.
+ (Contributed by Dong-hee Na in :issue:`42255`.)
+
+* The behavior of returning a value from a :class:`~unittest.TestCase` and
+ :class:`~unittest.IsolatedAsyncioTestCase` test methods (other than the
+ default ``None`` value) is now deprecated.
+
+* Deprecated the following not-formally-documented :mod:`unittest` functions,
+ scheduled for removal in Python 3.13:
+
+ * :func:`!unittest.findTestCases`
+ * :func:`!unittest.makeSuite`
+ * :func:`!unittest.getTestCaseNames`
+
+ Use :class:`~unittest.TestLoader` methods instead:
+ * :meth:`unittest.TestLoader.loadTestsFromModule`
+ * :meth:`unittest.TestLoader.loadTestsFromTestCase`
+ * :meth:`unittest.TestLoader.getTestCaseNames`
+
+ (Contributed by Erlend E. Aasland in :issue:`5846`.)
+
+
++.. _whatsnew311-pending-removal:
++.. _whatsnew311-python-api-pending-removal:
+
.. _whatsnew311-pending-removal:
.. _whatsnew311-python-api-pending-removal:
#define PY_MAJOR_VERSION 3
#define PY_MINOR_VERSION 11
#define PY_MICRO_VERSION 0
- #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_GAMMA
- #define PY_RELEASE_SERIAL 2
+ #define PY_RELEASE_LEVEL PY_RELEASE_LEVEL_FINAL
+ #define PY_RELEASE_SERIAL 0
/* Version as a string */
-#define PY_VERSION "3.11.0"
+#define PY_VERSION "3.11.0rc2+"
/*--end constants--*/
/* Version as a single 4-byte hex number, e.g. 0x010502B2 == 1.5.2b2.
class TestIncompleteFrameAreInvisible(unittest.TestCase):
def test_issue95818(self):
- #See GH-95818 for details
- import gc
- self.addCleanup(gc.set_threshold, *gc.get_threshold())
-
- gc.set_threshold(1,1,1)
- class GCHello:
- def __del__(self):
- print("Destroyed from gc")
-
- def gen():
- yield
-
- fd = open(__file__)
- l = [fd, GCHello()]
- l.append(l)
- del fd
- del l
- gen()
+ # See GH-95818 for details
+ code = textwrap.dedent(f"""
+ import gc
+
+ gc.set_threshold(1,1,1)
+ class GCHello:
+ def __del__(self):
+ print("Destroyed from gc")
+
+ def gen():
+ yield
+
+ fd = open({__file__!r})
+ l = [fd, GCHello()]
+ l.append(l)
+ del fd
+ del l
+ gen()
+ """)
+ assert_python_ok("-c", code)
+
++ @support.cpython_only
++ def test_sneaky_frame_object(self):
++
++ def trace(frame, event, arg):
++ """
++ Don't actually do anything, just force a frame object to be created.
++ """
++
++ def callback(phase, info):
++ """
++ Yo dawg, I heard you like frames, so I'm allocating a frame while
++ you're allocating a frame, so you can have a frame while you have a
++ frame!
++ """
++ nonlocal sneaky_frame_object
++ sneaky_frame_object = sys._getframe().f_back
++ # We're done here:
++ gc.callbacks.remove(callback)
++
++ def f():
++ while True:
++ yield
++
++ old_threshold = gc.get_threshold()
++ old_callbacks = gc.callbacks[:]
++ old_enabled = gc.isenabled()
++ old_trace = sys.gettrace()
++ try:
++ # Stop the GC for a second while we set things up:
++ gc.disable()
++ # Create a paused generator:
++ g = f()
++ next(g)
++ # Move all objects to the oldest generation, and tell the GC to run
++ # on the *very next* allocation:
++ gc.collect()
++ gc.set_threshold(1, 0, 0)
++ # Okay, so here's the nightmare scenario:
++ # - We're tracing the resumption of a generator, which creates a new
++ # frame object.
++ # - The allocation of this frame object triggers a collection
++ # *before* the frame object is actually created.
++ # - During the collection, we request the exact same frame object.
++ # This test does it with a GC callback, but in real code it would
++ # likely be a trace function, weakref callback, or finalizer.
++ # - The collection finishes, and the original frame object is
++ # created. We now have two frame objects fighting over ownership
++ # of the same interpreter frame!
++ sys.settrace(trace)
++ gc.callbacks.append(callback)
++ sneaky_frame_object = None
++ gc.enable()
++ next(g)
++ # g.gi_frame should be the the frame object from the callback (the
++ # one that was *requested* second, but *created* first):
++ self.assertIs(g.gi_frame, sneaky_frame_object)
++ finally:
++ gc.set_threshold(*old_threshold)
++ gc.callbacks[:] = old_callbacks
++ sys.settrace(old_trace)
++ if old_enabled:
++ gc.enable()
+
+
@support.cpython_only
def test_sneaky_frame_object(self):
projects / thirdparty / Python / cpython.git / commitdiff
