Importing this module normally appends site-specific paths to the module search path
and adds :ref:`callables <site-consts>`, including :func:`help` to the built-in
-namespace. However, Python startup option :option:`-S` blocks this and this module
+namespace. However, Python startup option :option:`-S` blocks this, and this module
can be safely imported with no automatic modifications to the module search path
or additions to the builtins. To explicitly trigger the usual site-specific
additions, call the :func:`main` function.
single: # (hash); comment
pair: statement; import
-A path configuration file is a file whose name has the form :file:`{name}.pth`
-and exists in one of the four directories mentioned above; its contents are
-additional items (one per line) to be added to ``sys.path``. Non-existing items
-are never added to ``sys.path``, and no check is made that the item refers to a
-directory rather than a file. No item is added to ``sys.path`` more than
-once. Blank lines and lines beginning with ``#`` are skipped. Lines starting
-with ``import`` (followed by space or tab) are executed.
+The :mod:`!site` module recognizes two startup configuration files of the form
+:file:`{name}.pth` for path configurations, and :file:`{name}.start` for
+pre-first-line code execution. Both files can exist in one of the four
+directories mentioned above. Within each directory, these files are sorted
+alphabetically by filename, then parsed in sorted order.
-.. note::
+.. _site-pth-files:
- An executable line in a :file:`.pth` file is run at every Python startup,
- regardless of whether a particular module is actually going to be used.
- Its impact should thus be kept to a minimum.
- The primary intended purpose of executable lines is to make the
- corresponding module(s) importable
- (load 3rd-party import hooks, adjust :envvar:`PATH` etc).
- Any other initialization is supposed to be done upon a module's
- actual import, if and when it happens.
- Limiting a code chunk to a single line is a deliberate measure
- to discourage putting anything more complex here.
+Path extensions (:file:`.pth` files)
+------------------------------------
+
+:file:`{name}.pth` contains additional items (one per line) to be appended to
+``sys.path``. Items that name non-existing directories are never added to
+``sys.path``, and no check is made that the item refers to a directory rather
+than a file. No item is added to ``sys.path`` more than once. Blank lines
+and lines beginning with ``#`` are skipped.
+
+For backward compatibility, lines starting with ``import`` (followed by space
+or tab) are executed with :func:`exec`.
.. versionchanged:: 3.13
+
The :file:`.pth` files are now decoded by UTF-8 at first and then by the
:term:`locale encoding` if it fails.
+.. versionchanged:: next
+
+ :file:`.pth` file lines starting with ``import`` are deprecated. During
+ the deprecation period, such lines are still executed (except in the case
+ below), but a diagnostic message is emitted only when the :option:`-v` flag
+ is given.
+
+ ``import`` lines in :file:`{name}.pth` are silently ignored when a
+ :ref:`matching <site-start-files>` :file:`{name}.start` file exists.
+
+ Errors on individual lines no longer abort processing of the rest of the
+ file. Each error is reported and the remaining lines continue to be
+ processed.
+
+.. deprecated-removed:: next 3.20
+
+ Decoding :file:`{name}.pth` files in any encoding other than ``utf-8-sig``
+ is deprecated in Python 3.15, and support for decoding from the locale
+ encoding will be removed in Python 3.20.
+
+ ``import`` lines in :file:`{name}.pth` files are deprecated and will be
+ silently ignored in Python 3.18 and 3.19. In Python 3.20 a warning will be
+ produced for ``import`` lines in :file:`{name}.pth` files.
+
+
+.. _site-start-files:
+
+Startup entry points (:file:`.start` files)
+-------------------------------------------
+
+.. versionadded:: next
+
+A startup entry point file is a file whose name has the form
+:file:`{name}.start` and exists in one of the site-packages directories
+described above. Each file specifies entry points to be called during
+interpreter startup, using the ``pkg.mod:callable`` syntax understood by
+:func:`pkgutil.resolve_name`.
+
+Each non-blank line that does not begin with ``#`` must contain an entry
+point reference in the form ``pkg.mod:callable``. The colon and callable
+portion are mandatory. Each callable is invoked with no arguments, and
+any return value is discarded.
+
+:file:`.start` files are processed after all :file:`.pth` path extensions
+have been applied to :data:`sys.path`, ensuring that paths are available
+before any startup code runs.
+
+Unlike :data:`sys.path` extensions from :file:`.pth` files, duplicate entry
+points are **not** de-duplicated --- if an entry point appears more than once,
+it will be called more than once.
+
+If an exception occurs during resolution or invocation of an entry point,
+a traceback is printed to :data:`sys.stderr` and processing continues with
+the remaining entry points.
+
+:file:`.start` files must be encoded in UTF-8.
+
+:pep:`829` defined the original specification for these features.
+
+.. note::
+
+ If a :file:`{name}.start` file exists alongside a :file:`{name}.pth` file
+ with the same base name, any ``import`` lines in the :file:`.pth` file are
+ ignored in favor of the entry points in the :file:`.start` file.
+
+.. note::
+
+ Executable lines (``import`` lines in :file:`{name}.pth` files and
+ :file:`{name}.start` file entry points) are always run at Python startup
+ (unless :option:`-S` is given to disable the ``site.py`` module entirely),
+ regardless of whether a particular module is actually going to be used.
+
+.. note::
+
+ :file:`{name}.start` files invoke :func:`pkgutil.resolve_name` with
+ ``strict=True``, which requires the full ``pkg.mod:callable`` form.
+
.. index::
single: package
triple: path; configuration; file
+
+Startup file examples
+---------------------
+
For example, suppose ``sys.prefix`` and ``sys.exec_prefix`` are set to
:file:`/usr/local`. The Python X.Y library is then installed in
:file:`/usr/local/lib/python{X.Y}`. Suppose this has
a subdirectory :file:`/usr/local/lib/python{X.Y}/site-packages` with three
-subsubdirectories, :file:`foo`, :file:`bar` and :file:`spam`, and two path
+sub-subdirectories, :file:`foo`, :file:`bar` and :file:`spam`, and two path
configuration files, :file:`foo.pth` and :file:`bar.pth`. Assume
:file:`foo.pth` contains the following::
alphabetically before :file:`foo.pth`; and :file:`spam` is omitted because it is
not mentioned in either path configuration file.
+Let's say that there is also a :file:`foo.start` file containing the
+following::
+
+ # foo package startup code
+
+ foo.submod:initialize
+
+Now, after ``sys.path`` has been extended as above, and before Python turns
+control over to user code, the ``foo.submod`` module is imported and the
+``initialize()`` function from that module is called.
+
+
+.. _site-migration-guide:
+
+Migrating from ``import`` lines in ``.pth`` files to ``.start`` files
+---------------------------------------------------------------------
+
+If your package currently ships a :file:`{name}.pth` file, you can keep all
+``sys.path`` extension lines unchanged. Only ``import`` lines need to be
+migrated.
+
+To migrate, create a callable (taking zero arguments) within an importable
+module in your package. Reference it as a ``pkg.mod:callable`` entry point
+in a matching :file:`{name}.start` file. Move everything on your ``import``
+line after the first semi-colon into the ``callable()`` function.
+
+If your package must straddle older Pythons that do not support :pep:`829`
+and newer Pythons that do, change the ``import`` lines in your
+:file:`{name}.pth` to use the following form:
+
+.. code-block:: python
+
+ import pkg.mod; pkg.mod.callable()
+
+Older Pythons will execute these ``import`` lines, while newer Pythons will
+ignore them in favor of the :file:`{name}.start` file. After the straddling
+period, remove all ``import`` lines from your :file:`.pth` files.
+
+
:mod:`!sitecustomize`
---------------------
This function used to be called unconditionally.
-.. function:: addsitedir(sitedir, known_paths=None)
+.. function:: addsitedir(sitedir, known_paths=None, *, defer_processing_start_files=False)
+
+ Add a directory to sys.path and parse the :file:`.pth` and :file:`.start`
+ files found in that directory. Typically used in :mod:`sitecustomize` or
+ :mod:`usercustomize` (see above).
+
+ The *known_paths* argument is an optional set of case-normalized paths
+ used to prevent duplicate :data:`sys.path` entries. When ``None`` (the
+ default), the set is built from the current :data:`sys.path`.
+
+ While :file:`.pth` and :file:`.start` files are always parsed, set
+ *defer_processing_start_files* to ``True`` to prevent processing the
+ startup data found in those files, so that you can process them explicitly
+ (this is typically used by the :func:`main` function).
+
+ .. versionchanged:: next
- Add a directory to sys.path and process its :file:`.pth` files. Typically
- used in :mod:`sitecustomize` or :mod:`usercustomize` (see above).
+ Also processes :file:`.start` files. See :ref:`site-start-files`.
+ All :file:`.pth` and :file:`.start` files are now read and
+ accumulated before any path extensions, ``import`` line execution,
+ or entry point invocations take place.
.. function:: getsitepackages()
.. seealso::
* :pep:`370` -- Per user site-packages directory
+ * :pep:`829` -- Startup entry points and the deprecation of import lines in ``.pth`` files
* :ref:`sys-path-init` -- The initialization of :data:`sys.path`.
it is also checked for site-packages (sys.base_prefix and
sys.base_exec_prefix will always be the "real" prefixes of the Python
installation). If "pyvenv.cfg" (a bootstrap configuration file) contains
-the key "include-system-site-packages" is set to "true"
-(case-insensitive), the system-level prefixes will still also be
-searched for site-packages; otherwise they won't. If the system-level
-prefixes are not included then the user site prefixes are also implicitly
-not searched for site-packages.
-
-All of the resulting site-specific directories, if they exist, are
-appended to sys.path, and also inspected for path configuration
-files.
-
-A path configuration file is a file whose name has the form
-<package>.pth; its contents are additional directories (one per line)
-to be added to sys.path. Non-existing directories (or
-non-directories) are never added to sys.path; no directory is added to
-sys.path more than once. Blank lines and lines beginning with
-'#' are skipped. Lines starting with 'import' are executed.
-
-For example, suppose sys.prefix and sys.exec_prefix are set to
-/usr/local and there is a directory /usr/local/lib/python2.5/site-packages
-with three subdirectories, foo, bar and spam, and two path
-configuration files, foo.pth and bar.pth. Assume foo.pth contains the
-following:
-
- # foo package configuration
- foo
- bar
- bletch
-
-and bar.pth contains:
-
- # bar package configuration
- bar
-
-Then the following directories are added to sys.path, in this order:
-
- /usr/local/lib/python2.5/site-packages/bar
- /usr/local/lib/python2.5/site-packages/foo
-
-Note that bletch is omitted because it doesn't exist; bar precedes foo
-because bar.pth comes alphabetically before foo.pth; and spam is
-omitted because it is not mentioned in either path configuration file.
-
-The readline module is also automatically configured to enable
-completion for systems that support it. This can be overridden in
-sitecustomize, usercustomize or PYTHONSTARTUP. Starting Python in
-isolated mode (-I) disables automatic readline configuration.
-
-After these operations, an attempt is made to import a module
-named sitecustomize, which can perform arbitrary additional
-site-specific customizations. If this import fails with an
-ImportError exception, it is silently ignored.
+the key "include-system-site-packages" set to "true" (case-insensitive),
+the system-level prefixes will still also be searched for site-packages;
+otherwise they won't.
+
+Two kinds of configuration files are processed in each site-packages
+directory:
+
+- <name>.pth files extend sys.path with additional directories (one per
+ line). Lines starting with "import" are deprecated (see PEP 829).
+
+- <name>.start files specify startup entry points using the pkg.mod:callable
+ syntax. These are resolved via pkgutil.resolve_name() and called with no
+ arguments.
+
+When called from main(), all .pth path extensions are applied before any
+.start entry points are executed, ensuring that paths are available before
+startup code runs.
+
+See the documentation for the site module for full details:
+https://docs.python.org/3/library/site.html
"""
import sys
import stat
import errno
+lazy import locale
+lazy import pkgutil
+lazy import traceback
+lazy import warnings
+
# Prefixes for site-packages; add additional prefixes like /usr/local here
PREFIXES = [sys.prefix, sys.exec_prefix]
# Enable per user site-packages directory
USER_BASE = None
-def _trace(message):
+def _trace(message, exc=None):
if sys.flags.verbose:
- print(message, file=sys.stderr)
+ _print_error(message, exc)
-def _warn(*args, **kwargs):
- import warnings
+def _print_error(message, exc=None):
+ """Print an error message to stderr, optionally with a formatted traceback."""
+ print(message, file=sys.stderr)
+ if exc is not None:
+ for record in traceback.format_exception(exc):
+ for line in record.splitlines():
+ print(' ' + line, file=sys.stderr)
+
+def _warn(*args, **kwargs):
warnings.warn(*args, **kwargs)
+def _warn_future_us(message, remove):
+ # Don't call warnings._deprecated() directly because we're lazily importing warnings and don't
+ # want to have to trigger an eager import if it's not necessary. Startup time matters a lot
+ # here and warnings isn't cheap! This inlines the check from
+ # warnings._py_warnings._deprecated().
+ _version = sys.version_info
+ if (_version[:2] > remove) or (_version[:2] == remove and _version[3] != "alpha"):
+ warnings._deprecated(message, remove=remove)
+
+
def makepath(*paths):
dir = os.path.join(*paths)
try:
return d
-def addpackage(sitedir, name, known_paths):
- """Process a .pth file within the site-packages directory:
- For each line in the file, either combine it with sitedir to a path
- and add that to known_paths, or execute it if it starts with 'import '.
+# Accumulated entry points from .start files across all site-packages
+# directories. Execution is deferred until all paths in .pth files have been
+# appended to sys.path. Map the .pth/.start file the data is found in to the
+# data.
+_pending_entrypoints = {}
+_pending_syspaths = {}
+_pending_importexecs = {}
+
+
+def _read_pthstart_file(sitedir, name, suffix):
+ """Parse a .start or .pth file and return (lines, filename).
+
+ On success, ``lines`` is a (possibly empty) list of the file's lines.
+ On failure (file missing, hidden, unreadable, or .start with bad
+ encoding), ``lines`` is ``None`` so callers can distinguish a
+ successfully-read empty file from one that could not be read.
"""
- if known_paths is None:
- known_paths = _init_pathinfo()
- reset = True
- else:
- reset = False
- fullname = os.path.join(sitedir, name)
+ filename = os.path.join(sitedir, name)
+ _trace(f"Reading startup configuration file: {filename}")
+
try:
- st = os.lstat(fullname)
- except OSError:
- return
+ st = os.lstat(filename)
+ except OSError as exc:
+ _trace(f"Cannot stat {filename!r}", exc)
+ return None, filename
+
if ((getattr(st, 'st_flags', 0) & stat.UF_HIDDEN) or
(getattr(st, 'st_file_attributes', 0) & stat.FILE_ATTRIBUTE_HIDDEN)):
- _trace(f"Skipping hidden .pth file: {fullname!r}")
- return
- _trace(f"Processing .pth file: {fullname!r}")
+ _trace(f"Skipping hidden {suffix} file: {filename!r}")
+ return None, filename
+
+ _trace(f"Processing {suffix} file: {filename!r}")
try:
- with io.open_code(fullname) as f:
- pth_content = f.read()
- except OSError:
- return
+ with io.open_code(filename) as f:
+ raw_content = f.read()
+ except OSError as exc:
+ _trace(f"Cannot read {filename!r}", exc)
+ return None, filename
try:
- # Accept BOM markers in .pth files as we do in source files
- # (Windows PowerShell 5.1 makes it hard to emit UTF-8 files without a BOM)
- pth_content = pth_content.decode("utf-8-sig")
+ # Accept BOM markers in .start and .pth files as we do in source files (Windows PowerShell
+ # 5.1 makes it hard to emit UTF-8 files without a BOM).
+ content = raw_content.decode("utf-8-sig")
except UnicodeDecodeError:
- # Fallback to locale encoding for backward compatibility.
- # We will deprecate this fallback in the future.
- import locale
- pth_content = pth_content.decode(locale.getencoding())
- _trace(f"Cannot read {fullname!r} as UTF-8. "
- f"Using fallback encoding {locale.getencoding()!r}")
-
- for n, line in enumerate(pth_content.splitlines(), 1):
- if line.startswith("#"):
+ _trace(f"Cannot read {filename!r} as UTF-8.")
+ # For .pth files only, and then only until Python 3.20, fallback to locale encoding for
+ # backward compatibility.
+ _warn_future_us(
+ ".pth files decoded to locale encoding as a fallback",
+ remove=(3, 20)
+ )
+ if suffix == ".pth":
+ content = raw_content.decode(locale.getencoding())
+ _trace(f"Using fallback encoding {locale.getencoding()!r}")
+ else:
+ return None, filename
+
+ return content.splitlines(), filename
+
+
+def _read_pth_file(sitedir, name, known_paths):
+ """Parse a .pth file, accumulating sys.path extensions and import lines.
+
+ Errors on individual lines do not abort processing of the rest of the
+ file (PEP 829).
+ """
+ lines, filename = _read_pthstart_file(sitedir, name, ".pth")
+ if lines is None:
+ return
+
+ for n, line in enumerate(lines, 1):
+ line = line.strip()
+ if not line or line.startswith("#"):
continue
- if line.strip() == "":
+
+ # In Python 3.18 and 3.19, `import` lines are silently ignored. In
+ # Python 3.20 and beyond, issue a warning when `import` lines in .pth
+ # files are detected.
+ if line.startswith(("import ", "import\t")):
+ _warn_future_us(
+ "import lines in .pth files are silently ignored",
+ remove=(3, 18)
+ )
+ _warn_future_us(
+ "import lines in .pth files are noisily ignored",
+ remove=(3, 20)
+ )
+ _pending_importexecs.setdefault(filename, []).append(line)
continue
+
try:
- if line.startswith(("import ", "import\t")):
+ dir_, dircase = makepath(sitedir, line)
+ except Exception as exc:
+ _trace(f"Error in {filename!r}, line {n:d}: {line!r}", exc)
+ continue
+
+ if dircase in known_paths:
+ _trace(f"In {filename!r}, line {n:d}: "
+ f"skipping duplicate sys.path entry: {dir_}")
+ else:
+ _pending_syspaths.setdefault(filename, []).append(dir_)
+ known_paths.add(dircase)
+
+
+def _read_start_file(sitedir, name):
+ """Parse a .start file for a list of entry point strings."""
+ lines, filename = _read_pthstart_file(sitedir, name, ".start")
+ if lines is None:
+ return
+
+ # PEP 829: the *presence* of a matching .start file disables `import`
+ # line processing in the matched .pth file, regardless of whether the
+ # .start file produced any entry points. Register the filename as a
+ # key now so an empty (or comment-only) .start file still suppresses.
+ entrypoints = _pending_entrypoints.setdefault(filename, [])
+
+ for n, line in enumerate(lines, 1):
+ line = line.strip()
+ if not line or line.startswith("#"):
+ continue
+ # Syntax validation is deferred to entry-point execution time,
+ # where pkgutil.resolve_name(strict=True) enforces the
+ # pkg.mod:callable form.
+ entrypoints.append(line)
+
+
+def _extend_syspath():
+ # We've already filtered out duplicates, either in the existing sys.path
+ # or in all the .pth files we've seen. We've also abspath/normpath'd all
+ # the entries, so all that's left to do is to ensure that the path exists.
+ for filename, dirs in _pending_syspaths.items():
+ for dir_ in dirs:
+ if os.path.exists(dir_):
+ _trace(f"Extending sys.path with {dir_} from {filename}")
+ sys.path.append(dir_)
+ else:
+ _print_error(
+ f"In {filename}: {dir_} does not exist; "
+ f"skipping sys.path append")
+
+
+def _exec_imports():
+ # For all the `import` lines we've seen in .pth files, exec() them in
+ # order. However, if they come from a file with a matching .start, then
+ # we ignore these import lines. For the ones we do process, print a
+ # warning but only when -v was given.
+ for filename, imports in _pending_importexecs.items():
+ name, dot, pth = filename.rpartition(".")
+ assert dot == "." and pth == "pth", f"Bad startup filename: {filename}"
+
+ if f"{name}.start" in _pending_entrypoints:
+ # Skip import lines in favor of entry points.
+ continue
+
+ _trace(
+ f"import lines in {filename} are deprecated, "
+ f"use entry points in a {name}.start file instead."
+ )
+
+ for line in imports:
+ try:
+ _trace(f"Exec'ing from {filename}: {line}")
exec(line)
+ except Exception as exc:
+ _print_error(
+ f"Error in import line from {filename}: {line}", exc)
+
+
+def _execute_start_entrypoints():
+ """Execute all accumulated .start file entry points.
+
+ Called after all site-packages directories have been processed so that
+ sys.path is fully populated before any entry point code runs. Uses
+ pkgutil.resolve_name(strict=True) which both validates the strict
+ pkg.mod:callable form and resolves the entry point in one step.
+ """
+ for filename, entrypoints in _pending_entrypoints.items():
+ for entrypoint in entrypoints:
+ try:
+ _trace(f"Executing entry point: {entrypoint} from {filename}")
+ callable_ = pkgutil.resolve_name(entrypoint, strict=True)
+ except ValueError as exc:
+ _print_error(
+ f"Invalid entry point syntax in {filename}: "
+ f"{entrypoint!r}", exc)
continue
- line = line.rstrip()
- dir, dircase = makepath(sitedir, line)
- if dircase not in known_paths and os.path.exists(dir):
- sys.path.append(dir)
- known_paths.add(dircase)
- except Exception as exc:
- print(f"Error processing line {n:d} of {fullname}:\n",
- file=sys.stderr)
- import traceback
- for record in traceback.format_exception(exc):
- for line in record.splitlines():
- print(' '+line, file=sys.stderr)
- print("\nRemainder of file ignored", file=sys.stderr)
- break
+ except Exception as exc:
+ _print_error(
+ f"Error resolving entry point {entrypoint} "
+ f"from {filename}", exc)
+ continue
+ try:
+ callable_()
+ except Exception as exc:
+ _print_error(
+ f"Error in entry point {entrypoint} from {filename}",
+ exc)
+
+
+def process_startup_files():
+ """Flush all pending sys.path and entry points."""
+ _extend_syspath()
+ _exec_imports()
+ _execute_start_entrypoints()
+ _pending_syspaths.clear()
+ _pending_importexecs.clear()
+ _pending_entrypoints.clear()
+
+
+def addpackage(sitedir, name, known_paths):
+ """Process a .pth file within the site-packages directory."""
+ if known_paths is None:
+ known_paths = _init_pathinfo()
+ reset = True
+ else:
+ reset = False
+ _read_pth_file(sitedir, name, known_paths)
+ process_startup_files()
if reset:
known_paths = None
return known_paths
-def addsitedir(sitedir, known_paths=None):
- """Add 'sitedir' argument to sys.path if missing and handle .pth files in
- 'sitedir'"""
+def addsitedir(sitedir, known_paths=None, *, defer_processing_start_files=False):
+ """Add 'sitedir' argument to sys.path if missing and handle startup
+ files."""
_trace(f"Adding directory: {sitedir!r}")
if known_paths is None:
known_paths = _init_pathinfo()
names = os.listdir(sitedir)
except OSError:
return
- names = [name for name in names
- if name.endswith(".pth") and not name.startswith(".")]
- for name in sorted(names):
- addpackage(sitedir, name, known_paths)
+
+ # The following phases are defined by PEP 829.
+ # Phases 1-3: Read .pth files, accumulating paths and import lines.
+ pth_names = sorted(
+ name for name in names
+ if name.endswith(".pth") and not name.startswith(".")
+ )
+ for name in pth_names:
+ _read_pth_file(sitedir, name, known_paths)
+
+ # Phases 6-7: Discover .start files and accumulate their entry points.
+ # Import lines from .pth files with a matching .start file are discarded
+ # at flush time by _exec_imports().
+ start_names = sorted(
+ name for name in names
+ if name.endswith(".start") and not name.startswith(".")
+ )
+ for name in start_names:
+ _read_start_file(sitedir, name)
+
+ # Generally, when addsitedir() is called explicitly, we'll want to process
+ # all the startup file data immediately. However, when called through
+ # main(), we'll want to batch up all the startup file processing. main()
+ # will set this flag to True to defer processing.
+ if not defer_processing_start_files:
+ process_startup_files()
+
if reset:
known_paths = None
+
return known_paths
return USER_SITE
-def addusersitepackages(known_paths):
+def addusersitepackages(known_paths, *, defer_processing_start_files=False):
"""Add a per user site-package to sys.path
Each user has its own python directory with site-packages in the
user_site = getusersitepackages()
if ENABLE_USER_SITE and os.path.isdir(user_site):
- addsitedir(user_site, known_paths)
+ addsitedir(user_site, known_paths, defer_processing_start_files=defer_processing_start_files)
return known_paths
def getsitepackages(prefixes=None):
sitepackages.append(os.path.join(prefix, "Lib", "site-packages"))
return sitepackages
-def addsitepackages(known_paths, prefixes=None):
+def addsitepackages(known_paths, prefixes=None, *, defer_processing_start_files=False):
"""Add site-packages to sys.path"""
_trace("Processing global site-packages")
for sitedir in getsitepackages(prefixes):
if os.path.isdir(sitedir):
- addsitedir(sitedir, known_paths)
+ addsitedir(sitedir, known_paths, defer_processing_start_files=defer_processing_start_files)
return known_paths
known_paths = venv(known_paths)
if ENABLE_USER_SITE is None:
ENABLE_USER_SITE = check_enableusersite()
- known_paths = addusersitepackages(known_paths)
- known_paths = addsitepackages(known_paths)
+ known_paths = addusersitepackages(known_paths, defer_processing_start_files=True)
+ known_paths = addsitepackages(known_paths, defer_processing_start_files=True)
+ # PEP 829: flush accumulated data from all .pth and .start files.
+ # Paths are extended first, then deprecated import lines are exec'd,
+ # and finally .start entry points are executed — ensuring sys.path is
+ # fully populated before any startup code runs. process_startup_files()
+ # also clears the pending state so a later addsitedir() call does
+ # not re-apply already-processed data.
+ process_startup_files()
setquit()
setcopyright()
sethelper()
from test.support.script_helper import spawn_python, kill_python
import ast
import builtins
+import contextlib
import glob
import io
import os
import sysconfig
import tempfile
from textwrap import dedent
+from types import SimpleNamespace
import urllib.error
import urllib.request
from unittest import mock
# comment or import that is a valid directory name for where the .pth
# file resides; invalid directories are not added
pth_file = PthFile()
- pth_file.cleanup(prep=True) # to make sure that nothing is
- # pre-existing that shouldn't be
- try:
- pth_file.create()
+ # Ensure we have a clean slate.
+ pth_file.cleanup(prep=True)
+ with pth_file.create():
site.addpackage(pth_file.base_dir, pth_file.filename, set())
self.pth_file_tests(pth_file)
- finally:
- pth_file.cleanup()
def make_pth(self, contents, pth_dir='.', pth_name=TESTFN):
# Create a .pth file and return its (abspath, basename).
self.assertRegex(err_out.getvalue(), "line 1")
self.assertRegex(err_out.getvalue(),
re.escape(os.path.join(pth_dir, pth_fn)))
- # XXX: the previous two should be independent checks so that the
- # order doesn't matter. The next three could be a single check
- # but my regex foo isn't good enough to write it.
self.assertRegex(err_out.getvalue(), 'Traceback')
self.assertRegex(err_out.getvalue(), r'import bad-syntax')
self.assertRegex(err_out.getvalue(), 'SyntaxError')
pth_dir, pth_fn = self.make_pth("randompath\nimport nosuchmodule\n")
with captured_stderr() as err_out:
site.addpackage(pth_dir, pth_fn, set())
- self.assertRegex(err_out.getvalue(), "line 2")
self.assertRegex(err_out.getvalue(),
re.escape(os.path.join(pth_dir, pth_fn)))
- # XXX: ditto previous XXX comment.
self.assertRegex(err_out.getvalue(), 'Traceback')
self.assertRegex(err_out.getvalue(), 'ModuleNotFoundError')
def test_addpackage_import_bad_pth_file(self):
# Issue 5258
pth_dir, pth_fn = self.make_pth("abc\x00def\n")
- with captured_stderr() as err_out:
- self.assertFalse(site.addpackage(pth_dir, pth_fn, set()))
- self.maxDiff = None
- self.assertEqual(err_out.getvalue(), "")
for path in sys.path:
if isinstance(path, str):
self.assertNotIn("abc\x00def", path)
def test_addsitedir(self):
- # Same tests for test_addpackage since addsitedir() essentially just
- # calls addpackage() for every .pth file in the directory
+ # addsitedir() reads .pth files and, when called standalone
+ # (known_paths=None), flushes paths and import lines immediately.
pth_file = PthFile()
- pth_file.cleanup(prep=True) # Make sure that nothing is pre-existing
- # that is tested for
- try:
- pth_file.create()
- site.addsitedir(pth_file.base_dir, set())
+ # Ensure we have a clean slate.
+ pth_file.cleanup(prep=True)
+ with pth_file.create():
+ site.addsitedir(pth_file.base_dir)
+ self.pth_file_tests(pth_file)
+
+ def test_addsitedir_explicit_flush(self):
+ # addsitedir() reads .pth files and, with
+ # defer_processing_start_files=True, accumulates pending state
+ # without flushing. A subsequent process_startup_files() call
+ # then applies the paths and runs the import lines.
+ pth_file = PthFile()
+ # Ensure we have a clean slate.
+ pth_file.cleanup(prep=True)
+ with pth_file.create():
+ # Pass defer_processing_start_files=True to prevent flushing.
+ site.addsitedir(pth_file.base_dir, set(),
+ defer_processing_start_files=True)
+ self.assertNotIn(pth_file.imported, sys.modules)
+ site.process_startup_files()
self.pth_file_tests(pth_file)
- finally:
- pth_file.cleanup()
def test_addsitedir_dotfile(self):
pth_file = PthFile('.dotfile')
+ # Ensure we have a clean slate.
pth_file.cleanup(prep=True)
- try:
- pth_file.create()
- site.addsitedir(pth_file.base_dir, set())
+ with pth_file.create():
+ site.addsitedir(pth_file.base_dir)
self.assertNotIn(site.makepath(pth_file.good_dir_path)[0], sys.path)
self.assertIn(pth_file.base_dir, sys.path)
- finally:
- pth_file.cleanup()
@unittest.skipUnless(hasattr(os, 'chflags'), 'test needs os.chflags()')
def test_addsitedir_hidden_flags(self):
pth_file = PthFile()
+ # Ensure we have a clean slate.
pth_file.cleanup(prep=True)
- try:
- pth_file.create()
+ with pth_file.create():
st = os.stat(pth_file.file_path)
os.chflags(pth_file.file_path, st.st_flags | stat.UF_HIDDEN)
- site.addsitedir(pth_file.base_dir, set())
+ site.addsitedir(pth_file.base_dir)
self.assertNotIn(site.makepath(pth_file.good_dir_path)[0], sys.path)
self.assertIn(pth_file.base_dir, sys.path)
- finally:
- pth_file.cleanup()
@unittest.skipUnless(sys.platform == 'win32', 'test needs Windows')
@support.requires_subprocess()
def test_addsitedir_hidden_file_attribute(self):
pth_file = PthFile()
+ # Ensure we have a clean slate.
pth_file.cleanup(prep=True)
- try:
- pth_file.create()
+ with pth_file.create():
subprocess.check_call(['attrib', '+H', pth_file.file_path])
- site.addsitedir(pth_file.base_dir, set())
+ site.addsitedir(pth_file.base_dir)
self.assertNotIn(site.makepath(pth_file.good_dir_path)[0], sys.path)
self.assertIn(pth_file.base_dir, sys.path)
- finally:
- pth_file.cleanup()
# This tests _getuserbase, hence the double underline
# to distinguish from a test for getuserbase
self.assertEqual(sys.stderr.getvalue(), out)
-class PthFile(object):
+class PthFile:
"""Helper class for handling testing of .pth files"""
def __init__(self, filename_base=TESTFN, imported="time",
self.good_dir_path = os.path.join(self.base_dir, self.good_dirname)
self.bad_dir_path = os.path.join(self.base_dir, self.bad_dirname)
+ @contextlib.contextmanager
def create(self):
"""Create a .pth file with a comment, blank lines, an ``import
<self.imported>``, a line with self.good_dirname, and a line with
Creation of the directory for self.good_dir_path (based off of
self.good_dirname) is also performed.
- Make sure to call self.cleanup() to undo anything done by this method.
-
+ Used as a context manager: self.cleanup() is called on exit.
"""
FILE = open(self.file_path, 'w')
try:
finally:
FILE.close()
os.mkdir(self.good_dir_path)
+ try:
+ yield self
+ finally:
+ self.cleanup()
def cleanup(self, prep=False):
"""Make sure that the .pth file is deleted, self.imported is not in
self.assertEqual(output, excepted_output)
+class StartFileTests(unittest.TestCase):
+ """Tests for .start file processing (PEP 829)."""
+
+ def setUp(self):
+ self.enterContext(import_helper.DirsOnSysPath())
+ self.tmpdir = self.sitedir = self.enterContext(os_helper.temp_dir())
+ # Save and clear all pending dicts.
+ self.saved_entrypoints = site._pending_entrypoints.copy()
+ self.saved_syspaths = site._pending_syspaths.copy()
+ self.saved_importexecs = site._pending_importexecs.copy()
+ site._pending_entrypoints.clear()
+ site._pending_syspaths.clear()
+ site._pending_importexecs.clear()
+
+ def tearDown(self):
+ site._pending_entrypoints = self.saved_entrypoints.copy()
+ site._pending_syspaths = self.saved_syspaths.copy()
+ site._pending_importexecs = self.saved_importexecs.copy()
+
+ def _make_start(self, content, name='testpkg'):
+ """Write a <name>.start file and return its basename."""
+ basename = f"{name}.start"
+ filepath = os.path.join(self.tmpdir, basename)
+ with open(filepath, 'w', encoding='utf-8') as f:
+ f.write(content)
+ return basename
+
+ def _make_pth(self, content, name='testpkg'):
+ """Write a <name>.pth file and return its basename."""
+ basename = f"{name}.pth"
+ filepath = os.path.join(self.tmpdir, basename)
+ with open(filepath, 'w', encoding='utf-8') as f:
+ f.write(content)
+ return basename
+
+ def _all_entrypoints(self):
+ """Flatten _pending_entrypoints dict into a list of (filename, entry) tuples."""
+ result = []
+ for filename, entries in site._pending_entrypoints.items():
+ for entry in entries:
+ result.append((filename, entry))
+ return result
+
+ def _just_entrypoints(self):
+ return [entry for filename, entry in self._all_entrypoints()]
+
+ # --- _read_start_file tests ---
+
+ def test_read_start_file_basic(self):
+ self._make_start("os.path:join\n", name='foo')
+ site._read_start_file(self.sitedir, 'foo.start')
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints[fullname], ['os.path:join'])
+
+ def test_read_start_file_multiple_entries(self):
+ self._make_start("os.path:join\nos.path:exists\n", name='foo')
+ site._read_start_file(self.sitedir, 'foo.start')
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints[fullname],
+ ['os.path:join', 'os.path:exists'])
+
+ def test_read_start_file_comments_and_blanks(self):
+ self._make_start("# a comment\n\nos.path:join\n \n", name='foo')
+ site._read_start_file(self.sitedir, 'foo.start')
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints[fullname], ['os.path:join'])
+
+ def test_read_start_file_accepts_all_non_blank_lines(self):
+ # Syntax validation is deferred to entry-point execution time
+ # (where pkgutil.resolve_name(strict=True) enforces the strict
+ # pkg.mod:callable form), so parsing accepts every non-blank,
+ # non-comment line, including syntactically invalid ones.
+ content = (
+ "os.path\n" # no colon
+ "pkg.mod:\n" # empty callable
+ ":callable\n" # empty module
+ "pkg.mod:callable:extra\n" # multiple colons
+ "os.path:join\n" # valid
+ )
+ self._make_start(content, name='foo')
+ site._read_start_file(self.sitedir, 'foo.start')
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints[fullname], [
+ 'os.path',
+ 'pkg.mod:',
+ ':callable',
+ 'pkg.mod:callable:extra',
+ 'os.path:join',
+ ])
+
+ def test_read_start_file_empty(self):
+ # PEP 829: an empty .start file is still registered as present
+ # (with an empty entry-point list) so that it suppresses `import`
+ # lines in any matching .pth file.
+ self._make_start("", name='foo')
+ site._read_start_file(self.sitedir, 'foo.start')
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints, {fullname: []})
+
+ def test_read_start_file_comments_only(self):
+ # As with an empty file, a comments-only .start file is registered
+ # as present so it can suppress matching .pth `import` lines.
+ self._make_start("# just a comment\n# another\n", name='foo')
+ site._read_start_file(self.sitedir, 'foo.start')
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints, {fullname: []})
+
+ def test_read_start_file_nonexistent(self):
+ with captured_stderr():
+ site._read_start_file(self.tmpdir, 'nonexistent.start')
+ self.assertEqual(site._pending_entrypoints, {})
+
+ @unittest.skipUnless(hasattr(os, 'chflags'), 'test needs os.chflags()')
+ def test_read_start_file_hidden_flags(self):
+ self._make_start("os.path:join\n", name='foo')
+ filepath = os.path.join(self.tmpdir, 'foo.start')
+ st = os.stat(filepath)
+ os.chflags(filepath, st.st_flags | stat.UF_HIDDEN)
+ site._read_start_file(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints, {})
+
+ def test_read_start_file_duplicates_not_deduplicated(self):
+ # PEP 829: duplicate entry points are NOT deduplicated.
+ self._make_start("os.path:join\nos.path:join\n", name='foo')
+ site._read_start_file(self.sitedir, 'foo.start')
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints[fullname],
+ ['os.path:join', 'os.path:join'])
+
+ def test_read_start_file_accepts_utf8_bom(self):
+ # PEP 829: .start files MUST be utf-8-sig (UTF-8 with optional BOM).
+ filepath = os.path.join(self.tmpdir, 'foo.start')
+ with open(filepath, 'wb') as f:
+ f.write(b'\xef\xbb\xbf' + b'os.path:join\n')
+ site._read_start_file(self.sitedir, 'foo.start')
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertEqual(
+ site._pending_entrypoints[fullname], ['os.path:join'])
+
+ def test_read_start_file_invalid_utf8_silently_skipped(self):
+ # PEP 829: .start files MUST be utf-8-sig. Unlike .pth, there is
+ # no locale-encoding fallback -- a .start file that is not valid
+ # UTF-8 is silently skipped, with no key registered in
+ # _pending_entrypoints and no output to stderr (parsing errors
+ # are reported only under -v).
+ filepath = os.path.join(self.tmpdir, 'foo.start')
+ with open(filepath, 'wb') as f:
+ # Bare continuation byte -- invalid as a UTF-8 start byte.
+ f.write(b'\x80\x80\x80\n')
+ with captured_stderr() as err:
+ site._read_start_file(self.sitedir, 'foo.start')
+ self.assertEqual(site._pending_entrypoints, {})
+ self.assertEqual(err.getvalue(), "")
+
+ def test_two_start_files_with_duplicates_not_deduplicated(self):
+ self._make_start("os.path:join", name="foo")
+ self._make_start("os.path:join", name="bar")
+ site._read_start_file(self.sitedir, 'foo.start')
+ site._read_start_file(self.sitedir, 'bar.start')
+ self.assertEqual(self._just_entrypoints(),
+ ['os.path:join', 'os.path:join'])
+
+ # --- _read_pth_file tests ---
+
+ def test_read_pth_file_paths(self):
+ subdir = os.path.join(self.sitedir, 'mylib')
+ os.mkdir(subdir)
+ self._make_pth("mylib\n", name='foo')
+ site._read_pth_file(self.sitedir, 'foo.pth', set())
+ fullname = os.path.join(self.sitedir, 'foo.pth')
+ self.assertIn(subdir, site._pending_syspaths[fullname])
+
+ def test_read_pth_file_imports_collected(self):
+ self._make_pth("import sys\n", name='foo')
+ site._read_pth_file(self.sitedir, 'foo.pth', set())
+ fullname = os.path.join(self.sitedir, 'foo.pth')
+ self.assertEqual(site._pending_importexecs[fullname], ['import sys'])
+
+ def test_read_pth_file_comments_and_blanks(self):
+ self._make_pth("# comment\n\n \n", name='foo')
+ site._read_pth_file(self.sitedir, 'foo.pth', set())
+ self.assertEqual(site._pending_syspaths, {})
+ self.assertEqual(site._pending_importexecs, {})
+
+ def test_read_pth_file_deduplication(self):
+ subdir = os.path.join(self.sitedir, 'mylib')
+ os.mkdir(subdir)
+ known_paths = set()
+ self._make_pth("mylib\n", name='a')
+ self._make_pth("mylib\n", name='b')
+ site._read_pth_file(self.sitedir, 'a.pth', known_paths)
+ site._read_pth_file(self.sitedir, 'b.pth', known_paths)
+ # Only one entry across both files.
+ all_dirs = []
+ for dirs in site._pending_syspaths.values():
+ all_dirs.extend(dirs)
+ self.assertEqual(all_dirs, [subdir])
+
+ def test_read_pth_file_bad_line_continues(self):
+ # PEP 829: errors on individual lines don't abort the file.
+ subdir = os.path.join(self.sitedir, 'goodpath')
+ os.mkdir(subdir)
+ self._make_pth("abc\x00def\ngoodpath\n", name='foo')
+ with captured_stderr():
+ site._read_pth_file(self.sitedir, 'foo.pth', set())
+ fullname = os.path.join(self.sitedir, 'foo.pth')
+ self.assertIn(subdir, site._pending_syspaths.get(fullname, []))
+
+ def _flags_with_verbose(self, verbose):
+ # Build a sys.flags clone with verbose overridden but every
+ # other field preserved, so unrelated reads like
+ # sys.flags.optimize during io.open_code() continue to work.
+ attrs = {name: getattr(sys.flags, name)
+ for name in sys.flags.__match_args__}
+ attrs['verbose'] = verbose
+ return SimpleNamespace(**attrs)
+
+ def test_read_pth_file_parse_error_silent_by_default(self):
+ # PEP 829: parse-time errors are silent unless -v is given.
+ # Force the error path by making makepath() raise.
+ self._make_pth("badline\n", name='foo')
+ with mock.patch('site.makepath', side_effect=ValueError("boom")), \
+ mock.patch('sys.flags', self._flags_with_verbose(False)), \
+ captured_stderr() as err:
+ site._read_pth_file(self.sitedir, 'foo.pth', set())
+ self.assertEqual(err.getvalue(), "")
+
+ def test_read_pth_file_parse_error_reported_under_verbose(self):
+ # PEP 829: parse-time errors are reported when -v is given.
+ self._make_pth("badline\n", name='foo')
+ with mock.patch('site.makepath', side_effect=ValueError("boom")), \
+ mock.patch('sys.flags', self._flags_with_verbose(True)), \
+ captured_stderr() as err:
+ site._read_pth_file(self.sitedir, 'foo.pth', set())
+ out = err.getvalue()
+ self.assertIn('Error in', out)
+ self.assertIn('foo.pth', out)
+
+ def test_read_pth_file_locale_fallback(self):
+ # PEP 829: .pth files that fail UTF-8 decoding fall back to the
+ # locale encoding for backward compatibility (deprecated in
+ # 3.15, to be removed in 3.20). Mock locale.getencoding() so
+ # the test does not depend on the host's actual locale.
+ subdir = os.path.join(self.sitedir, 'mylib')
+ os.mkdir(subdir)
+ filepath = os.path.join(self.tmpdir, 'foo.pth')
+ # \xe9 is invalid UTF-8 but valid in latin-1.
+ with open(filepath, 'wb') as f:
+ f.write(b'# caf\xe9 comment\nmylib\n')
+ with mock.patch('locale.getencoding', return_value='latin-1'), \
+ captured_stderr():
+ site._read_pth_file(self.sitedir, 'foo.pth', set())
+ fullname = os.path.join(self.sitedir, 'foo.pth')
+ self.assertIn(subdir, site._pending_syspaths.get(fullname, []))
+
+ # --- _execute_start_entrypoints tests ---
+
+ def test_execute_entrypoints_with_callable(self):
+ # Entrypoint with callable is invoked.
+ mod_dir = os.path.join(self.sitedir, 'epmod')
+ os.mkdir(mod_dir)
+ init_file = os.path.join(mod_dir, '__init__.py')
+ with open(init_file, 'w') as f:
+ f.write("""\
+called = False
+def startup():
+ global called
+ called = True
+""")
+ sys.path.insert(0, self.sitedir)
+ self.addCleanup(sys.modules.pop, 'epmod', None)
+ fullname = os.path.join(self.sitedir, 'epmod.start')
+ site._pending_entrypoints[fullname] = ['epmod:startup']
+ site._execute_start_entrypoints()
+ import epmod
+ self.assertTrue(epmod.called)
+
+ def test_execute_entrypoints_import_error(self):
+ # Import error prints traceback but continues.
+ fullname = os.path.join(self.sitedir, 'bad.start')
+ site._pending_entrypoints[fullname] = [
+ 'nosuchmodule_xyz:func', 'os.path:join']
+ with captured_stderr() as err:
+ site._execute_start_entrypoints()
+ self.assertIn('nosuchmodule_xyz', err.getvalue())
+ # os.path:join should still have been called (no exception for it)
+
+ def test_execute_entrypoints_strict_syntax_rejection(self):
+ # PEP 829: only the strict pkg.mod:callable form is valid.
+ # At entry-point execution, pkgutil.resolve_name(strict=True)
+ # raises ValueError for invalid syntax; the invalid entry is
+ # reported and execution continues with the next one.
+ fullname = os.path.join(self.sitedir, 'bad.start')
+ site._pending_entrypoints[fullname] = [
+ 'os.path', # no colon
+ 'pkg.mod:', # empty callable
+ ':callable', # empty module
+ 'pkg.mod:callable:extra', # multiple colons
+ ]
+ with captured_stderr() as err:
+ site._execute_start_entrypoints()
+ out = err.getvalue()
+ self.assertIn('Invalid entry point syntax', out)
+ for bad in ('os.path', 'pkg.mod:', ':callable',
+ 'pkg.mod:callable:extra'):
+ self.assertIn(bad, out)
+
+ def test_execute_entrypoints_callable_error(self):
+ # Callable that raises prints traceback but continues.
+ mod_dir = os.path.join(self.sitedir, 'badmod')
+ os.mkdir(mod_dir)
+ init_file = os.path.join(mod_dir, '__init__.py')
+ with open(init_file, 'w') as f:
+ f.write("""\
+def fail():
+ raise RuntimeError("boom")
+""")
+ sys.path.insert(0, self.sitedir)
+ self.addCleanup(sys.modules.pop, 'badmod', None)
+ fullname = os.path.join(self.sitedir, 'badmod.start')
+ site._pending_entrypoints[fullname] = ['badmod:fail']
+ with captured_stderr() as err:
+ site._execute_start_entrypoints()
+ self.assertIn('RuntimeError', err.getvalue())
+ self.assertIn('boom', err.getvalue())
+
+ def test_execute_entrypoints_duplicates_called_twice(self):
+ # PEP 829: duplicate entry points execute multiple times.
+ mod_dir = os.path.join(self.sitedir, 'countmod')
+ os.mkdir(mod_dir)
+ init_file = os.path.join(mod_dir, '__init__.py')
+ with open(init_file, 'w') as f:
+ f.write("""\
+call_count = 0
+def bump():
+ global call_count
+ call_count += 1
+""")
+ sys.path.insert(0, self.sitedir)
+ self.addCleanup(sys.modules.pop, 'countmod', None)
+ fullname = os.path.join(self.sitedir, 'countmod.start')
+ site._pending_entrypoints[fullname] = [
+ 'countmod:bump', 'countmod:bump']
+ site._execute_start_entrypoints()
+ import countmod
+ self.assertEqual(countmod.call_count, 2)
+
+ # --- _exec_imports tests ---
+
+ def test_exec_imports_suppressed_by_matching_start(self):
+ # Import lines from foo.pth are suppressed when foo.start exists.
+ pth_fullname = os.path.join(self.sitedir, 'foo.pth')
+ start_fullname = os.path.join(self.sitedir, 'foo.start')
+ site._pending_importexecs[pth_fullname] = ['import sys']
+ site._pending_entrypoints[start_fullname] = ['os.path:join']
+ # Should not exec the import line; no error expected.
+ site._exec_imports()
+
+ def test_exec_imports_not_suppressed_by_different_start(self):
+ # Import lines from foo.pth are NOT suppressed by bar.start.
+ pth_fullname = os.path.join(self.sitedir, 'foo.pth')
+ start_fullname = os.path.join(self.sitedir, 'bar.start')
+ site._pending_importexecs[pth_fullname] = ['import sys']
+ site._pending_entrypoints[start_fullname] = ['os.path:join']
+ # Should execute the import line without error.
+ site._exec_imports()
+
+ def test_exec_imports_suppressed_by_empty_matching_start(self):
+ self._make_start("", name='foo')
+ self._make_pth("import epmod; epmod.startup()", name='foo')
+ mod_dir = os.path.join(self.sitedir, 'epmod')
+ os.mkdir(mod_dir)
+ init_file = os.path.join(mod_dir, '__init__.py')
+ with open(init_file, 'w') as f:
+ f.write("""\
+called = False
+def startup():
+ global called
+ called = True
+""")
+ sys.path.insert(0, self.sitedir)
+ self.addCleanup(sys.modules.pop, 'epmod', None)
+ site._read_pth_file(self.sitedir, 'foo.pth', set())
+ site._read_start_file(self.sitedir, 'foo.start')
+ site._exec_imports()
+ import epmod
+ self.assertFalse(epmod.called)
+
+ # --- _extend_syspath tests ---
+
+ def test_extend_syspath_existing_dir(self):
+ subdir = os.path.join(self.sitedir, 'extlib')
+ os.mkdir(subdir)
+ site._pending_syspaths['test.pth'] = [subdir]
+ site._extend_syspath()
+ self.assertIn(subdir, sys.path)
+
+ def test_extend_syspath_nonexistent_dir(self):
+ nosuch = os.path.join(self.sitedir, 'nosuchdir')
+ site._pending_syspaths['test.pth'] = [nosuch]
+ with captured_stderr() as err:
+ site._extend_syspath()
+ self.assertNotIn(nosuch, sys.path)
+ self.assertIn('does not exist', err.getvalue())
+
+ # --- addsitedir integration tests ---
+
+ def test_addsitedir_discovers_start_files(self):
+ # addsitedir() should discover .start files and accumulate entries.
+ self._make_start("os.path:join\n", name='foo')
+ site.addsitedir(self.sitedir, set(),
+ defer_processing_start_files=True)
+ fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertIn('os.path:join', site._pending_entrypoints[fullname])
+
+ def test_addsitedir_start_suppresses_pth_imports(self):
+ # When foo.start exists, import lines in foo.pth are skipped
+ # at flush time by _exec_imports().
+ self._make_start("os.path:join\n", name='foo')
+ self._make_pth("import sys\n", name='foo')
+ site.addsitedir(self.sitedir, set(),
+ defer_processing_start_files=True)
+ pth_fullname = os.path.join(self.sitedir, 'foo.pth')
+ start_fullname = os.path.join(self.sitedir, 'foo.start')
+ # Import line was collected...
+ self.assertIn('import sys',
+ site._pending_importexecs.get(pth_fullname, []))
+ # ...but _exec_imports() will skip it because foo.start exists.
+ site._exec_imports()
+
+ def test_addsitedir_pth_paths_still_work_with_start(self):
+ # Path lines in .pth files still work even when a .start file exists.
+ subdir = os.path.join(self.sitedir, 'mylib')
+ os.mkdir(subdir)
+ self._make_start("os.path:join\n", name='foo')
+ self._make_pth("mylib\n", name='foo')
+ site.addsitedir(self.sitedir, set(),
+ defer_processing_start_files=True)
+ fullname = os.path.join(self.sitedir, 'foo.pth')
+ self.assertIn(subdir, site._pending_syspaths.get(fullname, []))
+
+ def test_addsitedir_start_alphabetical_order(self):
+ # Multiple .start files are discovered alphabetically.
+ self._make_start("os.path:join\n", name='zzz')
+ self._make_start("os.path:exists\n", name='aaa')
+ site.addsitedir(self.sitedir, set(),
+ defer_processing_start_files=True)
+ all_entries = self._all_entrypoints()
+ entries = [entry for _, entry in all_entries]
+ idx_a = entries.index('os.path:exists')
+ idx_z = entries.index('os.path:join')
+ self.assertLess(idx_a, idx_z)
+
+ def test_addsitedir_pth_before_start(self):
+ # PEP 829: .pth files are scanned before .start files.
+ # Create a .pth and .start with the same basename; verify
+ # the .pth data is collected before .start data.
+ subdir = os.path.join(self.sitedir, 'mylib')
+ os.mkdir(subdir)
+ self._make_pth("mylib\n", name='foo')
+ self._make_start("os.path:join\n", name='foo')
+ site.addsitedir(self.sitedir, set(),
+ defer_processing_start_files=True)
+ # Both should be collected.
+ pth_fullname = os.path.join(self.sitedir, 'foo.pth')
+ start_fullname = os.path.join(self.sitedir, 'foo.start')
+ self.assertIn(subdir, site._pending_syspaths.get(pth_fullname, []))
+ self.assertIn('os.path:join',
+ site._pending_entrypoints.get(start_fullname, []))
+
+ def test_addsitedir_dotfile_start_ignored(self):
+ # .start files starting with '.' are skipped. Defer flushing so
+ # the assertion against _pending_entrypoints is meaningful;
+ # otherwise process_startup_files() would clear the dict
+ # regardless of whether the dotfile was picked up.
+ self._make_start("os.path:join\n", name='.hidden')
+ site.addsitedir(self.sitedir, set(),
+ defer_processing_start_files=True)
+ self.assertEqual(site._pending_entrypoints, {})
+
+ def test_addsitedir_standalone_flushes(self):
+ # When called with known_paths=None (standalone), addsitedir
+ # flushes immediately so the caller sees the effect.
+ subdir = os.path.join(self.sitedir, 'flushlib')
+ os.mkdir(subdir)
+ self._make_pth("flushlib\n", name='foo')
+ site.addsitedir(self.sitedir) # known_paths=None
+ self.assertIn(subdir, sys.path)
+ # Pending dicts should be cleared after flush.
+ self.assertEqual(site._pending_syspaths, {})
+
+ def test_addsitedir_defer_does_not_flush(self):
+ # With defer_processing_start_files=True, addsitedir accumulates
+ # pending state but does not flush; sys.path is updated only when
+ # process_startup_files() is called explicitly.
+ subdir = os.path.join(self.sitedir, 'acclib')
+ os.mkdir(subdir)
+ self._make_pth("acclib\n", name='foo')
+ site.addsitedir(self.sitedir, set(),
+ defer_processing_start_files=True)
+ # Path is pending, not yet on sys.path.
+ self.assertNotIn(subdir, sys.path)
+ fullname = os.path.join(self.sitedir, 'foo.pth')
+ self.assertIn(subdir, site._pending_syspaths.get(fullname, []))
+
+ def test_pth_path_is_available_to_start_entrypoint(self):
+ # Core PEP 829 invariant: all .pth path extensions are applied to
+ # sys.path *before* any .start entry point runs, so an entry
+ # point may live in a module reachable only via a .pth-extended
+ # path. If the flush phases were inverted, resolving the entry
+ # point would fail with ModuleNotFoundError.
+ extdir = os.path.join(self.sitedir, 'extdir')
+ os.mkdir(extdir)
+ modpath = os.path.join(extdir, 'mod.py')
+ with open(modpath, 'w') as f:
+ f.write("""\
+called = False
+def hook():
+ global called
+ called = True
+""")
+ self.addCleanup(sys.modules.pop, 'mod', None)
+
+ # extdir is not on sys.path; only the .pth file makes it so.
+ self.assertNotIn(extdir, sys.path)
+ self._make_pth("extdir\n", name='extlib')
+ self._make_start("mod:hook\n", name='extlib')
+
+ # Standalone addsitedir() triggers the full flush sequence.
+ site.addsitedir(self.sitedir)
+
+ self.assertIn(extdir, sys.path)
+ import mod
+ self.assertTrue(
+ mod.called,
+ "entry point did not run; .pth path was likely not applied "
+ "before .start entry-point execution")
+
+
if __name__ == "__main__":
unittest.main()
projects / thirdparty / Python / cpython.git / commitdiff
