* :exc:`FloatingPointError`
* * .. c:var:: PyObject *PyExc_GeneratorExit
* :exc:`GeneratorExit`
+ * * .. c:var:: PyObject *PyExc_ImportCycleError
+ * :exc:`ImportCycleError`
* * .. c:var:: PyObject *PyExc_ImportError
* :exc:`ImportError`
* * .. c:var:: PyObject *PyExc_IndentationError
.. versionadded:: 3.14
+.. c:function:: PyImport_LazyImportsMode PyImport_GetLazyImportsMode()
+
+ Gets the current lazy imports mode.
+
+ .. versionadded:: next
+
+.. c:function:: PyObject* PyImport_GetLazyImportsFilter()
+
+ Return a :term:`strong reference` to the current lazy imports filter,
+ or ``NULL`` if none exists. This function always succeeds.
+
+ .. versionadded:: next
+
+.. c:function:: int PyImport_SetLazyImportsMode(PyImport_LazyImportsMode mode)
+
+ Similar to :c:func:`PyImport_ImportModuleAttr`, but names are UTF-8 encoded
+ strings instead of Python :class:`str` objects.
+
+ This function always returns ``0``.
+
+ .. versionadded:: next
+
+.. c:function:: int PyImport_SetLazyImportsFilter(PyObject *filter)
+
+ Sets the current lazy imports filter. The *filter* should be a callable that
+ will receive ``(importing_module_name, imported_module_name, [fromlist])``
+ when an import can potentially be lazy and that must return ``True`` if
+ the import should be lazy and ``False`` otherwise.
+
+ Return ``0`` on success and ``-1`` with an exception set otherwise.
+
+ .. versionadded:: next
+
+.. c:type:: PyImport_LazyImportsMode
+
+ Enumeration of possible lazy import modes.
+
+ .. c:enumerator:: PyImport_LAZY_NORMAL
+
+ Respect the ``lazy`` keyword in source code. This is the default mode.
+
+ .. c:enumerator:: PyImport_LAZY_ALL
+
+ Make all imports lazy by default.
+
+ .. c:enumerator:: PyImport_LAZY_NONE
+
+ Disable lazy imports entirely. Even explicit ``lazy`` statements become
+ eager imports.
+
+ .. versionadded:: next
+
.. c:function:: PyObject* PyImport_CreateModuleFromInitfunc(PyObject *spec, PyObject* (*initfunc)(void))
This function is a building block that enables embedders to implement
names=[
alias(name='x'),
alias(name='y'),
- alias(name='z')])])
+ alias(name='z')],
+ is_lazy=0)])
.. class:: ImportFrom(module, names, level)
alias(name='x'),
alias(name='y'),
alias(name='z')],
- level=0)])
+ level=0,
+ is_lazy=0)])
.. class:: alias(name, asname)
names=[
alias(name='a', asname='b'),
alias(name='c')],
- level=2)])
+ level=2,
+ is_lazy=0)])
Control flow
^^^^^^^^^^^^
.. versionadded:: 3.6
+.. exception:: ImportCycleError
+
+ A subclass of :exc:`ImportError` which is raised when a lazy import fails
+ because it (directly or indirectly) tries to import itself.
+
+ .. versionadded:: next
.. exception:: IndexError
.. versionadded:: 3.11
+
+.. function:: get_lazy_imports()
+
+ Returns the current lazy imports mode as a string.
+
+ * ``"normal"``: Only imports explicitly marked with the ``lazy`` keyword
+ are lazy
+ * ``"all"``: All top-level imports are potentially lazy
+ * ``"none"``: All lazy imports are suppressed (even explicitly marked
+ ones)
+
+ See also :func:`set_lazy_imports` and :pep:`810`.
+
+ .. versionadded:: next
+
+
+.. function:: get_lazy_imports_filter()
+
+ Returns the current lazy imports filter function, or ``None`` if no
+ filter is set.
+
+ The filter function is called for every potentially lazy import to
+ determine whether it should actually be lazy. See
+ :func:`set_lazy_imports_filter` for details on the filter function
+ signature.
+
+ .. versionadded:: next
+
+
.. function:: getrefcount(object)
Return the reference count of the *object*. The count returned is generally one
.. versionadded:: 3.11
+
+.. function:: set_lazy_imports(mode)
+
+ Sets the global lazy imports mode. The *mode* parameter must be one of
+ the following strings:
+
+ * ``"normal"``: Only imports explicitly marked with the ``lazy`` keyword
+ are lazy
+ * ``"all"``: All top-level imports become potentially lazy
+ * ``"none"``: All lazy imports are suppressed (even explicitly marked
+ ones)
+
+ This function is intended for advanced users who need to control lazy
+ imports across their entire application. Library developers should
+ generally not use this function as it affects the runtime execution of
+ applications.
+
+ In addition to the mode, lazy imports can be controlled via the filter
+ provided by :func:`set_lazy_imports_filter`.
+
+ See also :func:`get_lazy_imports` and :pep:`810`.
+
+ .. versionadded:: next
+
+
+.. function:: set_lazy_imports_filter(filter)
+
+ Sets the lazy imports filter callback. The *filter* parameter must be a
+ callable or ``None`` to clear the filter.
+
+ The filter function is called for every potentially lazy import to
+ determine whether it should actually be lazy. It must have the following
+ signature::
+
+ def filter(importing_module: str, imported_module: str,
+ fromlist: tuple[str, ...] | None) -> bool
+
+ Where:
+
+ * *importing_module* is the name of the module doing the import
+ * *imported_module* is the name of the module being imported
+ * *fromlist* is the tuple of names being imported (for ``from ... import``
+ statements), or ``None`` for regular imports
+
+ The filter should return ``True`` to allow the import to be lazy, or
+ ``False`` to force an eager import.
+
+ This is an advanced feature intended for specialized users who need
+ fine-grained control over lazy import behavior.
+
+ See also :func:`get_lazy_imports_filter` and :pep:`810`.
+
+ .. versionadded:: next
+
+
.. function:: setprofile(profilefunc)
.. index::
.. seealso:: :pep:`667`
+.. data:: LazyImportType
+
+ The type of lazy import proxy objects. These objects are created when a
+ module is lazily imported and serve as placeholders until the module is
+ actually accessed. This type can be used to detect lazy imports
+ programmatically.
+
+ .. versionadded:: next
+
+ .. seealso:: :pep:`810`
+
+
.. data:: GetSetDescriptorType
The type of objects defined in extension modules with ``PyGetSetDef``, such
- ``match``, ``case``, and ``_``, when used in the :keyword:`match` statement.
- ``type``, when used in the :keyword:`type` statement.
+- ``lazy``, when used before an :keyword:`import` statement.
These syntactically act as keywords in their specific contexts,
but this distinction is done at the parser level, not when tokenizing.
.. versionchanged:: 3.12
``type`` is now a soft keyword.
+.. versionchanged:: next
+ ``lazy`` is now a soft keyword.
+
.. index::
single: _, identifiers
single: __, identifiers
pair: name; binding
pair: keyword; from
pair: keyword; as
+ pair: keyword; lazy
pair: exception; ImportError
single: , (comma); import statement
.. productionlist:: python-grammar
- import_stmt: "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])*
- : | "from" `relative_module` "import" `identifier` ["as" `identifier`]
+ import_stmt: ["lazy"] "import" `module` ["as" `identifier`] ("," `module` ["as" `identifier`])*
+ : | ["lazy"] "from" `relative_module` "import" `identifier` ["as" `identifier`]
: ("," `identifier` ["as" `identifier`])*
- : | "from" `relative_module` "import" "(" `identifier` ["as" `identifier`]
+ : | ["lazy"] "from" `relative_module` "import" "(" `identifier` ["as" `identifier`]
: ("," `identifier` ["as" `identifier`])* [","] ")"
: | "from" `relative_module` "import" "*"
module: (`identifier` ".")* `identifier`
.. _normalization form: https://www.unicode.org/reports/tr15/#Norm_Forms
+.. _lazy-imports:
+.. _lazy:
+
+Lazy imports
+------------
+
+.. index::
+ pair: lazy; import
+ single: lazy import
+
+The :keyword:`lazy` keyword is a :ref:`soft keyword <soft-keywords>` that
+only has special meaning when it appears immediately before an
+:keyword:`import` or :keyword:`from` statement. When an import statement is
+preceded by the :keyword:`lazy` keyword, the import becomes *lazy*: the
+module is not loaded immediately at the import statement. Instead, a lazy
+proxy object is created and bound to the name. The actual module is loaded
+on first use of that name.
+
+Lazy imports are only permitted at module scope. Using :keyword:`lazy`
+inside a function, class body, or
+:keyword:`try`/:keyword:`except`/:keyword:`finally` block raises a
+:exc:`SyntaxError`. Star imports cannot be lazy (``lazy from module import
+*`` is a syntax error), and :ref:`future statements <future>` cannot be
+lazy.
+
+When using ``lazy from ... import``, each imported name is bound to a lazy
+proxy object. The first access to any of these names triggers loading of the
+entire module and resolves only that specific name to its actual value.
+Other names remain as lazy proxies until they are accessed.
+
+Example::
+
+ lazy import json
+ import sys
+
+ print('json' in sys.modules) # False - json module not yet loaded
+
+ # First use triggers loading
+ result = json.dumps({"hello": "world"})
+
+ print('json' in sys.modules) # True - now loaded
+
+If an error occurs during module loading (such as :exc:`ImportError` or
+:exc:`SyntaxError`), it is raised at the point where the lazy import is first
+used, not at the import statement itself.
+
+See :pep:`810` for the full specification of lazy imports.
+
+.. versionadded:: next
+
.. _future:
Future statements
.. versionadded:: 3.14
+ * :samp:`-X lazy_imports={all,none,normal}` controls lazy import behavior.
+ ``all`` makes all imports lazy by default, ``none`` disables lazy imports
+ entirely (even explicit ``lazy`` statements become eager), and ``normal``
+ (the default) respects the ``lazy`` keyword in source code.
+ See also :envvar:`PYTHON_LAZY_IMPORTS`.
+
+ .. versionadded:: next
+
It also allows passing arbitrary values and retrieving them through the
:data:`sys._xoptions` dictionary.
.. versionadded:: 3.14
+.. envvar:: PYTHON_LAZY_IMPORTS
+
+ Controls lazy import behavior. Accepts three values: ``all`` makes all
+ imports lazy by default, ``none`` disables lazy imports entirely (even
+ explicit ``lazy`` statements become eager), and ``normal`` (the default)
+ respects the ``lazy`` keyword in source code.
+
+ See also the :option:`-X lazy_imports <-X>` command-line option.
+
+ .. versionadded:: next
+
Debug-mode variables
~~~~~~~~~~~~~~~~~~~~
.. PEP-sized items next.
+* :pep:`810`: :ref:`Explicit lazy imports for faster startup times
+ <whatsnew315-pep810>`
* :pep:`799`: :ref:`A dedicated profiling package for organizing Python
profiling tools <whatsnew315-profiling-package>`
* :pep:`799`: :ref:`Tachyon: High frequency statistical sampling profiler
New features
============
+.. _whatsnew315-pep810:
+
+:pep:`810`: Explicit lazy imports
+---------------------------------
+
+Large Python applications often suffer from slow startup times. A
+significant contributor to this problem is the import system: when a module
+is imported, Python must locate the file, read it from disk, compile it to
+bytecode, and execute all top-level code. For applications with deep
+dependency trees, this process can take seconds, even when most of the
+imported code is never actually used during a particular run.
+
+Developers have worked around this by moving imports inside functions, using
+:mod:`importlib` to load modules on demand, or restructuring code to avoid
+unnecessary dependencies. These approaches work but make code harder to read
+and maintain, scatter import statements throughout the codebase, and require
+discipline to apply consistently.
+
+Python now provides a cleaner solution through explicit :keyword:`lazy`
+imports using the new ``lazy`` soft keyword. When you mark an import as
+lazy, Python defers the actual module loading until the imported name is
+first used. This gives you the organizational benefits of declaring all
+imports at the top of the file while only paying the loading cost for
+modules you actually use.
+
+The ``lazy`` keyword works with both ``import`` and ``from ... import``
+statements. When you write ``lazy import heavy_module``, Python does not
+immediately load the module. Instead, it creates a lightweight proxy object.
+The actual module loading happens transparently when you first access the
+name:
+
+.. code-block:: python
+
+ lazy import json
+ lazy from datetime import datetime
+
+ print("Starting up...") # json and datetime not loaded yet
+
+ data = json.loads('{"key": "value"}') # json gets loads here
+ now = datetime() # datetime loads here
+
+This mechanism is particularly useful for applications that import many
+modules at the top level but may only use a subset of them in any given run.
+The deferred loading reduces startup latency without requiring code
+restructuring or conditional imports scattered throughout the codebase.
+
+In the case where loading a lazily imported module fails (for example, if
+the module does not exist), Python raises the exception at the point of
+first use rather than at import time. The associated traceback includes both
+the location where the name was accessed and the original import statement,
+making it straightforward to diagnose & debug the failure.
+
+For cases where you want to enable lazy loading globally without modifying
+source code, Python provides the :option:`-X lazy_imports <-X>` command-line
+option and the :envvar:`PYTHON_LAZY_IMPORTS` environment variable. Both
+accept three values: ``all`` makes all imports lazy by default, ``none``
+disables lazy imports entirely (even explicit ``lazy`` statements become
+eager), and ``normal`` (the default) respects the ``lazy`` keyword in source
+code. The :func:`sys.set_lazy_imports` and :func:`sys.get_lazy_imports`
+functions allow changing and querying this mode at runtime.
+
+For more selective control, :func:`sys.set_lazy_imports_filter` accepts a
+callable that determines whether a specific module should be loaded lazily.
+The filter receives three arguments: the importing module's name (or
+``None``), the imported module's name, and the fromlist (or ``None`` for
+regular imports). It should return ``True`` to allow the import to be lazy,
+or ``False`` to force eager loading. This allows patterns like making only
+your own application's modules lazy while keeping third-party dependencies
+eager:
+
+.. code-block:: python
+
+ import sys
+
+ def myapp_filter(importing, imported, fromlist):
+ return imported.startswith("myapp.")
+ sys.set_lazy_imports_filter(myapp_filter)
+ sys.set_lazy_imports("all")
+
+ import myapp.slow_module # lazy (matches filter)
+ import json # eager (does not match filter)
+
+The proxy type itself is available as :data:`types.LazyImportType` for code
+that needs to detect lazy imports programmatically.
+
+There are some restrictions on where the ``lazy`` keyword can be used. Lazy
+imports are only permitted at module scope; using ``lazy`` inside a
+function, class body, or ``try``/``except``/``finally`` block raises a
+:exc:`SyntaxError`. Neither star imports nor future imports can be lazy
+(``lazy from module import *`` and ``lazy from __future__ import ...`` both
+raise :exc:`SyntaxError`).
+
+.. seealso:: :pep:`810` for the full specification and rationale.
+
+(Contributed by Pablo Galindo Salgado and Dino Viehland in :gh:`142349`.)
+
.. _whatsnew315-profiling-package:
:pep:`799`: A dedicated profiling package
(Contributed by Yashp002 in :gh:`143504`.)
+symtable
+--------
+
+* Add :meth:`symtable.Function.get_cells` and :meth:`symtable.Symbol.is_cell` methods.
+ (Contributed by Yashp002 in :gh:`143504`.)
+
+
sys
---
simple_stmt[stmt_ty] (memo):
| assignment
| &"type" type_alias
+ | &('import' | 'from' | "lazy") import_stmt
| e=star_expressions { _PyAST_Expr(e, EXTRA) }
| &'return' return_stmt
- | &('import' | 'from') import_stmt
| &'raise' raise_stmt
| &'pass' pass_stmt
| &'del' del_stmt
| invalid_assert_stmt
| 'assert' a=expression b=[',' z=expression { z }] { _PyAST_Assert(a, b, EXTRA) }
-import_stmt[stmt_ty]:
+import_stmt[stmt_ty](memo):
| invalid_import
| import_name
| import_from
# Import statements
# -----------------
-import_name[stmt_ty]: 'import' a=dotted_as_names { _PyAST_Import(a, EXTRA) }
+import_name[stmt_ty]:
+ | lazy="lazy"? 'import' a=dotted_as_names { _PyAST_Import(a, lazy ? 1 : 0, EXTRA) }
+
# note below: the ('.' | '...') is necessary because '...' is tokenized as ELLIPSIS
import_from[stmt_ty]:
- | 'from' a=('.' | '...')* b=dotted_name 'import' c=import_from_targets {
- _PyPegen_checked_future_import(p, b->v.Name.id, c, _PyPegen_seq_count_dots(a), EXTRA) }
- | 'from' a=('.' | '...')+ 'import' b=import_from_targets {
- _PyAST_ImportFrom(NULL, b, _PyPegen_seq_count_dots(a), EXTRA) }
+ | lazy="lazy"? 'from' a=('.' | '...')* b=dotted_name 'import' c=import_from_targets {
+ _PyPegen_checked_future_import(p, b->v.Name.id, c, _PyPegen_seq_count_dots(a), lazy, EXTRA) }
+ | lazy="lazy"? 'from' a=('.' | '...')+ 'import' b=import_from_targets {
+ _PyAST_ImportFrom(NULL, b, _PyPegen_seq_count_dots(a), lazy ? 1 : 0, EXTRA) }
import_from_targets[asdl_alias_seq*]:
| '(' a=import_from_as_names [','] ')' { a }
| import_from_as_names !','
int enable_gil;
int tlbc_enabled;
#endif
+ int lazy_imports;
/* --- Path configuration inputs ------------ */
int pathconfig_warnings;
PyObject* (*initfunc)(void)
);
+typedef enum {
+ PyImport_LAZY_NORMAL,
+ PyImport_LAZY_ALL,
+ PyImport_LAZY_NONE,
+} PyImport_LazyImportsMode;
+
+#ifndef Py_LIMITED_API
+PyAPI_FUNC(int) PyImport_SetLazyImportsMode(PyImport_LazyImportsMode mode);
+PyAPI_FUNC(int) PyImport_SetLazyImportsFilter(PyObject *filter);
+
+PyAPI_FUNC(PyImport_LazyImportsMode) PyImport_GetLazyImportsMode(void);
+PyAPI_FUNC(PyObject *) PyImport_GetLazyImportsFilter(void);
+#endif
+
#ifndef Py_LIMITED_API
# define Py_CPYTHON_IMPORT_H
# include "cpython/import.h"
struct {
asdl_alias_seq *names;
+ int is_lazy;
} Import;
struct {
identifier module;
asdl_alias_seq *names;
int level;
+ int is_lazy;
} ImportFrom;
struct {
end_col_offset, PyArena *arena);
stmt_ty _PyAST_Assert(expr_ty test, expr_ty msg, int lineno, int col_offset,
int end_lineno, int end_col_offset, PyArena *arena);
-stmt_ty _PyAST_Import(asdl_alias_seq * names, int lineno, int col_offset, int
- end_lineno, int end_col_offset, PyArena *arena);
+stmt_ty _PyAST_Import(asdl_alias_seq * names, int is_lazy, int lineno, int
+ col_offset, int end_lineno, int end_col_offset, PyArena
+ *arena);
stmt_ty _PyAST_ImportFrom(identifier module, asdl_alias_seq * names, int level,
- int lineno, int col_offset, int end_lineno, int
- end_col_offset, PyArena *arena);
+ int is_lazy, int lineno, int col_offset, int
+ end_lineno, int end_col_offset, PyArena *arena);
stmt_ty _PyAST_Global(asdl_identifier_seq * names, int lineno, int col_offset,
int end_lineno, int end_col_offset, PyArena *arena);
stmt_ty _PyAST_Nonlocal(asdl_identifier_seq * names, int lineno, int
PyObject *id;
PyObject *ifs;
PyObject *is_async;
+ PyObject *is_lazy;
PyObject *items;
PyObject *iter;
PyObject *key;
PyAPI_FUNC(void) _PyEval_FormatExcUnbound(PyThreadState *tstate, PyCodeObject *co, int oparg);
PyAPI_FUNC(void) _PyEval_FormatKwargsError(PyThreadState *tstate, PyObject *func, PyObject *kwargs);
PyAPI_FUNC(PyObject *) _PyEval_ImportFrom(PyThreadState *, PyObject *, PyObject *);
-PyAPI_FUNC(PyObject *) _PyEval_ImportName(PyThreadState *, _PyInterpreterFrame *, PyObject *, PyObject *, PyObject *);
+
+PyAPI_FUNC(PyObject *) _PyEval_LazyImportName(
+ PyThreadState *tstate, PyObject *builtins, PyObject *globals,
+ PyObject *locals, PyObject *name, PyObject *fromlist, PyObject *level,
+ int lazy);
+PyAPI_FUNC(PyObject *) _PyEval_LazyImportFrom(
+ PyThreadState *tstate, _PyInterpreterFrame *frame, PyObject *v, PyObject *name);
+PyAPI_FUNC(PyObject *) _PyEval_ImportName(
+ PyThreadState *tstate, PyObject *builtins, PyObject *globals,
+ PyObject *locals, PyObject *name, PyObject *fromlist, PyObject *level);
+PyObject * _PyEval_ImportNameWithImport(
+ PyThreadState *tstate, PyObject *import_func, PyObject *globals,
+ PyObject *locals, PyObject *name, PyObject *fromlist, PyObject *level);
PyAPI_FUNC(PyObject *)_PyEval_MatchClass(PyThreadState *tstate, PyObject *subject, PyObject *type, Py_ssize_t nargs, PyObject *kwargs);
PyAPI_FUNC(PyObject *)_PyEval_MatchKeys(PyThreadState *tstate, PyObject *map, PyObject *keys);
PyAPI_FUNC(void) _PyEval_MonitorRaise(PyThreadState *tstate, _PyInterpreterFrame *frame, _Py_CODEUNIT *instr);
void _PyCompile_PopFBlock(struct _PyCompiler *c, enum _PyCompile_FBlockType t,
_PyJumpTargetLabel block_label);
_PyCompile_FBlockInfo *_PyCompile_TopFBlock(struct _PyCompiler *c);
+bool _PyCompile_InExceptionHandler(struct _PyCompiler *c);
int _PyCompile_EnterScope(struct _PyCompiler *c, identifier name, int scope_type,
void *key, int lineno, PyObject *private,
extern int _PyDict_Contains_KnownHash(PyObject *, PyObject *, Py_hash_t);
+extern void _PyDict_ClearKeysVersionLockHeld(PyObject *mp);
+
extern int _PyDict_Next(
PyObject *mp, Py_ssize_t *pos, PyObject **key, PyObject **value, Py_hash_t *hash);
#define DICT_UNIQUE_ID_SHIFT (32)
#define DICT_UNIQUE_ID_MAX ((UINT64_C(1) << (64 - DICT_UNIQUE_ID_SHIFT)) - 1)
+/* The first three dict watcher IDs are reserved for CPython,
+ * so we don't need to check that they haven't been used */
+#define BUILTINS_WATCHER_ID 0
+#define GLOBALS_WATCHER_ID 1
+#define MODULE_WATCHER_ID 2
+#define FIRST_AVAILABLE_WATCHER 3
+
PyAPI_FUNC(void)
_PyDict_SendEvent(int watcher_bits,
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__iter__));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__itruediv__));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__ixor__));
+ _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__lazy_import__));
+ _PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__lazy_modules__));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__le__));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__len__));
_PyStaticObject_CheckRefcnt((PyObject *)&_Py_ID(__length_hint__));
STRUCT_FOR_ID(__iter__)
STRUCT_FOR_ID(__itruediv__)
STRUCT_FOR_ID(__ixor__)
+ STRUCT_FOR_ID(__lazy_import__)
+ STRUCT_FOR_ID(__lazy_modules__)
STRUCT_FOR_ID(__le__)
STRUCT_FOR_ID(__len__)
STRUCT_FOR_ID(__length_hint__)
PyObject *modules
);
+extern PyObject * _PyImport_ResolveName(
+ PyThreadState *tstate, PyObject *name, PyObject *globals, int level);
+extern PyObject * _PyImport_GetAbsName(
+ PyThreadState *tstate, PyObject *name, PyObject *globals, int level);
+// Symbol is exported for the JIT on Windows builds.
+PyAPI_FUNC(PyObject *) _PyImport_LoadLazyImportTstate(
+ PyThreadState *tstate, PyObject *lazy_import);
+extern PyObject * _PyImport_LazyImportModuleLevelObject(
+ PyThreadState *tstate, PyObject *name, PyObject *builtins,
+ PyObject *globals, PyObject *locals, PyObject *fromlist, int level);
+
+
#ifdef HAVE_DLOPEN
# include <dlfcn.h> // RTLD_NOW, RTLD_LAZY
# if HAVE_DECL_RTLD_NOW
extern void _PyImport_ClearModulesByIndex(PyInterpreterState *interp);
+extern PyObject * _PyImport_InitLazyModules(
+ PyInterpreterState *interp);
+extern void _PyImport_ClearLazyModules(PyInterpreterState *interp);
+
extern int _PyImport_InitDefaultImportFunc(PyInterpreterState *interp);
extern int _PyImport_IsDefaultImportFunc(
PyInterpreterState *interp,
PyObject *func);
+extern int _PyImport_IsDefaultLazyImportFunc(
+ PyInterpreterState *interp,
+ PyObject *func);
+
extern PyObject * _PyImport_GetImportlibLoader(
PyInterpreterState *interp,
const char *loader_name);
int dlopenflags;
#endif
PyObject *import_func;
+ PyObject *lazy_import_func;
+ int lazy_imports_mode;
+ PyObject *lazy_imports_filter;
+ PyObject *lazy_importing_modules;
+ PyObject *lazy_modules;
+#ifdef Py_GIL_DISABLED
+ PyMutex lazy_mutex;
+#endif
/* The global import lock. */
_PyRecursiveMutex lock;
/* diagnostic info in PyImport_ImportModuleLevelObject() */
--- /dev/null
+// Lazy object interface.
+
+#ifndef Py_INTERNAL_LAZYIMPORTOBJECT_H
+#define Py_INTERNAL_LAZYIMPORTOBJECT_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#ifndef Py_BUILD_CORE
+# error "this header requires Py_BUILD_CORE define"
+#endif
+
+PyAPI_DATA(PyTypeObject) PyLazyImport_Type;
+#define PyLazyImport_CheckExact(op) Py_IS_TYPE((op), &PyLazyImport_Type)
+
+typedef struct {
+ PyObject_HEAD
+ PyObject *lz_builtins;
+ PyObject *lz_from;
+ PyObject *lz_attr;
+ // Frame information for the original import location.
+ PyCodeObject *lz_code; // Code object where the lazy import was created.
+ int lz_instr_offset; // Instruction offset where the lazy import was created.
+} PyLazyImportObject;
+
+
+PyAPI_FUNC(PyObject *) _PyLazyImport_GetName(PyObject *lazy_import);
+PyAPI_FUNC(PyObject *) _PyLazyImport_New(
+ struct _PyInterpreterFrame *frame, PyObject *import_func, PyObject *from, PyObject *attr);
+
+#ifdef __cplusplus
+}
+#endif
+#endif // !Py_INTERNAL_LAZYIMPORTOBJECT_H
Python 3.15a1 3653 (Fix handling of opcodes that may leave operands on the stack when optimizing LOAD_FAST)
Python 3.15a1 3654 (Fix missing exception handlers in logical expression)
Python 3.15a1 3655 (Fix miscompilation of some module-level annotations)
- Python 3.15a2 3656 (Add TRACE_RECORD instruction, for platforms with switch based interpreter)
+ Python 3.15a1 3656 (Add TRACE_RECORD instruction, for platforms with switch based interpreter)
Python 3.15a4 3657 (Add BINARY_OP_SUBSCR_USTR_INT)
Python 3.15a4 3658 (Optimize bytecode for list/set called on genexp)
Python 3.15a4 3659 (Add CALL_FUNCTION_EX specialization)
Python 3.15a4 3660 (Change generator preamble code)
+ Python 3.15a4 3661 (Lazy imports IMPORT_NAME opcode changes)
Python 3.16 will start with 3700
*/
-#define PYC_MAGIC_NUMBER 3660
+#define PYC_MAGIC_NUMBER 3661
/* This is equivalent to converting PYC_MAGIC_NUMBER to 2 bytes
(little-endian) and then appending b'\r\n'. */
#define PYC_MAGIC_NUMBER_TOKEN \
extern int _PyModule_IsExtension(PyObject *obj);
+extern int _PyModule_InitModuleDictWatcher(PyInterpreterState *interp);
+
typedef int (*_Py_modexecfunc)(PyObject *);
typedef struct {
INIT_ID(__iter__), \
INIT_ID(__itruediv__), \
INIT_ID(__ixor__), \
+ INIT_ID(__lazy_import__), \
+ INIT_ID(__lazy_modules__), \
INIT_ID(__le__), \
INIT_ID(__len__), \
INIT_ID(__length_hint__), \
unsigned ste_method : 1; /* true if block is a function block defined in class scope */
unsigned ste_has_conditional_annotations : 1; /* true if block has conditionally executed annotations */
unsigned ste_in_conditional_block : 1; /* set while we are inside a conditionally executed block */
+ unsigned ste_in_try_block : 1; /* set while we are inside a try/except block */
unsigned ste_in_unevaluated_annotation : 1; /* set while we are processing an annotation that will not be evaluated */
int ste_comp_iter_expr; /* non-zero if visiting a comprehension range expression */
_Py_SourceLocation ste_loc; /* source location of block */
_PyUnicode_InternStatic(interp, &string);
assert(_PyUnicode_CheckConsistency(string, 1));
assert(PyUnicode_GET_LENGTH(string) != 1);
+ string = &_Py_ID(__lazy_import__);
+ _PyUnicode_InternStatic(interp, &string);
+ assert(_PyUnicode_CheckConsistency(string, 1));
+ assert(PyUnicode_GET_LENGTH(string) != 1);
+ string = &_Py_ID(__lazy_modules__);
+ _PyUnicode_InternStatic(interp, &string);
+ assert(_PyUnicode_CheckConsistency(string, 1));
+ assert(PyUnicode_GET_LENGTH(string) != 1);
string = &_Py_ID(__le__);
_PyUnicode_InternStatic(interp, &string);
assert(_PyUnicode_CheckConsistency(string, 1));
PyAPI_DATA(PyObject *) PyExc_FloatingPointError;
PyAPI_DATA(PyObject *) PyExc_OSError;
PyAPI_DATA(PyObject *) PyExc_ImportError;
+#if !defined(Py_LIMITED_API)
+PyAPI_DATA(PyObject *) PyExc_ImportCycleError;
+#endif
#if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x03060000
PyAPI_DATA(PyObject *) PyExc_ModuleNotFoundError;
#endif
REVERSE_NAME_MAPPING[('builtins', excname)] = ('exceptions', 'OSError')
PYTHON3_IMPORTERROR_EXCEPTIONS = (
+ 'ImportCycleError',
'ModuleNotFoundError',
)
TI(T.NAME, string=s)
):
return not keyword.iskeyword(s)
+ case (
+ None | TI(T.NEWLINE) | TI(T.INDENT) | TI(T.DEDENT) | TI(string=":" | ";"),
+ TI(string="lazy"),
+ TI(string="import") | TI(string="from")
+ ):
+ return True
case _:
return False
FUNCTION_ATTR_FLAGS = ('defaults', 'kwdefaults', 'annotations', 'closure', 'annotate')
ENTER_EXECUTOR = opmap['ENTER_EXECUTOR']
+IMPORT_NAME = opmap['IMPORT_NAME']
LOAD_GLOBAL = opmap['LOAD_GLOBAL']
LOAD_SMALL_INT = opmap['LOAD_SMALL_INT']
BINARY_OP = opmap['BINARY_OP']
argval, argrepr = _get_name_info(arg//4, get_name)
if (arg & 1) and argrepr:
argrepr = f"{argrepr} + NULL|self"
+ elif deop == IMPORT_NAME:
+ argval, argrepr = _get_name_info(arg//4, get_name)
+ if (arg & 1) and argrepr:
+ argrepr = f"{argrepr} + lazy"
+ elif (arg & 2) and argrepr:
+ argrepr = f"{argrepr} + eager"
else:
argval, argrepr = _get_name_info(arg, get_name)
elif deop in hasjump or deop in hasexc:
(level_op[0] in hasconst or level_op[0] == LOAD_SMALL_INT)):
level = _get_const_value(level_op[0], level_op[1], consts)
fromlist = _get_const_value(from_op[0], from_op[1], consts)
- yield (names[oparg], level, fromlist)
+ # IMPORT_NAME encodes lazy/eager flags in bits 0-1,
+ # name index in bits 2+.
+ yield (names[oparg >> 2], level, fromlist)
def _find_store_names(co):
"""Find names of variables which are written in the code
]) +
r"))"
)
+ lazy_softkw = (
+ r"^[ \t]*" + # at beginning of line + possible indentation
+ r"(?P<LAZY_SOFTKW>lazy)" +
+ r"(?=[ \t]+(?:import|from)\b)" # followed by 'import' or 'from'
+ )
builtinlist = [str(name) for name in dir(builtins)
if not name.startswith('_') and
name not in keyword.kwlist]
prog = re.compile("|".join([
builtin, comment, string, kw,
match_softkw, case_default,
- case_softkw_and_pattern,
+ case_softkw_and_pattern, lazy_softkw,
any("SYNC", [r"\n"]),
]),
re.DOTALL | re.MULTILINE)
"CASE_SOFTKW": "KEYWORD",
"CASE_DEFAULT_UNDERSCORE": "KEYWORD",
"CASE_SOFTKW2": "KEYWORD",
+ "LAZY_SOFTKW": "KEYWORD",
}
self._assert_highlighting('case _:', {'KEYWORD': [('1.0', '1.4'),
('1.5', '1.6')]})
+ def test_lazy_soft_keyword(self):
+ # lazy followed by import
+ self._assert_highlighting('lazy import foo',
+ {'KEYWORD': [('1.0', '1.4'),
+ ('1.5', '1.11')]})
+ self._assert_highlighting(' lazy import foo',
+ {'KEYWORD': [('1.4', '1.8'),
+ ('1.9', '1.15')]})
+
+ # lazy followed by from
+ self._assert_highlighting('lazy from foo import bar',
+ {'KEYWORD': [('1.0', '1.4'), ('1.5', '1.9'),
+ ('1.14', '1.20')]})
+
+ # lazy not followed by import/from (not highlighted)
+ self._assert_highlighting('lazy = 1', {})
+ self._assert_highlighting('lazy foo', {})
+
def test_long_multiline_string(self):
source = textwrap.dedent('''\
"""a
except AttributeError:
msg = f"Cannot set an attribute on {parent!r} for child module {child!r}"
_warnings.warn(msg, ImportWarning)
+ # Set attributes to lazy submodules on the module.
+ try:
+ _imp._set_lazy_attributes(module, name)
+ except Exception as e:
+ msg = f"Cannot set lazy attributes on {name!r}: {e!r}"
+ _warnings.warn(msg, ImportWarning)
return module
softkwlist = [
'_',
'case',
+ 'lazy',
'match',
'type'
]
import re
import __main__
import warnings
+import types
__all__ = ["Completer"]
# property method, which is not desirable.
matches.append(match)
continue
- if (value := getattr(thisobject, word, None)) is not None:
+
+ if (isinstance(thisobject, types.ModuleType)
+ and
+ isinstance(thisobject.__dict__.get(word),
+ types.LazyImportType)
+ ):
+ value = thisobject.__dict__.get(word)
+ else:
+ value = getattr(thisobject, word, None)
+
+ if value is not None:
matches.append(self._callable_postfix(value, match))
else:
matches.append(match)
# New grammar constructions may not yet be recognized by Ruff,
# and tests re-use the same names as only the grammar is being checked.
"test_grammar.py",
+ # Lazy import syntax (PEP 810) is not yet supported by Ruff
+ "test_import/data/lazy_imports/*.py",
+ "test_import/data/lazy_imports/**/*.py",
]
[lint]
├── EOFError
├── ExceptionGroup [BaseExceptionGroup]
├── ImportError
+ │ └── ImportCycleError
│ └── ModuleNotFoundError
├── LookupError
│ ├── IndexError
Module(body=[TryStar(body=[Pass()], handlers=[ExceptHandler(type=Name(...), name='exc', body=[Pass(...)])], orelse=[Pass()], finalbody=[Pass()])], type_ignores=[])
Module(body=[Assert(test=Name(id='v', ctx=Load(...)), msg=None)], type_ignores=[])
Module(body=[Assert(test=Name(id='v', ctx=Load(...)), msg=Constant(value='message', kind=None))], type_ignores=[])
-Module(body=[Import(names=[alias(name='sys', asname=None)])], type_ignores=[])
-Module(body=[Import(names=[alias(name='foo', asname='bar')])], type_ignores=[])
-Module(body=[ImportFrom(module='sys', names=[alias(name='x', asname='y')], level=0)], type_ignores=[])
-Module(body=[ImportFrom(module='sys', names=[alias(name='v', asname=None)], level=0)], type_ignores=[])
+Module(body=[Import(names=[alias(name='sys', asname=None)], is_lazy=0)], type_ignores=[])
+Module(body=[Import(names=[alias(name='foo', asname='bar')], is_lazy=0)], type_ignores=[])
+Module(body=[ImportFrom(module='sys', names=[alias(name='x', asname='y')], level=0, is_lazy=0)], type_ignores=[])
+Module(body=[ImportFrom(module='sys', names=[alias(name='v', asname=None)], level=0, is_lazy=0)], type_ignores=[])
+Module(body=[Import(names=[alias(name='sys', asname=None)], is_lazy=1)], type_ignores=[])
+Module(body=[Import(names=[alias(name='foo', asname='bar')], is_lazy=1)], type_ignores=[])
+Module(body=[ImportFrom(module='sys', names=[alias(name='x', asname='y')], level=0, is_lazy=1)], type_ignores=[])
+Module(body=[ImportFrom(module='sys', names=[alias(name='v', asname=None)], level=0, is_lazy=1)], type_ignores=[])
Module(body=[Global(names=['v'])], type_ignores=[])
Module(body=[Expr(value=Constant(value=1, kind=None))], type_ignores=[])
Module(body=[Pass()], type_ignores=[])
# ImportFrom
"from sys import x as y",
"from sys import v",
+ # Lazy Import
+ "lazy import sys",
+ "lazy import foo as bar",
+ # Lazy ImportFrom
+ "lazy from sys import x as y",
+ "lazy from sys import v",
# Global
"global v",
# Expr
('Module', [('TryStar', (1, 0, 7, 6), [('Pass', (2, 2, 2, 6))], [('ExceptHandler', (3, 0, 4, 6), ('Name', (3, 8, 3, 17), 'Exception', ('Load',)), 'exc', [('Pass', (4, 2, 4, 6))])], [('Pass', (5, 7, 5, 11))], [('Pass', (7, 2, 7, 6))])], []),
('Module', [('Assert', (1, 0, 1, 8), ('Name', (1, 7, 1, 8), 'v', ('Load',)), None)], []),
('Module', [('Assert', (1, 0, 1, 19), ('Name', (1, 7, 1, 8), 'v', ('Load',)), ('Constant', (1, 10, 1, 19), 'message', None))], []),
-('Module', [('Import', (1, 0, 1, 10), [('alias', (1, 7, 1, 10), 'sys', None)])], []),
-('Module', [('Import', (1, 0, 1, 17), [('alias', (1, 7, 1, 17), 'foo', 'bar')])], []),
-('Module', [('ImportFrom', (1, 0, 1, 22), 'sys', [('alias', (1, 16, 1, 22), 'x', 'y')], 0)], []),
-('Module', [('ImportFrom', (1, 0, 1, 17), 'sys', [('alias', (1, 16, 1, 17), 'v', None)], 0)], []),
+('Module', [('Import', (1, 0, 1, 10), [('alias', (1, 7, 1, 10), 'sys', None)], 0)], []),
+('Module', [('Import', (1, 0, 1, 17), [('alias', (1, 7, 1, 17), 'foo', 'bar')], 0)], []),
+('Module', [('ImportFrom', (1, 0, 1, 22), 'sys', [('alias', (1, 16, 1, 22), 'x', 'y')], 0, 0)], []),
+('Module', [('ImportFrom', (1, 0, 1, 17), 'sys', [('alias', (1, 16, 1, 17), 'v', None)], 0, 0)], []),
+('Module', [('Import', (1, 0, 1, 15), [('alias', (1, 12, 1, 15), 'sys', None)], 1)], []),
+('Module', [('Import', (1, 0, 1, 22), [('alias', (1, 12, 1, 22), 'foo', 'bar')], 1)], []),
+('Module', [('ImportFrom', (1, 0, 1, 27), 'sys', [('alias', (1, 21, 1, 27), 'x', 'y')], 0, 1)], []),
+('Module', [('ImportFrom', (1, 0, 1, 22), 'sys', [('alias', (1, 21, 1, 22), 'v', None)], 0, 1)], []),
('Module', [('Global', (1, 0, 1, 8), ['v'])], []),
('Module', [('Expr', (1, 0, 1, 1), ('Constant', (1, 0, 1, 1), 1, None))], []),
('Module', [('Pass', (1, 0, 1, 4))], []),
check_text(
"import _ast as ast; from module import sub",
- empty="Module(body=[Import(names=[alias(name='_ast', asname='ast')]), ImportFrom(module='module', names=[alias(name='sub')], level=0)])",
- full="Module(body=[Import(names=[alias(name='_ast', asname='ast')]), ImportFrom(module='module', names=[alias(name='sub')], level=0)], type_ignores=[])",
+ empty="Module(body=[Import(names=[alias(name='_ast', asname='ast')], is_lazy=0), ImportFrom(module='module', names=[alias(name='sub')], level=0, is_lazy=0)])",
+ full="Module(body=[Import(names=[alias(name='_ast', asname='ast')], is_lazy=0), ImportFrom(module='module', names=[alias(name='sub')], level=0, is_lazy=0)], type_ignores=[])",
)
def test_copy_location(self):
("int_max_str_digits", int, None),
("interactive", bool, None),
("isolated", bool, None),
+ ("lazy_imports", int, None),
("malloc_stats", bool, None),
("pymalloc_hugepages", bool, None),
("module_search_paths", list[str], "path"),
1 LOAD_SMALL_INT 0
LOAD_CONST 1 (('*',))
- IMPORT_NAME 0 (math)
+ IMPORT_NAME 2 (math + eager)
CALL_INTRINSIC_1 2 (INTRINSIC_IMPORT_STAR)
POP_TOP
LOAD_CONST 2 (None)
'tracemalloc': 0,
'perf_profiling': 0,
'import_time': 0,
+ 'lazy_imports': -1,
'thread_inherit_context': DEFAULT_THREAD_INHERIT_CONTEXT,
'context_aware_warnings': DEFAULT_CONTEXT_AWARE_WARNINGS,
'code_debug_ranges': True,
--- /dev/null
+def f():
+ pass
+
+x = 42
--- /dev/null
+__lazy_modules__ = ['test.test_import.data.lazy_imports.basic2']
+import test.test_import.data.lazy_imports.basic2
--- /dev/null
+__lazy_modules__ = ['test.test_import.data.lazy_imports.basic2']
+lazy from .basic2 import f
--- /dev/null
+__lazy_modules__ = ['test.test_import.data.lazy_imports.basic2']
+import test.test_import.data.lazy_imports.basic2
+test.test_import.data.lazy_imports.basic2.f()
--- /dev/null
+lazy import test.test_import.data.lazy_imports.basic2
+x = dir()
--- /dev/null
+lazy from test.test_import.data.lazy_imports import basic2
--- /dev/null
+lazy import test.test_import.data.lazy_imports.basic2
--- /dev/null
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
+
+basic2.f()
--- /dev/null
+# Module that exists but doesn't have expected attributes
+x = 42
+# No 'nonexistent_attr' here
--- /dev/null
+# Module that raises an error during import
+raise ValueError("This module always fails to import")
--- /dev/null
+__lazy_modules__ = ['test.test_import.data.lazy_imports.basic2']
+def f():
+ import test.test_import.data.lazy_imports.basic2
+
+f()
--- /dev/null
+__lazy_modules__ = ['test.test_import.data.lazy_imports.basic2']
+try:
+ import test.test_import.data.lazy_imports.basic2
+except:
+ pass
--- /dev/null
+basic = __lazy_import__('test.test_import.data.lazy_imports.basic2')
--- /dev/null
+import sys
+
+def myimport(*args):
+ return sys.modules[__name__]
+
+
+new_globals = dict(globals())
+new_globals["__builtins__"] = {
+ "__import__": myimport,
+}
+basic2 = 42
+basic = __lazy_import__("test.test_import.data.lazy_imports.basic2",
+ globals=new_globals)
+basic
--- /dev/null
+basic = __lazy_import__('test.test_import.data.lazy_imports',
+ fromlist=("basic2", ))
+basic
--- /dev/null
+def f():
+ import test.test_import.data.lazy_imports.basic2 as basic2
+ return basic2
--- /dev/null
+import sys
+
+def filter(module_name, imported_name, from_list):
+ assert module_name == __name__
+ assert imported_name == "test.test_import.data.lazy_imports.basic2"
+ return False
+
+sys.set_lazy_imports_filter(filter)
+
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
--- /dev/null
+import importlib
+
+def filter(module_name, imported_name, from_list):
+ assert module_name == __name__
+ assert imported_name == "test.test_import.data.lazy_imports.basic2"
+ assert from_list == ['f']
+ return False
+
+importlib.set_lazy_imports(None, filter)
+
+lazy from import test.test_import.data.lazy_imports.basic2 import f
--- /dev/null
+import importlib
+
+def filter(module_name, imported_name, from_list):
+ assert module_name == __name__
+ assert imported_name == "test.test_import.data.lazy_imports.basic2"
+ assert from_list == ['f']
+ return True
+
+importlib.set_lazy_imports(None, filter)
+
+lazy from import test.test_import.data.lazy_imports.basic2 import f
--- /dev/null
+import sys
+
+def filter(module_name, imported_name, from_list):
+ assert module_name == __name__
+ assert imported_name == "test.test_import.data.lazy_imports.basic2"
+ return True
+
+sys.set_lazy_imports("normal")
+sys.set_lazy_imports_filter(filter)
+
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
--- /dev/null
+import sys
+
+sys.set_lazy_imports("none")
+
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
--- /dev/null
+import sys
+
+sys.set_lazy_imports("all")
+
+import test.test_import.data.lazy_imports.basic2 as basic2
--- /dev/null
+# Test that globals() returns lazy proxy objects without reifying
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
+
+def get_from_globals():
+ g = globals()
+ return g['basic2']
+
+def get_direct():
+ return basic2
--- /dev/null
+# SyntaxError: lazy import inside class body is not allowed
+class Foo:
+ lazy import json
--- /dev/null
+# Test __lazy_modules__ with from imports
+__lazy_modules__ = ['test.test_import.data.lazy_imports.basic2']
+from test.test_import.data.lazy_imports.basic2 import x, f
+
+def get_x():
+ return x
--- /dev/null
+lazy from __future__ import annotations
--- /dev/null
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
+
+def f():
+ x = globals()
+ return x['basic2'].resolve()
+
+f()
--- /dev/null
+def f():
+ lazy import foo
--- /dev/null
+lazy import test.test_import.data.lazy_imports.pkg.bar
+x = test.test_import.data.lazy_imports.pkg.bar.f
--- /dev/null
+try:
+ lazy import foo
+except:
+ pass
--- /dev/null
+try:
+ lazy from foo import bar
+except:
+ pass
--- /dev/null
+lazy from foo import *
--- /dev/null
+import contextlib
+with contextlib.nullcontext():
+ lazy import test.test_import.data.lazy_imports.basic2
--- /dev/null
+import contextlib
+with contextlib.nullcontext():
+ lazy import test.test_import.data.lazy_imports.basic2 as basic2
--- /dev/null
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
+
+import sys
+mod = sys.modules[__name__]
+x = mod.__dict__
--- /dev/null
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
+
+import sys
+mod = sys.modules[__name__]
+x = mod.basic2
--- /dev/null
+lazy import test.test_import.data.lazy_imports.basic2 as basic2
+
+import sys
+mod = sys.modules[__name__]
+x = mod.__name__
--- /dev/null
+# Test that lazy from import with multiple names only reifies accessed names
+lazy from test.test_import.data.lazy_imports.basic2 import f, x
+
+def get_globals():
+ return globals()
--- /dev/null
+def foo():
+ return 'foo'
--- /dev/null
+print("BAR_MODULE_LOADED")
+def f(): pass
--- /dev/null
+lazy from . import b, x
+
+def get_globals():
+ return globals()
--- /dev/null
+# Test relative imports with lazy keyword
+lazy from . import basic2
+
+def get_basic2():
+ return basic2
--- /dev/null
+# Test relative from imports with lazy keyword
+lazy from .basic2 import x, f
+
+def get_x():
+ return x
+
+def get_f():
+ return f
--- /dev/null
+try:
+ import test.test_import.data.lazy_imports.basic2
+except:
+ pass
--- /dev/null
+try:
+ from test.test_import.data.lazy_imports.basic2 import f
+except:
+ pass
--- /dev/null
+"""Tests for PEP 810 lazy imports."""
+
+import io
+import dis
+import subprocess
+import sys
+import textwrap
+import threading
+import types
+import unittest
+
+try:
+ import _testcapi
+except ImportError:
+ _testcapi = None
+
+
+class LazyImportTests(unittest.TestCase):
+ """Tests for basic lazy import functionality."""
+
+ def tearDown(self):
+ """Clean up any test modules from sys.modules."""
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+ sys.lazy_modules.clear()
+
+ def test_basic_unused(self):
+ """Lazy imported module should not be loaded if never accessed."""
+ import test.test_import.data.lazy_imports.basic_unused
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+ self.assertIn("test.test_import.data.lazy_imports", sys.lazy_modules)
+ self.assertEqual(sys.lazy_modules["test.test_import.data.lazy_imports"], {"basic2"})
+
+ def test_sys_lazy_modules(self):
+ try:
+ import test.test_import.data.lazy_imports.basic_from_unused
+ except ImportError as e:
+ self.fail('lazy import failed')
+
+ self.assertFalse("test.test_import.data.lazy_imports.basic2" in sys.modules)
+ self.assertIn("test.test_import.data.lazy_imports", sys.lazy_modules)
+ self.assertEqual(sys.lazy_modules["test.test_import.data.lazy_imports"], {"basic2"})
+ test.test_import.data.lazy_imports.basic_from_unused.basic2
+ self.assertNotIn("test.test_import.data", sys.lazy_modules)
+
+ def test_basic_unused_use_externally(self):
+ """Lazy import should load module when accessed from outside."""
+ from test.test_import.data.lazy_imports import basic_unused
+
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+ x = basic_unused.test.test_import.data.lazy_imports.basic2
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_basic_from_unused_use_externally(self):
+ """Lazy 'from' import should load when accessed from outside."""
+ from test.test_import.data.lazy_imports import basic_from_unused
+
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+ x = basic_from_unused.basic2
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_basic_unused_dir(self):
+ """dir() on module should not trigger lazy import reification."""
+ import test.test_import.data.lazy_imports.basic_unused
+
+ x = dir(test.test_import.data.lazy_imports.basic_unused)
+ self.assertIn("test", x)
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_basic_dir(self):
+ """dir() at module scope should not trigger lazy import reification."""
+ from test.test_import.data.lazy_imports import basic_dir
+
+ self.assertIn("test", basic_dir.x)
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_basic_used(self):
+ """Lazy import should load when accessed within the module."""
+ import test.test_import.data.lazy_imports.basic_used
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+
+class GlobalLazyImportModeTests(unittest.TestCase):
+ """Tests for sys.set_lazy_imports() global mode control."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_global_off(self):
+ """Mode 'none' should disable lazy imports entirely."""
+ import test.test_import.data.lazy_imports.global_off
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_global_on(self):
+ """Mode 'all' should make regular imports lazy."""
+ import test.test_import.data.lazy_imports.global_on
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_global_filter(self):
+ """Filter returning False should prevent lazy loading."""
+ import test.test_import.data.lazy_imports.global_filter
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_global_filter_true(self):
+ """Filter returning True should allow lazy loading."""
+ import test.test_import.data.lazy_imports.global_filter_true
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_global_filter_from(self):
+ """Filter should work with 'from' imports."""
+ import test.test_import.data.lazy_imports.global_filter
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_global_filter_from_true(self):
+ """Filter returning True should allow lazy 'from' imports."""
+ import test.test_import.data.lazy_imports.global_filter_true
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+
+class CompatibilityModeTests(unittest.TestCase):
+ """Tests for __lazy_modules__ compatibility mode."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_compatibility_mode(self):
+ """__lazy_modules__ should enable lazy imports for listed modules."""
+ import test.test_import.data.lazy_imports.basic_compatibility_mode
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_compatibility_mode_used(self):
+ """Using a lazy import from __lazy_modules__ should load the module."""
+ import test.test_import.data.lazy_imports.basic_compatibility_mode_used
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_compatibility_mode_func(self):
+ """Imports inside functions should be eager even in compatibility mode."""
+ import test.test_import.data.lazy_imports.compatibility_mode_func
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_compatibility_mode_try_except(self):
+ """Imports in try/except should be eager even in compatibility mode."""
+ import test.test_import.data.lazy_imports.compatibility_mode_try_except
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_compatibility_mode_relative(self):
+ """__lazy_modules__ should work with relative imports."""
+ import test.test_import.data.lazy_imports.basic_compatibility_mode_relative
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+
+class ModuleIntrospectionTests(unittest.TestCase):
+ """Tests for module dict and getattr behavior with lazy imports."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_modules_dict(self):
+ """Accessing module.__dict__ should not trigger reification."""
+ import test.test_import.data.lazy_imports.modules_dict
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_modules_getattr(self):
+ """Module __getattr__ for lazy import name should trigger reification."""
+ import test.test_import.data.lazy_imports.modules_getattr
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_modules_getattr_other(self):
+ """Module __getattr__ for other names should not trigger reification."""
+ import test.test_import.data.lazy_imports.modules_getattr_other
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+
+class LazyImportTypeTests(unittest.TestCase):
+ """Tests for the LazyImportType and its resolve() method."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_lazy_value_resolve(self):
+ """resolve() method should force the lazy import to load."""
+ import test.test_import.data.lazy_imports.lazy_get_value
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_lazy_import_type_exposed(self):
+ """LazyImportType should be exposed in types module."""
+ self.assertHasAttr(types, 'LazyImportType')
+ self.assertEqual(types.LazyImportType.__name__, 'lazy_import')
+
+ def test_lazy_import_type_cant_construct(self):
+ """LazyImportType should not be directly constructible."""
+ self.assertRaises(TypeError, types.LazyImportType, {}, "module")
+
+
+class SyntaxRestrictionTests(unittest.TestCase):
+ """Tests for syntax restrictions on lazy imports."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_lazy_try_except(self):
+ """lazy import inside try/except should raise SyntaxError."""
+ with self.assertRaises(SyntaxError):
+ import test.test_import.data.lazy_imports.lazy_try_except
+
+ def test_lazy_try_except_from(self):
+ """lazy from import inside try/except should raise SyntaxError."""
+ with self.assertRaises(SyntaxError):
+ import test.test_import.data.lazy_imports.lazy_try_except_from
+
+ def test_lazy_try_except_from_star(self):
+ """lazy from import * should raise SyntaxError."""
+ with self.assertRaises(SyntaxError):
+ import test.test_import.data.lazy_imports.lazy_try_except_from_star
+
+ def test_lazy_future_import(self):
+ """lazy from __future__ import should raise SyntaxError."""
+ with self.assertRaises(SyntaxError) as cm:
+ import test.test_import.data.lazy_imports.lazy_future_import
+ # Check we highlight 'lazy' (column offset 0, end offset 4)
+ self.assertEqual(cm.exception.offset, 1)
+ self.assertEqual(cm.exception.end_offset, 5)
+
+ def test_lazy_import_func(self):
+ """lazy import inside function should raise SyntaxError."""
+ with self.assertRaises(SyntaxError):
+ import test.test_import.data.lazy_imports.lazy_import_func
+
+ def test_lazy_import_exec_in_function(self):
+ """lazy import via exec() inside a function should raise SyntaxError."""
+ # exec() inside a function creates a non-module-level context
+ # where lazy imports are not allowed
+ def f():
+ exec("lazy import json")
+
+ with self.assertRaises(SyntaxError) as cm:
+ f()
+ self.assertIn("only allowed at module level", str(cm.exception))
+
+ def test_lazy_import_exec_at_module_level(self):
+ """lazy import via exec() at module level should work."""
+ # exec() at module level (globals == locals) should allow lazy imports
+ code = textwrap.dedent("""
+ import sys
+ exec("lazy import json")
+ # Should be lazy - not loaded yet
+ assert 'json' not in sys.modules
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+
+class EagerImportInLazyModeTests(unittest.TestCase):
+ """Tests for imports that should remain eager even in lazy mode."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_try_except_eager(self):
+ """Imports in try/except should be eager even with mode='all'."""
+ sys.set_lazy_imports("all")
+ import test.test_import.data.lazy_imports.try_except_eager
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_try_except_eager_from(self):
+ """From imports in try/except should be eager even with mode='all'."""
+ sys.set_lazy_imports("all")
+ import test.test_import.data.lazy_imports.try_except_eager_from
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_eager_import_func(self):
+ """Imports inside functions should return modules, not proxies."""
+ sys.set_lazy_imports("all")
+ import test.test_import.data.lazy_imports.eager_import_func
+
+ f = test.test_import.data.lazy_imports.eager_import_func.f
+ self.assertEqual(type(f()), type(sys))
+
+
+class WithStatementTests(unittest.TestCase):
+ """Tests for lazy imports in with statement context."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_lazy_with(self):
+ """lazy import with 'with' statement should work."""
+ import test.test_import.data.lazy_imports.lazy_with
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_lazy_with_from(self):
+ """lazy from import with 'with' statement should work."""
+ import test.test_import.data.lazy_imports.lazy_with_from
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+
+class PackageTests(unittest.TestCase):
+ """Tests for lazy imports with packages."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_lazy_import_pkg(self):
+ """lazy import of package submodule should load the package."""
+ import test.test_import.data.lazy_imports.lazy_import_pkg
+
+ self.assertIn("test.test_import.data.lazy_imports.pkg", sys.modules)
+ self.assertIn("test.test_import.data.lazy_imports.pkg.bar", sys.modules)
+
+ def test_lazy_import_pkg_cross_import(self):
+ """Cross-imports within package should preserve lazy imports."""
+ import test.test_import.data.lazy_imports.pkg.c
+
+ self.assertIn("test.test_import.data.lazy_imports.pkg", sys.modules)
+ self.assertIn("test.test_import.data.lazy_imports.pkg.c", sys.modules)
+ self.assertNotIn("test.test_import.data.lazy_imports.pkg.b", sys.modules)
+
+ g = test.test_import.data.lazy_imports.pkg.c.get_globals()
+ self.assertEqual(type(g["x"]), int)
+ self.assertEqual(type(g["b"]), types.LazyImportType)
+
+
+class DunderLazyImportTests(unittest.TestCase):
+ """Tests for __lazy_import__ builtin function."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_dunder_lazy_import(self):
+ """__lazy_import__ should create lazy import proxy."""
+ import test.test_import.data.lazy_imports.dunder_lazy_import
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_dunder_lazy_import_used(self):
+ """Using __lazy_import__ result should trigger module load."""
+ import test.test_import.data.lazy_imports.dunder_lazy_import_used
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_dunder_lazy_import_builtins(self):
+ """__lazy_import__ should use module's __builtins__ for __import__."""
+ from test.test_import.data.lazy_imports import dunder_lazy_import_builtins
+
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+ self.assertEqual(dunder_lazy_import_builtins.basic.basic2, 42)
+
+
+class SysLazyImportsAPITests(unittest.TestCase):
+ """Tests for sys lazy imports API functions."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_set_lazy_imports_requires_string(self):
+ """set_lazy_imports should reject non-string arguments."""
+ with self.assertRaises(TypeError):
+ sys.set_lazy_imports(True)
+ with self.assertRaises(TypeError):
+ sys.set_lazy_imports(None)
+ with self.assertRaises(TypeError):
+ sys.set_lazy_imports(1)
+
+ def test_set_lazy_imports_rejects_invalid_mode(self):
+ """set_lazy_imports should reject invalid mode strings."""
+ with self.assertRaises(ValueError):
+ sys.set_lazy_imports("invalid")
+ with self.assertRaises(ValueError):
+ sys.set_lazy_imports("on")
+ with self.assertRaises(ValueError):
+ sys.set_lazy_imports("off")
+
+ def test_get_lazy_imports_returns_string(self):
+ """get_lazy_imports should return string modes."""
+ sys.set_lazy_imports("normal")
+ self.assertEqual(sys.get_lazy_imports(), "normal")
+
+ sys.set_lazy_imports("all")
+ self.assertEqual(sys.get_lazy_imports(), "all")
+
+ sys.set_lazy_imports("none")
+ self.assertEqual(sys.get_lazy_imports(), "none")
+
+ def test_get_lazy_imports_filter_default(self):
+ """get_lazy_imports_filter should return None by default."""
+ sys.set_lazy_imports_filter(None)
+ self.assertIsNone(sys.get_lazy_imports_filter())
+
+ def test_set_and_get_lazy_imports_filter(self):
+ """set/get_lazy_imports_filter should round-trip filter function."""
+ def my_filter(name):
+ return name.startswith("test.")
+
+ sys.set_lazy_imports_filter(my_filter)
+ self.assertIs(sys.get_lazy_imports_filter(), my_filter)
+
+ def test_lazy_modules_attribute_is_set(self):
+ """sys.lazy_modules should be a set per PEP 810."""
+ self.assertIsInstance(sys.lazy_modules, dict)
+
+ def test_lazy_modules_tracks_lazy_imports(self):
+ """sys.lazy_modules should track lazily imported module names."""
+ code = textwrap.dedent("""
+ import sys
+ initial_count = len(sys.lazy_modules)
+ import test.test_import.data.lazy_imports.basic_unused
+ assert "test.test_import.data.lazy_imports" in sys.lazy_modules
+ assert sys.lazy_modules["test.test_import.data.lazy_imports"] == {"basic2"}
+ assert len(sys.lazy_modules) > initial_count
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+
+class ErrorHandlingTests(unittest.TestCase):
+ """Tests for error handling during lazy import reification.
+
+ PEP 810: Errors during reification should show exception chaining with
+ both the lazy import definition location and the access location.
+ """
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_import_error_shows_chained_traceback(self):
+ """ImportError during reification should chain to show both definition and access."""
+ # Errors at reification must show where the lazy import was defined
+ # AND where the access happened, per PEP 810 "Reification" section
+ code = textwrap.dedent("""
+ import sys
+ lazy import test.test_import.data.lazy_imports.nonexistent_module
+
+ try:
+ x = test.test_import.data.lazy_imports.nonexistent_module
+ except ImportError as e:
+ # Should have __cause__ showing the original error
+ # The exception chain shows both where import was defined and where access happened
+ assert e.__cause__ is not None, "Expected chained exception"
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_attribute_error_on_from_import_shows_chained_traceback(self):
+ """Accessing missing attribute from lazy from-import should chain errors."""
+ # Tests 'lazy from module import nonexistent' behavior
+ code = textwrap.dedent("""
+ import sys
+ lazy from test.test_import.data.lazy_imports.basic2 import nonexistent_name
+
+ try:
+ x = nonexistent_name
+ except ImportError as e:
+ # PEP 810: Enhanced error reporting through exception chaining
+ assert e.__cause__ is not None, "Expected chained exception"
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_reification_retries_on_failure(self):
+ """Failed reification should allow retry on subsequent access.
+
+ PEP 810: "If reification fails, the lazy object is not reified or replaced.
+ Subsequent uses of the lazy object will re-try the reification."
+ """
+ code = textwrap.dedent("""
+ import sys
+ import types
+
+ lazy import test.test_import.data.lazy_imports.broken_module
+
+ # First access - should fail
+ try:
+ x = test.test_import.data.lazy_imports.broken_module
+ except ValueError:
+ pass
+
+ # The lazy object should still be a lazy proxy (not reified)
+ g = globals()
+ lazy_obj = g['test']
+ # The root 'test' binding should still allow retry
+ # Second access - should also fail (retry the import)
+ try:
+ x = test.test_import.data.lazy_imports.broken_module
+ except ValueError:
+ print("OK - retry worked")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_error_during_module_execution_propagates(self):
+ """Errors in module code during reification should propagate correctly."""
+ # Module that raises during import should propagate with chaining
+ code = textwrap.dedent("""
+ import sys
+ lazy import test.test_import.data.lazy_imports.broken_module
+
+ try:
+ _ = test.test_import.data.lazy_imports.broken_module
+ print("FAIL - should have raised")
+ except ValueError as e:
+ # The ValueError from the module should be the cause
+ if "always fails" in str(e) or (e.__cause__ and "always fails" in str(e.__cause__)):
+ print("OK")
+ else:
+ print(f"FAIL - wrong error: {e}")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+
+class GlobalsAndDictTests(unittest.TestCase):
+ """Tests for globals() and __dict__ behavior with lazy imports.
+
+ PEP 810: "Calling globals() or accessing a module's __dict__ does not trigger
+ reification – they return the module's dictionary, and accessing lazy objects
+ through that dictionary still returns lazy proxy objects."
+ """
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_globals_returns_lazy_proxy_when_accessed_from_function(self):
+ """globals() accessed from a function should return lazy proxy without reification.
+
+ Note: At module level, accessing globals()['name'] triggers LOAD_NAME which
+ automatically resolves lazy imports. Inside a function, accessing globals()['name']
+ uses BINARY_SUBSCR which returns the lazy proxy without resolution.
+ """
+ code = textwrap.dedent("""
+ import sys
+ import types
+
+ lazy from test.test_import.data.lazy_imports.basic2 import x
+
+ # Check that module is not yet loaded
+ assert 'test.test_import.data.lazy_imports.basic2' not in sys.modules
+
+ def check_lazy():
+ # Access through globals() from inside a function
+ g = globals()
+ lazy_obj = g['x']
+ return type(lazy_obj) is types.LazyImportType
+
+ # Inside function, should get lazy proxy
+ is_lazy = check_lazy()
+ assert is_lazy, "Expected LazyImportType from function scope"
+
+ # Module should STILL not be loaded
+ assert 'test.test_import.data.lazy_imports.basic2' not in sys.modules
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_globals_dict_access_returns_lazy_proxy_inline(self):
+ """Accessing globals()['name'] inline should return lazy proxy.
+
+ Note: Assigning g['name'] to a local variable at module level triggers
+ reification due to STORE_NAME bytecode. Inline access preserves laziness.
+ """
+ code = textwrap.dedent("""
+ import sys
+ import types
+ lazy import json
+ # Inline access without assignment to local variable preserves lazy proxy
+ assert type(globals()['json']) is types.LazyImportType
+ assert 'json' not in sys.modules
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_module_dict_returns_lazy_proxy_without_reifying(self):
+ """module.__dict__ access should not trigger reification."""
+ import test.test_import.data.lazy_imports.globals_access
+
+ # Module not loaded yet via direct dict access
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ # Access via get_from_globals should return lazy proxy
+ lazy_obj = test.test_import.data.lazy_imports.globals_access.get_from_globals()
+ self.assertEqual(type(lazy_obj), types.LazyImportType)
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_direct_access_triggers_reification(self):
+ """Direct name access (not through globals()) should trigger reification."""
+ import test.test_import.data.lazy_imports.globals_access
+
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ # Direct access should reify
+ result = test.test_import.data.lazy_imports.globals_access.get_direct()
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_resolve_method_forces_reification(self):
+ """Calling resolve() on lazy proxy should force reification.
+
+ Note: Must access lazy proxy from within a function to avoid automatic
+ reification by LOAD_NAME at module level.
+ """
+ code = textwrap.dedent("""
+ import sys
+ import types
+
+ lazy from test.test_import.data.lazy_imports.basic2 import x
+
+ assert 'test.test_import.data.lazy_imports.basic2' not in sys.modules
+
+ def test_resolve():
+ g = globals()
+ lazy_obj = g['x']
+ assert type(lazy_obj) is types.LazyImportType, f"Expected lazy proxy, got {type(lazy_obj)}"
+
+ resolved = lazy_obj.resolve()
+
+ # Now module should be loaded
+ assert 'test.test_import.data.lazy_imports.basic2' in sys.modules
+ assert resolved == 42 # x is 42 in basic2.py
+ return True
+
+ assert test_resolve()
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_add_lazy_to_globals(self):
+ code = textwrap.dedent("""
+ import sys
+ import types
+
+ lazy from test.test_import.data.lazy_imports import basic2
+
+ assert 'test.test_import.data.lazy_imports.basic2' not in sys.modules
+
+ class C: pass
+ sneaky = C()
+ sneaky.x = 1
+
+ def f():
+ t = 0
+ for _ in range(5):
+ t += sneaky.x
+ return t
+
+ f()
+ globals()["sneaky"] = globals()["basic2"]
+ assert f() == 210
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+
+class MultipleNameFromImportTests(unittest.TestCase):
+ """Tests for lazy from ... import with multiple names.
+
+ PEP 810: "When using lazy from ... import, each imported name is bound to a
+ lazy proxy object. The first access to any of these names triggers loading
+ of the entire module and reifies only that specific name to its actual value.
+ Other names remain as lazy proxies until they are accessed."
+ """
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_accessing_one_name_leaves_others_as_proxies(self):
+ """Accessing one name from multi-name import should leave others lazy."""
+ code = textwrap.dedent("""
+ import sys
+ import types
+
+ lazy from test.test_import.data.lazy_imports.basic2 import f, x
+
+ # Neither should be loaded yet
+ assert 'test.test_import.data.lazy_imports.basic2' not in sys.modules
+
+ g = globals()
+ assert type(g['f']) is types.LazyImportType
+ assert type(g['x']) is types.LazyImportType
+
+ # Access 'x' - this loads the module and reifies only 'x'
+ value = x
+ assert value == 42
+
+ # Module is now loaded
+ assert 'test.test_import.data.lazy_imports.basic2' in sys.modules
+
+ # 'x' should be reified (int), 'f' should still be lazy proxy
+ assert type(g['x']) is int, f"Expected int, got {type(g['x'])}"
+ assert type(g['f']) is types.LazyImportType, f"Expected LazyImportType, got {type(g['f'])}"
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_all_names_reified_after_all_accessed(self):
+ """All names should be reified after each is accessed."""
+ code = textwrap.dedent("""
+ import sys
+ import types
+
+ lazy from test.test_import.data.lazy_imports.basic2 import f, x
+
+ g = globals()
+
+ # Access both
+ _ = x
+ _ = f
+
+ # Both should be reified now
+ assert type(g['x']) is int
+ assert callable(g['f'])
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+
+class SysLazyModulesTrackingTests(unittest.TestCase):
+ """Tests for sys.lazy_modules tracking behavior.
+
+ PEP 810: "When the module is reified, it's removed from sys.lazy_modules"
+ """
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_module_added_to_lazy_modules_on_lazy_import(self):
+ """Module should be added to sys.lazy_modules when lazily imported."""
+ # PEP 810 states lazy_modules tracks modules that have been lazily imported
+ # Note: The current implementation keeps modules in lazy_modules even after
+ # reification (primarily for diagnostics and introspection)
+ code = textwrap.dedent("""
+ import sys
+
+ initial_count = len(sys.lazy_modules)
+
+ lazy import test.test_import.data.lazy_imports.basic2
+
+ # Should be in lazy_modules after lazy import
+ assert "test.test_import.data.lazy_imports" in sys.lazy_modules
+ assert sys.lazy_modules["test.test_import.data.lazy_imports"] == {"basic2"}
+ assert len(sys.lazy_modules) > initial_count
+
+ # Trigger reification
+ _ = test.test_import.data.lazy_imports.basic2.x
+
+ # Module should still be tracked (for diagnostics per PEP 810)
+ assert "test.test_import.data.lazy_imports" not in sys.lazy_modules
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_lazy_modules_is_per_interpreter(self):
+ """Each interpreter should have independent sys.lazy_modules."""
+ # Basic test that sys.lazy_modules exists and is a set
+ self.assertIsInstance(sys.lazy_modules, dict)
+
+
+class CommandLineAndEnvVarTests(unittest.TestCase):
+ """Tests for command-line and environment variable control.
+
+ PEP 810: The global lazy imports flag can be controlled through:
+ - The -X lazy_imports=<mode> command-line option
+ - The PYTHON_LAZY_IMPORTS=<mode> environment variable
+ """
+
+ def test_cli_lazy_imports_all_makes_regular_imports_lazy(self):
+ """-X lazy_imports=all should make all imports potentially lazy."""
+ code = textwrap.dedent("""
+ import sys
+ # In 'all' mode, regular imports become lazy
+ import json
+ # json should not be in sys.modules yet (lazy)
+ # Actually accessing it triggers reification
+ if 'json' not in sys.modules:
+ print("LAZY")
+ else:
+ print("EAGER")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-X", "lazy_imports=all", "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
+ self.assertIn("LAZY", result.stdout)
+
+ def test_cli_lazy_imports_none_forces_all_imports_eager(self):
+ """-X lazy_imports=none should force all imports to be eager."""
+ code = textwrap.dedent("""
+ import sys
+ # Even explicit lazy imports should be eager in 'none' mode
+ lazy import json
+ if 'json' in sys.modules:
+ print("EAGER")
+ else:
+ print("LAZY")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-X", "lazy_imports=none", "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
+ self.assertIn("EAGER", result.stdout)
+
+ def test_cli_lazy_imports_normal_respects_lazy_keyword_only(self):
+ """-X lazy_imports=normal should respect lazy keyword only."""
+ # Note: Use test modules instead of stdlib modules to avoid
+ # modules already loaded by the interpreter startup
+ code = textwrap.dedent("""
+ import sys
+ import test.test_import.data.lazy_imports.basic2 # Should be eager
+ lazy import test.test_import.data.lazy_imports.pkg.b # Should be lazy
+
+ eager_loaded = 'test.test_import.data.lazy_imports.basic2' in sys.modules
+ lazy_loaded = 'test.test_import.data.lazy_imports.pkg.b' in sys.modules
+
+ if eager_loaded and not lazy_loaded:
+ print("OK")
+ else:
+ print(f"FAIL: eager={eager_loaded}, lazy={lazy_loaded}")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-X", "lazy_imports=normal", "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_env_var_lazy_imports_all_enables_global_lazy(self):
+ """PYTHON_LAZY_IMPORTS=all should enable global lazy imports."""
+ code = textwrap.dedent("""
+ import sys
+ import json
+ if 'json' not in sys.modules:
+ print("LAZY")
+ else:
+ print("EAGER")
+ """)
+ import os
+ env = os.environ.copy()
+ env["PYTHON_LAZY_IMPORTS"] = "all"
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True,
+ env=env
+ )
+ self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
+ self.assertIn("LAZY", result.stdout)
+
+ def test_env_var_lazy_imports_none_disables_all_lazy(self):
+ """PYTHON_LAZY_IMPORTS=none should disable all lazy imports."""
+ code = textwrap.dedent("""
+ import sys
+ lazy import json
+ if 'json' in sys.modules:
+ print("EAGER")
+ else:
+ print("LAZY")
+ """)
+ import os
+ env = os.environ.copy()
+ env["PYTHON_LAZY_IMPORTS"] = "none"
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True,
+ env=env
+ )
+ self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
+ self.assertIn("EAGER", result.stdout)
+
+ def test_cli_overrides_env_var(self):
+ """Command-line option should take precedence over environment variable."""
+ # PEP 810: -X lazy_imports takes precedence over PYTHON_LAZY_IMPORTS
+ code = textwrap.dedent("""
+ import sys
+ lazy import json
+ if 'json' in sys.modules:
+ print("EAGER")
+ else:
+ print("LAZY")
+ """)
+ import os
+ env = os.environ.copy()
+ env["PYTHON_LAZY_IMPORTS"] = "all" # env says all
+ result = subprocess.run(
+ [sys.executable, "-X", "lazy_imports=none", "-c", code], # CLI says none
+ capture_output=True,
+ text=True,
+ env=env
+ )
+ self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
+ # CLI should win - imports should be eager
+ self.assertIn("EAGER", result.stdout)
+
+ def test_sys_set_lazy_imports_overrides_cli(self):
+ """sys.set_lazy_imports() should take precedence over CLI option."""
+ code = textwrap.dedent("""
+ import sys
+ sys.set_lazy_imports("none") # Override CLI
+ lazy import json
+ if 'json' in sys.modules:
+ print("EAGER")
+ else:
+ print("LAZY")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-X", "lazy_imports=all", "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
+ self.assertIn("EAGER", result.stdout)
+
+
+class FilterFunctionSignatureTests(unittest.TestCase):
+ """Tests for the filter function signature per PEP 810.
+
+ PEP 810: func(importer: str, name: str, fromlist: tuple[str, ...] | None) -> bool
+ """
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_filter_receives_correct_arguments_for_import(self):
+ """Filter should receive (importer, name, fromlist=None) for 'import x'."""
+ code = textwrap.dedent("""
+ import sys
+
+ received_args = []
+
+ def my_filter(importer, name, fromlist):
+ received_args.append((importer, name, fromlist))
+ return True
+
+ sys.set_lazy_imports_filter(my_filter)
+
+ lazy import json
+
+ assert len(received_args) == 1, f"Expected 1 call, got {len(received_args)}"
+ importer, name, fromlist = received_args[0]
+ assert name == "json", f"Expected name='json', got {name!r}"
+ assert fromlist is None, f"Expected fromlist=None, got {fromlist!r}"
+ assert isinstance(importer, str), f"Expected str importer, got {type(importer)}"
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_filter_receives_fromlist_for_from_import(self):
+ """Filter should receive fromlist tuple for 'from x import y, z'."""
+ code = textwrap.dedent("""
+ import sys
+
+ received_args = []
+
+ def my_filter(importer, name, fromlist):
+ received_args.append((importer, name, fromlist))
+ return True
+
+ sys.set_lazy_imports_filter(my_filter)
+
+ lazy from json import dumps, loads
+
+ assert len(received_args) == 1, f"Expected 1 call, got {len(received_args)}"
+ importer, name, fromlist = received_args[0]
+ assert name == "json", f"Expected name='json', got {name!r}"
+ assert fromlist == ("dumps", "loads"), f"Expected ('dumps', 'loads'), got {fromlist!r}"
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_filter_returning_false_forces_eager_import(self):
+ """Filter returning False should make import eager."""
+ code = textwrap.dedent("""
+ import sys
+
+ def deny_filter(importer, name, fromlist):
+ return False
+
+ sys.set_lazy_imports_filter(deny_filter)
+
+ lazy import json
+
+ # Should be eager due to filter
+ if 'json' in sys.modules:
+ print("EAGER")
+ else:
+ print("LAZY")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stderr: {result.stderr}")
+ self.assertIn("EAGER", result.stdout)
+
+
+class AdditionalSyntaxRestrictionTests(unittest.TestCase):
+ """Additional syntax restriction tests per PEP 810."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_lazy_import_inside_class_raises_syntax_error(self):
+ """lazy import inside class body should raise SyntaxError."""
+ # PEP 810: "The soft keyword is only allowed at the global (module) level,
+ # not inside functions, class bodies, try blocks, or import *"
+ with self.assertRaises(SyntaxError):
+ import test.test_import.data.lazy_imports.lazy_class_body
+
+
+class MixedLazyEagerImportTests(unittest.TestCase):
+ """Tests for mixing lazy and eager imports of the same module.
+
+ PEP 810: "If module foo is imported both lazily and eagerly in the same
+ program, the eager import takes precedence and both bindings resolve to
+ the same module object."
+ """
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_eager_import_before_lazy_resolves_to_same_module(self):
+ """Eager import before lazy should make lazy resolve to same module."""
+ code = textwrap.dedent("""
+ import sys
+ import json # Eager import first
+
+ lazy import json as lazy_json # Lazy import same module
+
+ # lazy_json should resolve to the same object
+ assert json is lazy_json, "Lazy and eager imports should resolve to same module"
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_lazy_import_before_eager_resolves_to_same_module(self):
+ """Lazy import followed by eager should both point to same module."""
+ code = textwrap.dedent("""
+ import sys
+
+ lazy import json as lazy_json
+
+ # Lazy not loaded yet
+ assert 'json' not in sys.modules
+
+ import json # Eager import triggers load
+
+ # Both should be the same object
+ assert json is lazy_json
+ print("OK")
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+
+class RelativeImportTests(unittest.TestCase):
+ """Tests for relative imports with lazy keyword."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_relative_lazy_import(self):
+ """lazy from . import submodule should work."""
+ from test.test_import.data.lazy_imports import relative_lazy
+
+ # basic2 should not be loaded yet
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ # Access triggers reification
+ result = relative_lazy.get_basic2()
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ def test_relative_lazy_from_import(self):
+ """lazy from .module import name should work."""
+ from test.test_import.data.lazy_imports import relative_lazy_from
+
+ # basic2 should not be loaded yet
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ # Access triggers reification
+ result = relative_lazy_from.get_x()
+ self.assertEqual(result, 42)
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+
+class LazyModulesCompatibilityFromImportTests(unittest.TestCase):
+ """Tests for __lazy_modules__ with from imports.
+
+ PEP 810: "When a module is made lazy this way, from-imports using that
+ module are also lazy"
+ """
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_lazy_modules_makes_from_imports_lazy(self):
+ """__lazy_modules__ should make from imports of listed modules lazy."""
+ from test.test_import.data.lazy_imports import lazy_compat_from
+
+ # basic2 should not be loaded yet because it's in __lazy_modules__
+ self.assertNotIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+ # Access triggers reification
+ result = lazy_compat_from.get_x()
+ self.assertEqual(result, 42)
+ self.assertIn("test.test_import.data.lazy_imports.basic2", sys.modules)
+
+
+class ImportStateAtReificationTests(unittest.TestCase):
+ """Tests for import system state at reification time.
+
+ PEP 810: "Reification still calls __import__ to resolve the import, which uses
+ the state of the import system (e.g. sys.path, sys.meta_path, sys.path_hooks
+ and __import__) at reification time, not the state when the lazy import
+ statement was evaluated."
+ """
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_sys_path_at_reification_time_is_used(self):
+ """sys.path changes after lazy import should affect reification."""
+ code = textwrap.dedent("""
+ import sys
+ import tempfile
+ import os
+
+ # Create a temporary module
+ with tempfile.TemporaryDirectory() as tmpdir:
+ mod_path = os.path.join(tmpdir, "dynamic_test_module.py")
+ with open(mod_path, "w") as f:
+ f.write("VALUE = 'from_temp_dir'\\n")
+
+ # Lazy import before adding to path
+ lazy import dynamic_test_module
+
+ # Module cannot be found yet
+ try:
+ _ = dynamic_test_module
+ print("FAIL - should not find module")
+ except ModuleNotFoundError:
+ pass
+
+ # Now add temp dir to path
+ sys.path.insert(0, tmpdir)
+
+ # Now reification should succeed using current sys.path
+ assert dynamic_test_module.VALUE == 'from_temp_dir'
+ print("OK")
+
+ sys.path.remove(tmpdir)
+ """)
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+
+class ThreadSafetyTests(unittest.TestCase):
+ """Tests for thread-safety of lazy imports."""
+
+ def tearDown(self):
+ for key in list(sys.modules.keys()):
+ if key.startswith('test.test_import.data.lazy_imports'):
+ del sys.modules[key]
+
+ sys.set_lazy_imports_filter(None)
+ sys.set_lazy_imports("normal")
+
+ def test_concurrent_lazy_import_reification(self):
+ """Multiple threads racing to reify the same lazy import should succeed."""
+ from test.test_import.data.lazy_imports import basic_unused
+
+ num_threads = 10
+ results = [None] * num_threads
+ errors = []
+ barrier = threading.Barrier(num_threads)
+
+ def access_lazy_import(idx):
+ try:
+ barrier.wait()
+ module = basic_unused.test.test_import.data.lazy_imports.basic2
+ results[idx] = module
+ except Exception as e:
+ errors.append((idx, e))
+
+ threads = [
+ threading.Thread(target=access_lazy_import, args=(i,))
+ for i in range(num_threads)
+ ]
+
+ for t in threads:
+ t.start()
+ for t in threads:
+ t.join()
+
+ self.assertEqual(errors, [], f"Errors occurred: {errors}")
+ self.assertTrue(all(r is not None for r in results))
+ first_module = results[0]
+ for r in results[1:]:
+ self.assertIs(r, first_module)
+
+ def test_concurrent_reification_multiple_modules(self):
+ """Multiple threads reifying different lazy imports concurrently."""
+ code = textwrap.dedent("""
+ import sys
+ import threading
+
+ sys.set_lazy_imports("all")
+
+ lazy import json
+ lazy import os
+ lazy import io
+ lazy import re
+
+ num_threads = 8
+ results = {}
+ errors = []
+ barrier = threading.Barrier(num_threads)
+
+ def access_modules(idx):
+ try:
+ barrier.wait()
+ mods = [json, os, io, re]
+ results[idx] = [type(m).__name__ for m in mods]
+ except Exception as e:
+ errors.append((idx, e))
+
+ threads = [
+ threading.Thread(target=access_modules, args=(i,))
+ for i in range(num_threads)
+ ]
+
+ for t in threads:
+ t.start()
+ for t in threads:
+ t.join()
+
+ assert not errors, f"Errors: {errors}"
+ for idx, mods in results.items():
+ assert all(m == 'module' for m in mods), f"Thread {idx} got: {mods}"
+
+ print("OK")
+ """)
+
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_concurrent_lazy_modules_set_updates(self):
+ """Multiple threads creating lazy imports should safely update sys.lazy_modules."""
+ code = textwrap.dedent("""
+ import sys
+ import threading
+
+ num_threads = 16
+ iterations = 50
+ errors = []
+ barrier = threading.Barrier(num_threads)
+
+ def create_lazy_imports(idx):
+ try:
+ barrier.wait()
+ for i in range(iterations):
+ exec(f"lazy import json as json_{idx}_{i}", globals())
+ exec(f"lazy import os as os_{idx}_{i}", globals())
+ except Exception as e:
+ errors.append((idx, e))
+
+ threads = [
+ threading.Thread(target=create_lazy_imports, args=(i,))
+ for i in range(num_threads)
+ ]
+
+ for t in threads:
+ t.start()
+ for t in threads:
+ t.join()
+
+ assert not errors, f"Errors: {errors}"
+ assert isinstance(sys.lazy_modules, dict), "sys.lazy_modules is not a dict"
+ print("OK")
+ """)
+
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_concurrent_reification_same_module_high_contention(self):
+ """High contention: many threads reifying the exact same lazy import."""
+ code = textwrap.dedent("""
+ import sys
+ import threading
+ import types
+
+ sys.set_lazy_imports("all")
+
+ lazy import json
+
+ num_threads = 20
+ results = [None] * num_threads
+ errors = []
+ barrier = threading.Barrier(num_threads)
+
+ def access_json(idx):
+ try:
+ barrier.wait()
+ for _ in range(100):
+ _ = json.dumps
+ _ = json.loads
+ results[idx] = json
+ except Exception as e:
+ errors.append((idx, e))
+
+ threads = [
+ threading.Thread(target=access_json, args=(i,))
+ for i in range(num_threads)
+ ]
+
+ for t in threads:
+ t.start()
+ for t in threads:
+ t.join()
+
+ assert not errors, f"Errors: {errors}"
+ assert all(r is not None for r in results), "Some threads got None"
+ first = results[0]
+ assert all(r is first for r in results), "Inconsistent module objects"
+ assert not isinstance(first, types.LazyImportType), "Got lazy import instead of module"
+ print("OK")
+ """)
+
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+ def test_concurrent_reification_with_module_attribute_access(self):
+ """Threads racing to reify and immediately access module attributes."""
+ code = textwrap.dedent("""
+ import sys
+ import threading
+
+ sys.set_lazy_imports("all")
+
+ lazy import collections
+ lazy import functools
+ lazy import itertools
+
+ num_threads = 12
+ results = {}
+ errors = []
+ barrier = threading.Barrier(num_threads)
+
+ def stress_lazy_imports(idx):
+ try:
+ barrier.wait()
+ for _ in range(50):
+ _ = collections.OrderedDict
+ _ = functools.partial
+ _ = itertools.chain
+ _ = collections.defaultdict
+ _ = functools.lru_cache
+ _ = itertools.islice
+ results[idx] = (
+ type(collections).__name__,
+ type(functools).__name__,
+ type(itertools).__name__,
+ )
+ except Exception as e:
+ errors.append((idx, e))
+
+ threads = [
+ threading.Thread(target=stress_lazy_imports, args=(i,))
+ for i in range(num_threads)
+ ]
+
+ for t in threads:
+ t.start()
+ for t in threads:
+ t.join()
+
+ assert not errors, f"Errors: {errors}"
+ for idx, types_tuple in results.items():
+ assert all(t == 'module' for t in types_tuple), f"Thread {idx}: {types_tuple}"
+ print("OK")
+ """)
+
+ result = subprocess.run(
+ [sys.executable, "-c", code],
+ capture_output=True,
+ text=True
+ )
+ self.assertEqual(result.returncode, 0, f"stdout: {result.stdout}, stderr: {result.stderr}")
+ self.assertIn("OK", result.stdout)
+
+
+class LazyImportDisTests(unittest.TestCase):
+ def test_lazy_import_dis(self):
+ """dis should properly show lazy import"""
+ code = compile("lazy import foo", "exec", "exec")
+ f = io.StringIO()
+ dis.dis(code, file=f)
+ self.assertIn("foo + lazy", f.getvalue())
+
+ def test_normal_import_dis(self):
+ """non lazy imports should just show the name"""
+ code = compile("import foo", "exec", "exec")
+ f = io.StringIO()
+ dis.dis(code, file=f)
+ for line in f.getvalue().split('\n'):
+ if "IMPORT_NAME" in line:
+ self.assertIn("(foo)", line)
+ break
+ else:
+ self.assertFail("IMPORT_NAME not found")
+
+
+@unittest.skipIf(_testcapi is None, 'need the _testcapi module')
+class LazyCApiTests(unittest.TestCase):
+ def tearDown(self):
+ sys.set_lazy_imports("normal")
+ sys.set_lazy_imports_filter(None)
+
+ def test_set_matches_sys(self):
+ self.assertEqual(_testcapi.PyImport_GetLazyImportsMode(), sys.get_lazy_imports())
+ for mode in ("normal", "all", "none"):
+ _testcapi.PyImport_SetLazyImportsMode(mode)
+ self.assertEqual(_testcapi.PyImport_GetLazyImportsMode(), sys.get_lazy_imports())
+
+ def test_filter_matches_sys(self):
+ self.assertEqual(_testcapi.PyImport_GetLazyImportsFilter(), sys.get_lazy_imports_filter())
+
+ def filter(*args):
+ pass
+
+ _testcapi.PyImport_SetLazyImportsFilter(filter)
+ self.assertEqual(_testcapi.PyImport_GetLazyImportsFilter(), sys.get_lazy_imports_filter())
+
+ def test_set_bad_filter(self):
+ self.assertRaises(ValueError, _testcapi.PyImport_SetLazyImportsFilter, 42)
+
+
+if __name__ == '__main__':
+ unittest.main()
("obj.list", [(".", "op")]),
("obj.match", [(".", "op")]),
("b. \\\n format", [(".", "op")]),
+ ("lazy", []),
+ ("lazy()", [('(', 'op'), (')', 'op')]),
# highlights
("set", [("set", "builtin")]),
("list", [("list", "builtin")]),
(" \n dict", [("dict", "builtin")]),
+ (
+ " lazy import",
+ [("lazy", "soft_keyword"), ("import", "keyword")],
+ ),
+ (
+ "lazy from cool_people import pablo",
+ [
+ ("lazy", "soft_keyword"),
+ ("from", "keyword"),
+ ("import", "keyword"),
+ ],
+ ),
+ (
+ "if sad: lazy import happy",
+ [
+ ("if", "keyword"),
+ (":", "op"),
+ ("lazy", "soft_keyword"),
+ ("import", "keyword"),
+ ],
+ ),
+ (
+ "pass; lazy import z",
+ [
+ ("pass", "keyword"),
+ (";", "op"),
+ ("lazy", "soft_keyword"),
+ ("import", "keyword"),
+ ],
+ ),
]
for code, expected_highlights in cases:
with self.subTest(code=code):
]:
self._check_error(f"x = {lhs_stmt} if 1 else {rhs_stmt}", msg)
+
+class LazyImportRestrictionTestCase(SyntaxErrorTestCase):
+ """Test syntax restrictions for lazy imports."""
+
+ def test_lazy_import_in_try_block(self):
+ """Test that lazy imports are not allowed inside try blocks."""
+ self._check_error("""\
+try:
+ lazy import os
+except:
+ pass
+""", "lazy import not allowed inside try/except blocks")
+
+ self._check_error("""\
+try:
+ lazy from sys import path
+except ImportError:
+ pass
+""", "lazy from ... import not allowed inside try/except blocks")
+
+ def test_lazy_import_in_trystar_block(self):
+ """Test that lazy imports are not allowed inside try* blocks."""
+ self._check_error("""\
+try:
+ lazy import json
+except* Exception:
+ pass
+""", "lazy import not allowed inside try/except blocks")
+
+ self._check_error("""\
+try:
+ lazy from collections import defaultdict
+except* ImportError:
+ pass
+""", "lazy from ... import not allowed inside try/except blocks")
+
+ def test_lazy_import_in_except_block(self):
+ """Test that lazy imports are not allowed inside except blocks."""
+ self._check_error("""\
+try:
+ sys.modules # trigger the except block
+except* Exception:
+ lazy import sys
+""", "lazy import not allowed inside try/except blocks")
+
+ def test_lazy_import_in_function(self):
+ """Test that lazy imports are not allowed inside functions."""
+ self._check_error("""\
+def func():
+ lazy import math
+""", "lazy import not allowed inside functions")
+
+ self._check_error("""\
+def func():
+ lazy from datetime import datetime
+""", "lazy from ... import not allowed inside functions")
+
+ def test_lazy_import_in_async_function(self):
+ """Test that lazy imports are not allowed inside async functions."""
+ self._check_error("""\
+async def async_func():
+ lazy import asyncio
+""", "lazy import not allowed inside functions")
+
+ self._check_error("""\
+async def async_func():
+ lazy from json import loads
+""", "lazy from ... import not allowed inside functions")
+
+ def test_lazy_import_in_class(self):
+ """Test that lazy imports are not allowed inside classes."""
+ self._check_error("""\
+class MyClass:
+ lazy import typing
+""", "lazy import not allowed inside classes")
+
+ self._check_error("""\
+class MyClass:
+ lazy from abc import ABC
+""", "lazy from ... import not allowed inside classes")
+
+ def test_lazy_import_star_forbidden(self):
+ """Test that 'lazy from ... import *' is forbidden everywhere."""
+ # At module level should also be forbidden
+ self._check_error("lazy from os import *",
+ "lazy from ... import \\* is not allowed")
+
+ # Inside function should give lazy function error first
+ self._check_error("""\
+def func():
+ lazy from sys import *
+""", "lazy from ... import not allowed inside functions")
+
+ def test_lazy_import_nested_scopes(self):
+ """Test lazy imports in nested scopes."""
+ self._check_error("""\
+class Outer:
+ def method(self):
+ lazy import sys
+""", "lazy import not allowed inside functions")
+
+ self._check_error("""\
+def outer():
+ class Inner:
+ lazy import json
+""", "lazy import not allowed inside classes")
+
+ self._check_error("""\
+def outer():
+ def inner():
+ lazy from collections import deque
+""", "lazy from ... import not allowed inside functions")
+
+ def test_lazy_import_valid_cases(self):
+ """Test that lazy imports work at module level."""
+ # These should compile without errors
+ compile("lazy import os", "<test>", "exec")
+ compile("lazy from sys import path", "<test>", "exec")
+ compile("lazy import json as j", "<test>", "exec")
+ compile("lazy from datetime import datetime as dt", "<test>", "exec")
+
+
def load_tests(loader, tests, pattern):
tests.addTest(doctest.DocTestSuite())
return tests
"dont_write_bytecode", "no_user_site", "no_site",
"ignore_environment", "verbose", "bytes_warning", "quiet",
"hash_randomization", "isolated", "dev_mode", "utf8_mode",
- "warn_default_encoding", "safe_path", "int_max_str_digits")
+ "warn_default_encoding", "safe_path", "int_max_str_digits",
+ "lazy_imports")
for attr in attrs:
self.assertHasAttr(sys.flags, attr)
attr_type = bool if attr in ("dev_mode", "safe_path") else int
]
self.assertEqual(actual, expected(**colors))
+
+class TestLazyImportSuggestions(unittest.TestCase):
+ """Test that lazy imports are not reified when computing AttributeError suggestions."""
+
+ def test_attribute_error_does_not_reify_lazy_imports(self):
+ """Printing an AttributeError should not trigger lazy import reification."""
+ # pkg.bar prints "BAR_MODULE_LOADED" when imported.
+ # If lazy import is reified during suggestion computation, we'll see it.
+ code = textwrap.dedent("""
+ lazy import test.test_import.data.lazy_imports.pkg.bar
+ test.test_import.data.lazy_imports.pkg.nonexistent
+ """)
+ rc, stdout, stderr = assert_python_failure('-c', code)
+ self.assertNotIn(b"BAR_MODULE_LOADED", stdout)
+
+ def test_traceback_formatting_does_not_reify_lazy_imports(self):
+ """Formatting a traceback should not trigger lazy import reification."""
+ code = textwrap.dedent("""
+ import traceback
+ lazy import test.test_import.data.lazy_imports.pkg.bar
+ try:
+ test.test_import.data.lazy_imports.pkg.nonexistent
+ except AttributeError:
+ traceback.format_exc()
+ print("OK")
+ """)
+ rc, stdout, stderr = assert_python_ok('-c', code)
+ self.assertIn(b"OK", stdout)
+ self.assertNotIn(b"BAR_MODULE_LOADED", stdout)
+
+ def test_suggestion_still_works_for_non_lazy_attributes(self):
+ """Suggestions should still work for non-lazy module attributes."""
+ code = textwrap.dedent("""
+ lazy import test.test_import.data.lazy_imports.pkg.bar
+ # Typo for __name__
+ test.test_import.data.lazy_imports.pkg.__nme__
+ """)
+ rc, stdout, stderr = assert_python_failure('-c', code)
+ self.assertIn(b"__name__", stderr)
+ self.assertNotIn(b"BAR_MODULE_LOADED", stdout)
+
+
if __name__ == "__main__":
unittest.main()
class TypesTests(unittest.TestCase):
def test_names(self):
- c_only_names = {'CapsuleType'}
+ c_only_names = {'CapsuleType', 'LazyImportType'}
ignored = {'new_class', 'resolve_bases', 'prepare_class',
'get_original_bases', 'DynamicClassAttribute', 'coroutine'}
'CoroutineType', 'EllipsisType', 'FrameType', 'FunctionType',
'FrameLocalsProxyType',
'GeneratorType', 'GenericAlias', 'GetSetDescriptorType',
- 'LambdaType', 'MappingProxyType', 'MemberDescriptorType',
- 'MethodDescriptorType', 'MethodType', 'MethodWrapperType',
- 'ModuleType', 'NoneType', 'NotImplementedType', 'SimpleNamespace',
- 'TracebackType', 'UnionType', 'WrapperDescriptorType',
+ 'LambdaType', 'LazyImportType', 'MappingProxyType',
+ 'MemberDescriptorType', 'MethodDescriptorType', 'MethodType',
+ 'MethodWrapperType', 'ModuleType', 'NoneType',
+ 'NotImplementedType', 'SimpleNamespace', 'TracebackType',
+ 'UnionType', 'WrapperDescriptorType',
}
self.assertEqual(all_names, set(c_types.__all__))
self.assertEqual(all_names - c_only_names, set(py_types.__all__))
import linecache
import sys
import textwrap
+import types
import warnings
import codeop
import keyword
return _MOVE_COST
+def _is_lazy_import(obj, attr_name):
+ """Check if attr_name in obj's __dict__ is a lazy import.
+
+ Returns True if obj is a module and the attribute is a LazyImportType,
+ False otherwise. This avoids triggering module loading when computing
+ suggestions for AttributeError.
+ """
+ if not isinstance(obj, types.ModuleType):
+ return False
+ obj_dict = getattr(obj, '__dict__', None)
+ if obj_dict is None:
+ return False
+ attr_value = obj_dict.get(attr_name)
+ return isinstance(attr_value, types.LazyImportType)
+
+
def _check_for_nested_attribute(obj, wrong_name, attrs):
"""Check if any attribute of obj has the wrong_name as a nested attribute.
Returns the first nested attribute suggestion found, or None.
Limited to checking 20 attributes.
Only considers non-descriptor attributes to avoid executing arbitrary code.
+ Skips lazy imports to avoid triggering module loading.
"""
# Check for nested attributes (only one level deep)
attrs_to_check = [x for x in attrs if not x.startswith('_')][:20] # Limit number of attributes to check
if attr_from_class is not None and hasattr(attr_from_class, '__get__'):
continue # Skip descriptors to avoid executing arbitrary code
+ # Skip lazy imports to avoid triggering module loading
+ if _is_lazy_import(obj, attr_name):
+ continue
+
# Safe to get the attribute since it's not a descriptor
attr_obj = getattr(obj, attr_name)
except TypeError: # Attributes are unsortable, e.g. int and str
d = list(obj.__class__.__dict__.keys()) + list(obj.__dict__.keys())
d = sorted([x for x in d if isinstance(x, str)])
+ # Filter out lazy imports to avoid triggering module loading
+ d = [x for x in d if not _is_lazy_import(obj, x)]
hide_underscored = (wrong_name[:1] != '_')
if hide_underscored and tb is not None:
while tb.tb_next is not None:
except TypeError: # Attributes are unsortable, e.g. int and str
d = list(mod.__dict__.keys())
d = sorted([x for x in d if isinstance(x, str)])
+ # Filter out lazy imports to avoid triggering module loading
+ d = [x for x in d if not _is_lazy_import(mod, x)]
if wrong_name[:1] != '_':
d = [x for x in d if x[:1] != '_']
except Exception:
# CapsuleType cannot be accessed from pure Python,
# so there is no fallback definition.
+ # LazyImportType in pure Python cannot be guaranteed
+ # without overriding the filter, so there is no fallback.
+
del sys, _f, _g, _C, _c, _ag, _cell_factory # Not for export
Objects/funcobject.o \
Objects/interpolationobject.o \
Objects/iterobject.o \
+ Objects/lazyimportobject.o \
Objects/listobject.o \
Objects/longobject.o \
Objects/dictobject.o \
$(srcdir)/Include/internal/pycore_interpolation.h \
$(srcdir)/Include/internal/pycore_intrinsics.h \
$(srcdir)/Include/internal/pycore_jit.h \
+ $(srcdir)/Include/internal/pycore_lazyimportobject.h \
$(srcdir)/Include/internal/pycore_list.h \
$(srcdir)/Include/internal/pycore_llist.h \
$(srcdir)/Include/internal/pycore_lock.h \
test/test_import/data/package3 \
test/test_import/data/package4 \
test/test_import/data/unwritable \
+ test/test_import/data/lazy_imports \
+ test/test_import/data/lazy_imports/pkg \
test/test_importlib \
test/test_importlib/builtin \
test/test_importlib/extension \
--- /dev/null
+Implement :pep:`810`. Patch by Pablo Galindo and Dino Viehland.
}
+static PyObject *
+pyimport_setlazyimportsmode(PyObject *self, PyObject *args)
+{
+ PyObject *mode;
+ if (!PyArg_ParseTuple(args, "U", &mode)) {
+ return NULL;
+ }
+ if (strcmp(PyUnicode_AsUTF8(mode), "normal") == 0) {
+ PyImport_SetLazyImportsMode(PyImport_LAZY_NORMAL);
+ } else if (strcmp(PyUnicode_AsUTF8(mode), "all") == 0) {
+ PyImport_SetLazyImportsMode(PyImport_LAZY_ALL);
+ } else if (strcmp(PyUnicode_AsUTF8(mode), "none") == 0) {
+ PyImport_SetLazyImportsMode(PyImport_LAZY_NONE);
+ } else {
+ PyErr_SetString(PyExc_ValueError, "invalid mode");
+ return NULL;
+ }
+
+ Py_RETURN_NONE;
+}
+
+static PyObject *
+pyimport_getlazyimportsmode(PyObject *self, PyObject *args)
+{
+ switch (PyImport_GetLazyImportsMode()) {
+ case PyImport_LAZY_NORMAL:
+ return PyUnicode_FromString("normal");
+ case PyImport_LAZY_ALL:
+ return PyUnicode_FromString("all");
+ case PyImport_LAZY_NONE:
+ return PyUnicode_FromString("none");
+ default:
+ PyErr_SetString(PyExc_ValueError, "unknown mode");
+ return NULL;
+ }
+}
+
+static PyObject *
+pyimport_setlazyimportsfilter(PyObject *self, PyObject *args)
+{
+ PyObject *filter;
+ if (!PyArg_ParseTuple(args, "O", &filter)) {
+ return NULL;
+ }
+
+ if (PyImport_SetLazyImportsFilter(filter) < 0) {
+ return NULL;
+ }
+
+ Py_RETURN_NONE;
+}
+
+static PyObject *
+pyimport_getlazyimportsfilter(PyObject *self, PyObject *args)
+{
+ PyObject *res = PyImport_GetLazyImportsFilter();
+ if (res == NULL) {
+ Py_RETURN_NONE;
+ }
+ return res;
+}
+
static PyMethodDef test_methods[] = {
{"PyImport_ImportModuleAttr", pyimport_importmoduleattr, METH_VARARGS},
{"PyImport_ImportModuleAttrString", pyimport_importmoduleattrstring, METH_VARARGS},
+ {"PyImport_SetLazyImportsMode", pyimport_setlazyimportsmode, METH_VARARGS},
+ {"PyImport_GetLazyImportsMode", pyimport_getlazyimportsmode, METH_NOARGS},
+ {"PyImport_SetLazyImportsFilter", pyimport_setlazyimportsfilter, METH_VARARGS},
+ {"PyImport_GetLazyImportsFilter", pyimport_getlazyimportsfilter, METH_NOARGS},
{NULL},
};
{
return PyModule_AddFunctions(m, test_methods);
}
-
_PyStackRef res;
from = stack_pointer[-1];
PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
- _PyFrame_SetStackPointer(frame, stack_pointer);
- PyObject *res_o = _PyEval_ImportFrom(tstate, PyStackRef_AsPyObjectBorrow(from), name);
- stack_pointer = _PyFrame_GetStackPointer(frame);
+ PyObject *res_o;
+ if (PyLazyImport_CheckExact(PyStackRef_AsPyObjectBorrow(from))) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_LazyImportFrom(
+ tstate, frame, PyStackRef_AsPyObjectBorrow(from), name);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
+ else {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_ImportFrom(
+ tstate, PyStackRef_AsPyObjectBorrow(from), name);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
if (res_o == NULL) {
JUMP_TO_LABEL(error);
}
_PyStackRef res;
fromlist = stack_pointer[-1];
level = stack_pointer[-2];
- PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2);
+ PyObject *res_o;
+ if (!(oparg & 0x02)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_LazyImportName(tstate, BUILTINS(), GLOBALS(),
+ LOCALS(), name,
+ PyStackRef_AsPyObjectBorrow(fromlist),
+ PyStackRef_AsPyObjectBorrow(level),
+ oparg & 0x01);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
+ else {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_ImportName(tstate, BUILTINS(), GLOBALS(),
+ LOCALS(), name,
+ PyStackRef_AsPyObjectBorrow(fromlist),
+ PyStackRef_AsPyObjectBorrow(level));
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
_PyFrame_SetStackPointer(frame, stack_pointer);
- PyObject *res_o = _PyEval_ImportName(tstate, frame, name,
- PyStackRef_AsPyObjectBorrow(fromlist),
- PyStackRef_AsPyObjectBorrow(level));
_PyStackRef tmp = fromlist;
fromlist = PyStackRef_NULL;
stack_pointer[-1] = fromlist;
}
JUMP_TO_LABEL(error);
}
+ if (PyLazyImport_CheckExact(v_o)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ Py_SETREF(v_o, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (v_o == NULL) {
+ JUMP_TO_LABEL(error);
+ }
+ }
}
else {
_PyFrame_SetStackPointer(frame, stack_pointer);
JUMP_TO_LABEL(error);
}
}
+ if (PyLazyImport_CheckExact(v_o)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ Py_SETREF(v_o, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (v_o == NULL) {
+ JUMP_TO_LABEL(error);
+ }
+ }
}
}
v = PyStackRef_FromPyObjectSteal(v_o);
if (v_o == NULL) {
JUMP_TO_LABEL(error);
}
+ if (PyLazyImport_CheckExact(v_o)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (l_v == NULL) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_DECREF(v_o);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ JUMP_TO_LABEL(error);
+ }
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ int err = PyDict_SetItem(GLOBALS(), name, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (err < 0) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_DECREF(v_o);
+ Py_DECREF(l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ JUMP_TO_LABEL(error);
+ }
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_SETREF(v_o, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
v = PyStackRef_FromPyObjectSteal(v_o);
stack_pointer[0] = v;
stack_pointer += 1;
#include "Python.h"
#include "pycore_descrobject.h" // _PyMethodWrapper_Type
+#include "pycore_lazyimportobject.h" // PyLazyImport_Type
#include "pycore_namespace.h" // _PyNamespace_Type
#include "pycore_object.h" // _PyNone_Type, _PyNotImplemented_Type
#include "pycore_unionobject.h" // _PyUnion_Type
EXPORT_STATIC_TYPE("GetSetDescriptorType", PyGetSetDescr_Type);
// LambdaType is the same as FunctionType
EXPORT_STATIC_TYPE("LambdaType", PyFunction_Type);
+ EXPORT_STATIC_TYPE("LazyImportType", PyLazyImport_Type);
EXPORT_STATIC_TYPE("MappingProxyType", PyDictProxy_Type);
EXPORT_STATIC_TYPE("MemberDescriptorType", PyMemberDescr_Type);
EXPORT_STATIC_TYPE("MethodDescriptorType", PyMethodDescr_Type);
return (Py_ssize_t)res;
}
+void
+_PyDict_ClearKeysVersionLockHeld(PyObject *mp)
+{
+ ASSERT_DICT_LOCKED(mp);
+
+ FT_ATOMIC_STORE_UINT32_RELAXED(((PyDictObject *)mp)->ma_keys->dk_version, 0);
+}
+
Py_ssize_t
_PyDict_SizeOf(PyDictObject *mp)
{
{
PyInterpreterState *interp = _PyInterpreterState_GET();
- /* Start at 2, as 0 and 1 are reserved for CPython */
- for (int i = 2; i < DICT_MAX_WATCHERS; i++) {
+ /* Some watchers are reserved for CPython, start at the first available one */
+ for (int i = FIRST_AVAILABLE_WATCHER; i < DICT_MAX_WATCHERS; i++) {
if (!interp->dict_state.watchers[i]) {
interp->dict_state.watchers[i] = callback;
return i;
};
PyObject *PyExc_ImportError = (PyObject *)&_PyExc_ImportError;
+/*
+ * ImportCycleError extends ImportError
+ */
+
+MiddlingExtendsException(PyExc_ImportError, ImportCycleError, ImportError,
+ "Import produces a cycle.");
/*
* ModuleNotFoundError extends ImportError
*/
{&_PyExc_IncompleteInputError, "_IncompleteInputError"}, // base: SyntaxError(Exception)
ITEM(IndexError), // base: LookupError(Exception)
ITEM(KeyError), // base: LookupError(Exception)
+ ITEM(ImportCycleError), // base: ImportError(Exception)
ITEM(ModuleNotFoundError), // base: ImportError(Exception)
ITEM(NotImplementedError), // base: RuntimeError(Exception)
ITEM(PythonFinalizationError), // base: RuntimeError(Exception)
Py_XDECREF(r);
return res;
}
-
--- /dev/null
+// Lazy object implementation.
+
+#include "Python.h"
+#include "pycore_ceval.h"
+#include "pycore_frame.h"
+#include "pycore_import.h"
+#include "pycore_interpframe.h"
+#include "pycore_lazyimportobject.h"
+#include "pycore_modsupport.h"
+
+#define PyLazyImportObject_CAST(op) ((PyLazyImportObject *)(op))
+
+PyObject *
+_PyLazyImport_New(_PyInterpreterFrame *frame, PyObject *builtins, PyObject *name, PyObject *fromlist)
+{
+ PyLazyImportObject *m;
+ if (!name || !PyUnicode_Check(name)) {
+ PyErr_SetString(PyExc_TypeError, "expected str for name");
+ return NULL;
+ }
+ if (fromlist == Py_None || fromlist == NULL) {
+ fromlist = NULL;
+ }
+ else if (!PyUnicode_Check(fromlist) && !PyTuple_Check(fromlist)) {
+ PyErr_SetString(PyExc_TypeError,
+ "lazy_import: fromlist must be None, a string, or a tuple");
+ return NULL;
+ }
+ m = PyObject_GC_New(PyLazyImportObject, &PyLazyImport_Type);
+ if (m == NULL) {
+ return NULL;
+ }
+ m->lz_builtins = Py_XNewRef(builtins);
+ m->lz_from = Py_NewRef(name);
+ m->lz_attr = Py_XNewRef(fromlist);
+
+ // Capture frame information for the original import location.
+ m->lz_code = NULL;
+ m->lz_instr_offset = -1;
+
+ if (frame != NULL) {
+ PyCodeObject *code = _PyFrame_GetCode(frame);
+ if (code != NULL) {
+ m->lz_code = (PyCodeObject *)Py_NewRef(code);
+ // Calculate the instruction offset from the current frame.
+ m->lz_instr_offset = _PyInterpreterFrame_LASTI(frame);
+ }
+ }
+
+ _PyObject_GC_TRACK(m);
+ return (PyObject *)m;
+}
+
+static int
+lazy_import_traverse(PyObject *op, visitproc visit, void *arg)
+{
+ PyLazyImportObject *m = PyLazyImportObject_CAST(op);
+ Py_VISIT(m->lz_builtins);
+ Py_VISIT(m->lz_from);
+ Py_VISIT(m->lz_attr);
+ Py_VISIT(m->lz_code);
+ return 0;
+}
+
+static int
+lazy_import_clear(PyObject *op)
+{
+ PyLazyImportObject *m = PyLazyImportObject_CAST(op);
+ Py_CLEAR(m->lz_builtins);
+ Py_CLEAR(m->lz_from);
+ Py_CLEAR(m->lz_attr);
+ Py_CLEAR(m->lz_code);
+ return 0;
+}
+
+static void
+lazy_import_dealloc(PyObject *op)
+{
+ _PyObject_GC_UNTRACK(op);
+ (void)lazy_import_clear(op);
+ Py_TYPE(op)->tp_free(op);
+}
+
+static PyObject *
+lazy_import_name(PyLazyImportObject *m)
+{
+ if (m->lz_attr != NULL) {
+ if (PyUnicode_Check(m->lz_attr)) {
+ return PyUnicode_FromFormat("%U.%U", m->lz_from, m->lz_attr);
+ }
+ else {
+ return PyUnicode_FromFormat("%U...", m->lz_from);
+ }
+ }
+ return Py_NewRef(m->lz_from);
+}
+
+static PyObject *
+lazy_import_repr(PyObject *op)
+{
+ PyLazyImportObject *m = PyLazyImportObject_CAST(op);
+ PyObject *name = lazy_import_name(m);
+ if (name == NULL) {
+ return NULL;
+ }
+ PyObject *res = PyUnicode_FromFormat("<%T '%U'>", op, name);
+ Py_DECREF(name);
+ return res;
+}
+
+PyObject *
+_PyLazyImport_GetName(PyObject *op)
+{
+ PyLazyImportObject *lazy_import = PyLazyImportObject_CAST(op);
+ assert(PyLazyImport_CheckExact(lazy_import));
+ return lazy_import_name(lazy_import);
+}
+
+static PyObject *
+lazy_import_resolve(PyObject *self, PyObject *args)
+{
+ return _PyImport_LoadLazyImportTstate(PyThreadState_GET(), self);
+}
+
+static PyMethodDef lazy_import_methods[] = {
+ {
+ "resolve", lazy_import_resolve, METH_NOARGS,
+ PyDoc_STR("resolves the lazy import and returns the actual object")
+ },
+ {NULL, NULL}
+};
+
+
+PyDoc_STRVAR(lazy_import_doc,
+"lazy_import(builtins, name, fromlist=None, /)\n"
+"--\n"
+"\n"
+"Represents a deferred import that will be resolved on first use.\n"
+"\n"
+"Instances of this object accessed from the global scope will be\n"
+"automatically imported based upon their name and then replaced with\n"
+"the imported value.");
+
+PyTypeObject PyLazyImport_Type = {
+ PyVarObject_HEAD_INIT(&PyType_Type, 0)
+ .tp_name = "lazy_import",
+ .tp_basicsize = sizeof(PyLazyImportObject),
+ .tp_dealloc = lazy_import_dealloc,
+ .tp_repr = lazy_import_repr,
+ .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,
+ .tp_doc = lazy_import_doc,
+ .tp_traverse = lazy_import_traverse,
+ .tp_clear = lazy_import_clear,
+ .tp_methods = lazy_import_methods,
+ .tp_alloc = PyType_GenericAlloc,
+ .tp_free = PyObject_GC_Del,
+};
#include "pycore_fileutils.h" // _Py_wgetcwd
#include "pycore_import.h" // _PyImport_GetNextModuleIndex()
#include "pycore_interp.h" // PyInterpreterState.importlib
+#include "pycore_lazyimportobject.h" // _PyLazyImportObject_Check()
#include "pycore_long.h" // _PyLong_GetOne()
#include "pycore_modsupport.h" // _PyModule_CreateInitialized()
#include "pycore_moduleobject.h" // _PyModule_GetDefOrNull()
#include "osdefs.h" // MAXPATHLEN
-
#define _PyModule_CAST(op) \
(assert(PyModule_Check(op)), _Py_CAST(PyModuleObject*, (op)))
return m;
}
+/* Module dict watcher callback.
+ * When a module dictionary is modified, we need to clear the keys version
+ * to invalidate any cached lookups that depend on the dictionary structure.
+ */
+static int
+module_dict_watcher(PyDict_WatchEvent event, PyObject *dict,
+ PyObject *key, PyObject *new_value)
+{
+ assert(PyDict_Check(dict));
+ // Only if a new lazy object shows up do we need to clear the dictionary. If
+ // this is adding a new key then the version will be reset anyway.
+ if (event == PyDict_EVENT_MODIFIED &&
+ new_value != NULL &&
+ PyLazyImport_CheckExact(new_value)) {
+ _PyDict_ClearKeysVersionLockHeld(dict);
+ }
+ return 0;
+}
+
+int
+_PyModule_InitModuleDictWatcher(PyInterpreterState *interp)
+{
+ // This is a reserved watcher for CPython so there's no need to check for non-NULL.
+ assert(interp->dict_state.watchers[MODULE_WATCHER_ID] == NULL);
+ interp->dict_state.watchers[MODULE_WATCHER_ID] = &module_dict_watcher;
+ return 0;
+}
+
static void
track_module(PyModuleObject *m)
{
_PyDict_EnablePerThreadRefcounting(m->md_dict);
_PyObject_SetDeferredRefcount((PyObject *)m);
+ PyDict_Watch(MODULE_WATCHER_ID, m->md_dict);
PyObject_GC_Track(m);
}
PyObject *attr, *mod_name, *getattr;
attr = _PyObject_GenericGetAttrWithDict((PyObject *)m, name, NULL, suppress);
if (attr) {
+ if (PyLazyImport_CheckExact(attr)) {
+ PyObject *new_value = _PyImport_LoadLazyImportTstate(
+ PyThreadState_GET(), attr);
+ if (new_value == NULL) {
+ if (suppress &&
+ PyErr_ExceptionMatches(PyExc_ImportCycleError)) {
+ // ImportCycleError is raised when a lazy object tries
+ // to import itself. In this case, the error should not
+ // propagate to the caller and instead treated as if the
+ // attribute doesn't exist.
+ PyErr_Clear();
+ }
+ Py_DECREF(attr);
+ return NULL;
+ }
+
+ if (PyDict_SetItem(m->md_dict, name, new_value) < 0) {
+ Py_CLEAR(new_value);
+ }
+ Py_DECREF(attr);
+ return new_value;
+ }
return attr;
}
if (suppress == 1) {
module_dir(PyObject *self, PyObject *args)
{
PyObject *result = NULL;
- PyObject *dict = PyObject_GetAttr(self, &_Py_ID(__dict__));
+ PyObject *dict;
+ if (PyModule_CheckExact(self)) {
+ dict = Py_NewRef(((PyModuleObject *)self)->md_dict);
+ } else {
+ dict = PyObject_GetAttr(self, &_Py_ID(__dict__));
+ }
if (dict != NULL) {
if (PyDict_Check(dict)) {
};
static PyObject *
-module_get_dict(PyModuleObject *m)
+module_load_dict(PyModuleObject *m)
{
PyObject *dict = PyObject_GetAttr((PyObject *)m, &_Py_ID(__dict__));
if (dict == NULL) {
{
PyModuleObject *m = _PyModule_CAST(self);
- PyObject *dict = module_get_dict(m);
+ PyObject *dict = module_load_dict(m);
if (dict == NULL) {
return NULL;
}
return -1;
}
- PyObject *dict = module_get_dict(m);
+ PyObject *dict = module_load_dict(m);
if (dict == NULL) {
return -1;
}
{
PyModuleObject *m = _PyModule_CAST(self);
- PyObject *dict = module_get_dict(m);
+ PyObject *dict = module_load_dict(m);
if (dict == NULL) {
return NULL;
}
{
PyModuleObject *m = _PyModule_CAST(self);
- PyObject *dict = module_get_dict(m);
+ PyObject *dict = module_load_dict(m);
if (dict == NULL) {
return -1;
}
return ret;
}
-
static PyGetSetDef module_getsets[] = {
{"__annotations__", module_get_annotations, module_set_annotations},
{"__annotate__", module_get_annotate, module_set_annotate},
<ClCompile Include="..\Objects\genobject.c" />
<ClCompile Include="..\Objects\interpolationobject.c" />
<ClCompile Include="..\Objects\iterobject.c" />
+ <ClCompile Include="..\Objects\lazyimportobject.c" />
<ClCompile Include="..\Objects\listobject.c" />
<ClCompile Include="..\Objects\longobject.c" />
<ClCompile Include="..\Objects\memoryobject.c" />
<ClCompile Include="..\Python\jit.c">
<Filter>Source Files</Filter>
</ClCompile>
+ <ClCompile Include="..\Objects\lazyimportobject.c">
+ <Filter>Source Files</Filter>
+ </ClCompile>
<ClCompile Include="..\Objects\listobject.c">
<Filter>Source Files</Filter>
</ClCompile>
<ClInclude Include="..\Include\internal\pycore_interpolation.h" />
<ClInclude Include="..\Include\internal\pycore_intrinsics.h" />
<ClInclude Include="..\Include\internal\pycore_jit.h" />
+ <ClInclude Include="..\Include\internal\pycore_lazyimportobject.h" />
<ClInclude Include="..\Include\internal\pycore_list.h" />
<ClInclude Include="..\Include\internal\pycore_llist.h" />
<ClInclude Include="..\Include\internal\pycore_lock.h" />
<ClCompile Include="..\Objects\longobject.c" />
<ClCompile Include="..\Objects\memoryobject.c" />
<ClCompile Include="..\Objects\methodobject.c" />
+ <ClCompile Include="..\Objects\lazyimportobject.c" />
<ClCompile Include="..\Objects\moduleobject.c" />
<ClCompile Include="..\Objects\namespaceobject.c" />
<ClCompile Include="..\Objects\object.c" />
<ClInclude Include="..\Include\internal\pycore_long.h">
<Filter>Include\internal</Filter>
</ClInclude>
+ <ClInclude Include="..\Include\internal\pycore_lazyimportobject.h">
+ <Filter>Include\internal</Filter>
+ </ClInclude>
<ClInclude Include="..\Include\internal\pycore_mmap.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClCompile Include="..\Objects\methodobject.c">
<Filter>Objects</Filter>
</ClCompile>
+ <ClCompile Include="..\Objects\lazyimportobject.c">
+ <Filter>Objects</Filter>
+ </ClCompile>
<ClCompile Include="..\Objects\moduleobject.c">
<Filter>Objects</Filter>
</ClCompile>
| TryStar(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody)
| Assert(expr test, expr? msg)
- | Import(alias* names)
- | ImportFrom(identifier? module, alias* names, int? level)
+ | Import(alias* names, int? is_lazy)
+ | ImportFrom(identifier? module, alias* names, int? level, int? is_lazy)
| Global(identifier* names)
| Nonlocal(identifier* names)
}
stmt_ty
-_PyPegen_checked_future_import(Parser *p, identifier module, asdl_alias_seq * names, int level,
- int lineno, int col_offset, int end_lineno, int end_col_offset,
- PyArena *arena) {
+_PyPegen_checked_future_import(Parser *p, identifier module, asdl_alias_seq * names,
+ int level, expr_ty lazy_token, int lineno,
+ int col_offset, int end_lineno, int end_col_offset,
+ PyArena *arena) {
if (level == 0 && PyUnicode_CompareWithASCIIString(module, "__future__") == 0) {
+ if (lazy_token) {
+ RAISE_SYNTAX_ERROR_KNOWN_LOCATION(lazy_token,
+ "lazy from __future__ import is not allowed");
+ return NULL;
+ }
for (Py_ssize_t i = 0; i < asdl_seq_LEN(names); i++) {
alias_ty alias = asdl_seq_GET(names, i);
if (PyUnicode_CompareWithASCIIString(alias->name, "barry_as_FLUFL") == 0) {
}
}
}
- return _PyAST_ImportFrom(module, names, level, lineno, col_offset, end_lineno, end_col_offset, arena);
+ return _PyAST_ImportFrom(module, names, level, lazy_token ? 1 : 0, lineno,
+ col_offset, end_lineno, end_col_offset, arena);
}
asdl_stmt_seq*
{NULL, -1},
},
(KeywordToken[]) {
- {"return", 522},
{"import", 647},
+ {"return", 522},
{"assert", 638},
{"global", 530},
{"except", 690},
static char *soft_keywords[] = {
"_",
"case",
+ "lazy",
"match",
"type",
NULL,
// simple_stmt:
// | assignment
// | &"type" type_alias
+// | &('import' | 'from' | "lazy") import_stmt
// | star_expressions
// | &'return' return_stmt
-// | &('import' | 'from') import_stmt
// | &'raise' raise_stmt
// | &'pass' pass_stmt
// | &'del' del_stmt
D(fprintf(stderr, "%*c%s simple_stmt[%d-%d]: %s failed!\n", p->level, ' ',
p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&\"type\" type_alias"));
}
+ { // &('import' | 'from' | "lazy") import_stmt
+ if (p->error_indicator) {
+ p->level--;
+ return NULL;
+ }
+ D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&('import' | 'from' | \"lazy\") import_stmt"));
+ stmt_ty import_stmt_var;
+ if (
+ _PyPegen_lookahead(1, _tmp_5_rule, p)
+ &&
+ (import_stmt_var = import_stmt_rule(p)) // import_stmt
+ )
+ {
+ D(fprintf(stderr, "%*c+ simple_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "&('import' | 'from' | \"lazy\") import_stmt"));
+ _res = import_stmt_var;
+ goto done;
+ }
+ p->mark = _mark;
+ D(fprintf(stderr, "%*c%s simple_stmt[%d-%d]: %s failed!\n", p->level, ' ',
+ p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('import' | 'from' | \"lazy\") import_stmt"));
+ }
{ // star_expressions
if (p->error_indicator) {
p->level--;
D(fprintf(stderr, "%*c%s simple_stmt[%d-%d]: %s failed!\n", p->level, ' ',
p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&'return' return_stmt"));
}
- { // &('import' | 'from') import_stmt
- if (p->error_indicator) {
- p->level--;
- return NULL;
- }
- D(fprintf(stderr, "%*c> simple_stmt[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "&('import' | 'from') import_stmt"));
- stmt_ty import_stmt_var;
- if (
- _PyPegen_lookahead(1, _tmp_5_rule, p)
- &&
- (import_stmt_var = import_stmt_rule(p)) // import_stmt
- )
- {
- D(fprintf(stderr, "%*c+ simple_stmt[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "&('import' | 'from') import_stmt"));
- _res = import_stmt_var;
- goto done;
- }
- p->mark = _mark;
- D(fprintf(stderr, "%*c%s simple_stmt[%d-%d]: %s failed!\n", p->level, ' ',
- p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "&('import' | 'from') import_stmt"));
- }
{ // &'raise' raise_stmt
if (p->error_indicator) {
p->level--;
return NULL;
}
stmt_ty _res = NULL;
+ if (_PyPegen_is_memoized(p, import_stmt_type, &_res)) {
+ p->level--;
+ return _res;
+ }
int _mark = p->mark;
if (p->call_invalid_rules) { // invalid_import
if (p->error_indicator) {
}
_res = NULL;
done:
+ _PyPegen_insert_memo(p, _mark, import_stmt_type, _res);
p->level--;
return _res;
}
-// import_name: 'import' dotted_as_names
+// import_name: "lazy"? 'import' dotted_as_names
static stmt_ty
import_name_rule(Parser *p)
{
UNUSED(_start_lineno); // Only used by EXTRA macro
int _start_col_offset = p->tokens[_mark]->col_offset;
UNUSED(_start_col_offset); // Only used by EXTRA macro
- { // 'import' dotted_as_names
+ { // "lazy"? 'import' dotted_as_names
if (p->error_indicator) {
p->level--;
return NULL;
}
- D(fprintf(stderr, "%*c> import_name[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'import' dotted_as_names"));
+ D(fprintf(stderr, "%*c> import_name[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "\"lazy\"? 'import' dotted_as_names"));
Token * _keyword;
asdl_alias_seq* a;
+ void *lazy;
if (
+ (lazy = _PyPegen_expect_soft_keyword(p, "lazy"), !p->error_indicator) // "lazy"?
+ &&
(_keyword = _PyPegen_expect_token(p, 647)) // token='import'
&&
(a = dotted_as_names_rule(p)) // dotted_as_names
)
{
- D(fprintf(stderr, "%*c+ import_name[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'import' dotted_as_names"));
+ D(fprintf(stderr, "%*c+ import_name[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "\"lazy\"? 'import' dotted_as_names"));
Token *_token = _PyPegen_get_last_nonnwhitespace_token(p);
if (_token == NULL) {
p->level--;
UNUSED(_end_lineno); // Only used by EXTRA macro
int _end_col_offset = _token->end_col_offset;
UNUSED(_end_col_offset); // Only used by EXTRA macro
- _res = _PyAST_Import ( a , EXTRA );
+ _res = _PyAST_Import ( a , lazy ? 1 : 0 , EXTRA );
if (_res == NULL && PyErr_Occurred()) {
p->error_indicator = 1;
p->level--;
}
p->mark = _mark;
D(fprintf(stderr, "%*c%s import_name[%d-%d]: %s failed!\n", p->level, ' ',
- p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'import' dotted_as_names"));
+ p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "\"lazy\"? 'import' dotted_as_names"));
}
_res = NULL;
done:
}
// import_from:
-// | 'from' (('.' | '...'))* dotted_name 'import' import_from_targets
-// | 'from' (('.' | '...'))+ 'import' import_from_targets
+// | "lazy"? 'from' (('.' | '...'))* dotted_name 'import' import_from_targets
+// | "lazy"? 'from' (('.' | '...'))+ 'import' import_from_targets
static stmt_ty
import_from_rule(Parser *p)
{
UNUSED(_start_lineno); // Only used by EXTRA macro
int _start_col_offset = p->tokens[_mark]->col_offset;
UNUSED(_start_col_offset); // Only used by EXTRA macro
- { // 'from' (('.' | '...'))* dotted_name 'import' import_from_targets
+ { // "lazy"? 'from' (('.' | '...'))* dotted_name 'import' import_from_targets
if (p->error_indicator) {
p->level--;
return NULL;
}
- D(fprintf(stderr, "%*c> import_from[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'from' (('.' | '...'))* dotted_name 'import' import_from_targets"));
+ D(fprintf(stderr, "%*c> import_from[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "\"lazy\"? 'from' (('.' | '...'))* dotted_name 'import' import_from_targets"));
Token * _keyword;
Token * _keyword_1;
asdl_seq * a;
expr_ty b;
asdl_alias_seq* c;
+ void *lazy;
if (
+ (lazy = _PyPegen_expect_soft_keyword(p, "lazy"), !p->error_indicator) // "lazy"?
+ &&
(_keyword = _PyPegen_expect_token(p, 646)) // token='from'
&&
(a = _loop0_17_rule(p)) // (('.' | '...'))*
(c = import_from_targets_rule(p)) // import_from_targets
)
{
- D(fprintf(stderr, "%*c+ import_from[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'from' (('.' | '...'))* dotted_name 'import' import_from_targets"));
+ D(fprintf(stderr, "%*c+ import_from[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "\"lazy\"? 'from' (('.' | '...'))* dotted_name 'import' import_from_targets"));
Token *_token = _PyPegen_get_last_nonnwhitespace_token(p);
if (_token == NULL) {
p->level--;
UNUSED(_end_lineno); // Only used by EXTRA macro
int _end_col_offset = _token->end_col_offset;
UNUSED(_end_col_offset); // Only used by EXTRA macro
- _res = _PyPegen_checked_future_import ( p , b -> v . Name . id , c , _PyPegen_seq_count_dots ( a ) , EXTRA );
+ _res = _PyPegen_checked_future_import ( p , b -> v . Name . id , c , _PyPegen_seq_count_dots ( a ) , lazy , EXTRA );
if (_res == NULL && PyErr_Occurred()) {
p->error_indicator = 1;
p->level--;
}
p->mark = _mark;
D(fprintf(stderr, "%*c%s import_from[%d-%d]: %s failed!\n", p->level, ' ',
- p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'from' (('.' | '...'))* dotted_name 'import' import_from_targets"));
+ p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "\"lazy\"? 'from' (('.' | '...'))* dotted_name 'import' import_from_targets"));
}
- { // 'from' (('.' | '...'))+ 'import' import_from_targets
+ { // "lazy"? 'from' (('.' | '...'))+ 'import' import_from_targets
if (p->error_indicator) {
p->level--;
return NULL;
}
- D(fprintf(stderr, "%*c> import_from[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "'from' (('.' | '...'))+ 'import' import_from_targets"));
+ D(fprintf(stderr, "%*c> import_from[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "\"lazy\"? 'from' (('.' | '...'))+ 'import' import_from_targets"));
Token * _keyword;
Token * _keyword_1;
asdl_seq * a;
asdl_alias_seq* b;
+ void *lazy;
if (
+ (lazy = _PyPegen_expect_soft_keyword(p, "lazy"), !p->error_indicator) // "lazy"?
+ &&
(_keyword = _PyPegen_expect_token(p, 646)) // token='from'
&&
(a = _loop1_18_rule(p)) // (('.' | '...'))+
(b = import_from_targets_rule(p)) // import_from_targets
)
{
- D(fprintf(stderr, "%*c+ import_from[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "'from' (('.' | '...'))+ 'import' import_from_targets"));
+ D(fprintf(stderr, "%*c+ import_from[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "\"lazy\"? 'from' (('.' | '...'))+ 'import' import_from_targets"));
Token *_token = _PyPegen_get_last_nonnwhitespace_token(p);
if (_token == NULL) {
p->level--;
UNUSED(_end_lineno); // Only used by EXTRA macro
int _end_col_offset = _token->end_col_offset;
UNUSED(_end_col_offset); // Only used by EXTRA macro
- _res = _PyAST_ImportFrom ( NULL , b , _PyPegen_seq_count_dots ( a ) , EXTRA );
+ _res = _PyAST_ImportFrom ( NULL , b , _PyPegen_seq_count_dots ( a ) , lazy ? 1 : 0 , EXTRA );
if (_res == NULL && PyErr_Occurred()) {
p->error_indicator = 1;
p->level--;
}
p->mark = _mark;
D(fprintf(stderr, "%*c%s import_from[%d-%d]: %s failed!\n", p->level, ' ',
- p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'from' (('.' | '...'))+ 'import' import_from_targets"));
+ p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "\"lazy\"? 'from' (('.' | '...'))+ 'import' import_from_targets"));
}
_res = NULL;
done:
return _res;
}
-// _tmp_5: 'import' | 'from'
+// _tmp_5: 'import' | 'from' | "lazy"
static void *
_tmp_5_rule(Parser *p)
{
D(fprintf(stderr, "%*c%s _tmp_5[%d-%d]: %s failed!\n", p->level, ' ',
p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "'from'"));
}
+ { // "lazy"
+ if (p->error_indicator) {
+ p->level--;
+ return NULL;
+ }
+ D(fprintf(stderr, "%*c> _tmp_5[%d-%d]: %s\n", p->level, ' ', _mark, p->mark, "\"lazy\""));
+ expr_ty _keyword;
+ if (
+ (_keyword = _PyPegen_expect_soft_keyword(p, "lazy")) // soft_keyword='"lazy"'
+ )
+ {
+ D(fprintf(stderr, "%*c+ _tmp_5[%d-%d]: %s succeeded!\n", p->level, ' ', _mark, p->mark, "\"lazy\""));
+ _res = _keyword;
+ goto done;
+ }
+ p->mark = _mark;
+ D(fprintf(stderr, "%*c%s _tmp_5[%d-%d]: %s failed!\n", p->level, ' ',
+ p->error_indicator ? "ERROR!" : "-", _mark, p->mark, "\"lazy\""));
+ }
_res = NULL;
done:
p->level--;
expr_ty _PyPegen_get_last_comprehension_item(comprehension_ty comprehension);
void *_PyPegen_nonparen_genexp_in_call(Parser *p, expr_ty args, asdl_comprehension_seq *comprehensions);
stmt_ty _PyPegen_checked_future_import(Parser *p, identifier module, asdl_alias_seq *,
- int , int, int , int , int , PyArena *);
+ int, expr_ty, int, int, int, int, PyArena *);
asdl_stmt_seq* _PyPegen_register_stmts(Parser *p, asdl_stmt_seq* stmts);
stmt_ty _PyPegen_register_stmt(Parser *p, stmt_ty s);
unsigned char M_test_frozenmain[] = {
227,0,0,0,0,0,0,0,0,0,0,0,0,9,0,0,
0,0,0,0,0,243,184,0,0,0,128,0,94,0,82,1,
- 73,0,116,0,94,0,82,1,73,1,116,1,93,2,33,0,
+ 73,0,116,0,94,0,82,1,73,4,116,1,93,2,33,0,
82,2,52,1,0,0,0,0,0,0,31,0,93,2,33,0,
82,3,93,0,80,6,0,0,0,0,0,0,0,0,0,0,
0,0,0,0,0,0,0,0,52,2,0,0,0,0,0,0,
Py_CLEAR(state->id);
Py_CLEAR(state->ifs);
Py_CLEAR(state->is_async);
+ Py_CLEAR(state->is_lazy);
Py_CLEAR(state->items);
Py_CLEAR(state->iter);
Py_CLEAR(state->key);
if ((state->id = PyUnicode_InternFromString("id")) == NULL) return -1;
if ((state->ifs = PyUnicode_InternFromString("ifs")) == NULL) return -1;
if ((state->is_async = PyUnicode_InternFromString("is_async")) == NULL) return -1;
+ if ((state->is_lazy = PyUnicode_InternFromString("is_lazy")) == NULL) return -1;
if ((state->items = PyUnicode_InternFromString("items")) == NULL) return -1;
if ((state->iter = PyUnicode_InternFromString("iter")) == NULL) return -1;
if ((state->key = PyUnicode_InternFromString("key")) == NULL) return -1;
};
static const char * const Import_fields[]={
"names",
+ "is_lazy",
};
static const char * const ImportFrom_fields[]={
"module",
"names",
"level",
+ "is_lazy",
};
static const char * const Global_fields[]={
"names",
return 0;
}
}
+ {
+ PyObject *type = (PyObject *)&PyLong_Type;
+ type = _Py_union_type_or(type, Py_None);
+ cond = type != NULL;
+ if (!cond) {
+ Py_DECREF(Import_annotations);
+ return 0;
+ }
+ cond = PyDict_SetItemString(Import_annotations, "is_lazy", type) == 0;
+ Py_DECREF(type);
+ if (!cond) {
+ Py_DECREF(Import_annotations);
+ return 0;
+ }
+ }
cond = PyObject_SetAttrString(state->Import_type, "_field_types",
Import_annotations) == 0;
if (!cond) {
return 0;
}
}
+ {
+ PyObject *type = (PyObject *)&PyLong_Type;
+ type = _Py_union_type_or(type, Py_None);
+ cond = type != NULL;
+ if (!cond) {
+ Py_DECREF(ImportFrom_annotations);
+ return 0;
+ }
+ cond = PyDict_SetItemString(ImportFrom_annotations, "is_lazy", type) ==
+ 0;
+ Py_DECREF(type);
+ if (!cond) {
+ Py_DECREF(ImportFrom_annotations);
+ return 0;
+ }
+ }
cond = PyObject_SetAttrString(state->ImportFrom_type, "_field_types",
ImportFrom_annotations) == 0;
if (!cond) {
" | Try(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody)\n"
" | TryStar(stmt* body, excepthandler* handlers, stmt* orelse, stmt* finalbody)\n"
" | Assert(expr test, expr? msg)\n"
- " | Import(alias* names)\n"
- " | ImportFrom(identifier? module, alias* names, int? level)\n"
+ " | Import(alias* names, int? is_lazy)\n"
+ " | ImportFrom(identifier? module, alias* names, int? level, int? is_lazy)\n"
" | Global(identifier* names)\n"
" | Nonlocal(identifier* names)\n"
" | Expr(expr value)\n"
if (PyObject_SetAttr(state->Assert_type, state->msg, Py_None) == -1)
return -1;
state->Import_type = make_type(state, "Import", state->stmt_type,
- Import_fields, 1,
- "Import(alias* names)");
+ Import_fields, 2,
+ "Import(alias* names, int? is_lazy)");
if (!state->Import_type) return -1;
+ if (PyObject_SetAttr(state->Import_type, state->is_lazy, Py_None) == -1)
+ return -1;
state->ImportFrom_type = make_type(state, "ImportFrom", state->stmt_type,
- ImportFrom_fields, 3,
- "ImportFrom(identifier? module, alias* names, int? level)");
+ ImportFrom_fields, 4,
+ "ImportFrom(identifier? module, alias* names, int? level, int? is_lazy)");
if (!state->ImportFrom_type) return -1;
if (PyObject_SetAttr(state->ImportFrom_type, state->module, Py_None) == -1)
return -1;
if (PyObject_SetAttr(state->ImportFrom_type, state->level, Py_None) == -1)
return -1;
+ if (PyObject_SetAttr(state->ImportFrom_type, state->is_lazy, Py_None) == -1)
+ return -1;
state->Global_type = make_type(state, "Global", state->stmt_type,
Global_fields, 1,
"Global(identifier* names)");
}
stmt_ty
-_PyAST_Import(asdl_alias_seq * names, int lineno, int col_offset, int
- end_lineno, int end_col_offset, PyArena *arena)
+_PyAST_Import(asdl_alias_seq * names, int is_lazy, int lineno, int col_offset,
+ int end_lineno, int end_col_offset, PyArena *arena)
{
stmt_ty p;
p = (stmt_ty)_PyArena_Malloc(arena, sizeof(*p));
return NULL;
p->kind = Import_kind;
p->v.Import.names = names;
+ p->v.Import.is_lazy = is_lazy;
p->lineno = lineno;
p->col_offset = col_offset;
p->end_lineno = end_lineno;
stmt_ty
_PyAST_ImportFrom(identifier module, asdl_alias_seq * names, int level, int
- lineno, int col_offset, int end_lineno, int end_col_offset,
- PyArena *arena)
+ is_lazy, int lineno, int col_offset, int end_lineno, int
+ end_col_offset, PyArena *arena)
{
stmt_ty p;
p = (stmt_ty)_PyArena_Malloc(arena, sizeof(*p));
p->v.ImportFrom.module = module;
p->v.ImportFrom.names = names;
p->v.ImportFrom.level = level;
+ p->v.ImportFrom.is_lazy = is_lazy;
p->lineno = lineno;
p->col_offset = col_offset;
p->end_lineno = end_lineno;
if (PyObject_SetAttr(result, state->names, value) == -1)
goto failed;
Py_DECREF(value);
+ value = ast2obj_int(state, o->v.Import.is_lazy);
+ if (!value) goto failed;
+ if (PyObject_SetAttr(result, state->is_lazy, value) == -1)
+ goto failed;
+ Py_DECREF(value);
break;
case ImportFrom_kind:
tp = (PyTypeObject *)state->ImportFrom_type;
if (PyObject_SetAttr(result, state->level, value) == -1)
goto failed;
Py_DECREF(value);
+ value = ast2obj_int(state, o->v.ImportFrom.is_lazy);
+ if (!value) goto failed;
+ if (PyObject_SetAttr(result, state->is_lazy, value) == -1)
+ goto failed;
+ Py_DECREF(value);
break;
case Global_kind:
tp = (PyTypeObject *)state->Global_type;
}
if (isinstance) {
asdl_alias_seq* names;
+ int is_lazy;
if (PyObject_GetOptionalAttr(obj, state->names, &tmp) < 0) {
return -1;
}
Py_CLEAR(tmp);
}
- *out = _PyAST_Import(names, lineno, col_offset, end_lineno,
+ if (PyObject_GetOptionalAttr(obj, state->is_lazy, &tmp) < 0) {
+ return -1;
+ }
+ if (tmp == NULL || tmp == Py_None) {
+ Py_CLEAR(tmp);
+ is_lazy = 0;
+ }
+ else {
+ int res;
+ if (_Py_EnterRecursiveCall(" while traversing 'Import' node")) {
+ goto failed;
+ }
+ res = obj2ast_int(state, tmp, &is_lazy, arena);
+ _Py_LeaveRecursiveCall();
+ if (res != 0) goto failed;
+ Py_CLEAR(tmp);
+ }
+ *out = _PyAST_Import(names, is_lazy, lineno, col_offset, end_lineno,
end_col_offset, arena);
if (*out == NULL) goto failed;
return 0;
identifier module;
asdl_alias_seq* names;
int level;
+ int is_lazy;
if (PyObject_GetOptionalAttr(obj, state->module, &tmp) < 0) {
return -1;
if (res != 0) goto failed;
Py_CLEAR(tmp);
}
- *out = _PyAST_ImportFrom(module, names, level, lineno, col_offset,
- end_lineno, end_col_offset, arena);
+ if (PyObject_GetOptionalAttr(obj, state->is_lazy, &tmp) < 0) {
+ return -1;
+ }
+ if (tmp == NULL || tmp == Py_None) {
+ Py_CLEAR(tmp);
+ is_lazy = 0;
+ }
+ else {
+ int res;
+ if (_Py_EnterRecursiveCall(" while traversing 'ImportFrom' node")) {
+ goto failed;
+ }
+ res = obj2ast_int(state, tmp, &is_lazy, arena);
+ _Py_LeaveRecursiveCall();
+ if (res != 0) goto failed;
+ Py_CLEAR(tmp);
+ }
+ *out = _PyAST_ImportFrom(module, names, level, is_lazy, lineno,
+ col_offset, end_lineno, end_col_offset, arena);
if (*out == NULL) goto failed;
return 0;
}
#include "pycore_fileutils.h" // _PyFile_Flush
#include "pycore_floatobject.h" // _PyFloat_ExactDealloc()
#include "pycore_interp.h" // _PyInterpreterState_GetConfig()
+#include "pycore_import.h" // _PyImport_LazyImportModuleLevelObject ()
#include "pycore_long.h" // _PyLong_CompactValue
#include "pycore_modsupport.h" // _PyArg_NoKwnames()
#include "pycore_object.h" // _Py_AddToAllObjects()
}
+/*[clinic input]
+__lazy_import__ as builtin___lazy_import__
+
+ name: object
+ globals: object(c_default="NULL") = None
+ locals: object(c_default="NULL") = None
+ fromlist: object(c_default="NULL") = ()
+ level: int = 0
+
+Lazily imports a module.
+
+Returns either the module to be imported or a imp.lazy_module object which
+indicates the module to be lazily imported.
+[clinic start generated code]*/
+
+static PyObject *
+builtin___lazy_import___impl(PyObject *module, PyObject *name,
+ PyObject *globals, PyObject *locals,
+ PyObject *fromlist, int level)
+/*[clinic end generated code: output=300f1771094b9e8c input=9394874f340b2948]*/
+{
+ PyObject *builtins;
+ PyThreadState *tstate = PyThreadState_GET();
+ if (globals == NULL) {
+ globals = PyEval_GetGlobals();
+ }
+ if (locals == NULL) {
+ locals = globals;
+ }
+
+ if (PyDict_GetItemRef(globals, &_Py_ID(__builtins__), &builtins) < 0) {
+ return NULL;
+ }
+ if (builtins == NULL) {
+ PyErr_SetString(PyExc_ValueError,
+ "unable to get builtins for lazy import");
+ return NULL;
+ }
+ if (PyModule_Check(builtins)) {
+ PyObject *builtins_dict = Py_XNewRef(PyModule_GetDict(builtins));
+ if (builtins_dict == NULL) {
+ Py_DECREF(builtins);
+ PyErr_SetString(PyExc_AttributeError,
+ "builtins module has no dict");
+ return NULL;
+ }
+ Py_SETREF(builtins, builtins_dict);
+ }
+
+ PyObject *res = _PyImport_LazyImportModuleLevelObject(
+ tstate, name, builtins, globals, locals, fromlist, level);
+ Py_DECREF(builtins);
+ return res;
+}
+
/*[clinic input]
abs as builtin_abs
{"__build_class__", _PyCFunction_CAST(builtin___build_class__),
METH_FASTCALL | METH_KEYWORDS, build_class_doc},
BUILTIN___IMPORT___METHODDEF
+ BUILTIN___LAZY_IMPORT___METHODDEF
BUILTIN_ABS_METHODDEF
BUILTIN_ALL_METHODDEF
BUILTIN_ANY_METHODDEF
#include "pycore_audit.h" // _PySys_Audit()
#include "pycore_backoff.h"
#include "pycore_cell.h" // PyCell_GetRef()
+#include "pycore_ceval.h" // _PyEval_LazyImportName(), _PyEval_LazyImportFrom()
#include "pycore_code.h"
#include "pycore_emscripten_signal.h" // _Py_CHECK_EMSCRIPTEN_SIGNALS
#include "pycore_function.h"
+#include "pycore_import.h" // _PyImport_LoadLazyImportTstate()
#include "pycore_instruments.h"
#include "pycore_interpolation.h" // _PyInterpolation_Build()
#include "pycore_intrinsics.h"
+#include "pycore_lazyimportobject.h" // PyLazyImport_CheckExact()
#include "pycore_long.h" // _PyLong_ExactDealloc(), _PyLong_GetZero()
#include "pycore_moduleobject.h" // PyModuleObject
#include "pycore_object.h" // _PyObject_GC_TRACK()
}
ERROR_NO_POP();
}
+
+ if (PyLazyImport_CheckExact(v_o)) {
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ Py_SETREF(v_o, l_v);
+ ERROR_IF(v_o == NULL);
+ }
}
else {
/* Slow-path if globals or builtins is not a dict */
ERROR_IF(true);
}
}
+ if (PyLazyImport_CheckExact(v_o)) {
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ Py_SETREF(v_o, l_v);
+ ERROR_IF(v_o == NULL);
+ }
}
}
v = PyStackRef_FromPyObjectSteal(v_o);
PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
PyObject *v_o = _PyEval_LoadName(tstate, frame, name);
ERROR_IF(v_o == NULL);
+ if (PyLazyImport_CheckExact(v_o)) {
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ // cannot early-decref v_o as it may cause a side-effect on l_v
+ if (l_v == NULL) {
+ Py_DECREF(v_o);
+ ERROR_IF(true);
+ }
+ int err = PyDict_SetItem(GLOBALS(), name, l_v);
+ if (err < 0) {
+ Py_DECREF(v_o);
+ Py_DECREF(l_v);
+ ERROR_IF(true);
+ }
+ Py_SETREF(v_o, l_v);
+ }
+
v = PyStackRef_FromPyObjectSteal(v_o);
}
op(_LOAD_GLOBAL, ( -- res[1])) {
PyObject *name = GETITEM(FRAME_CO_NAMES, oparg>>1);
_PyEval_LoadGlobalStackRef(GLOBALS(), BUILTINS(), name, res);
+
ERROR_IF(PyStackRef_IsNull(*res));
}
b = res ? PyStackRef_True : PyStackRef_False;
}
- inst(IMPORT_NAME, (level, fromlist -- res)) {
- PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
- PyObject *res_o = _PyEval_ImportName(tstate, frame, name,
- PyStackRef_AsPyObjectBorrow(fromlist),
- PyStackRef_AsPyObjectBorrow(level));
+ inst(IMPORT_NAME, (level, fromlist -- res)) {
+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2);
+ PyObject *res_o;
+ if (!(oparg & 0x02)) {
+ res_o = _PyEval_LazyImportName(tstate, BUILTINS(), GLOBALS(),
+ LOCALS(), name,
+ PyStackRef_AsPyObjectBorrow(fromlist),
+ PyStackRef_AsPyObjectBorrow(level),
+ oparg & 0x01);
+
+ }
+ else {
+ res_o = _PyEval_ImportName(tstate, BUILTINS(), GLOBALS(),
+ LOCALS(), name,
+ PyStackRef_AsPyObjectBorrow(fromlist),
+ PyStackRef_AsPyObjectBorrow(level));
+ }
DECREF_INPUTS();
ERROR_IF(res_o == NULL);
res = PyStackRef_FromPyObjectSteal(res_o);
inst(IMPORT_FROM, (from -- from, res)) {
PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
- PyObject *res_o = _PyEval_ImportFrom(tstate, PyStackRef_AsPyObjectBorrow(from), name);
+ PyObject *res_o;
+ if (PyLazyImport_CheckExact(PyStackRef_AsPyObjectBorrow(from))) {
+ res_o = _PyEval_LazyImportFrom(
+ tstate, frame, PyStackRef_AsPyObjectBorrow(from), name);
+ }
+ else {
+ res_o = _PyEval_ImportFrom(
+ tstate, PyStackRef_AsPyObjectBorrow(from), name);
+ }
+
ERROR_IF(res_o == NULL);
res = PyStackRef_FromPyObjectSteal(res_o);
}
}
PyObject *
-_PyEval_ImportName(PyThreadState *tstate, _PyInterpreterFrame *frame,
- PyObject *name, PyObject *fromlist, PyObject *level)
+_PyEval_ImportName(PyThreadState *tstate, PyObject *builtins,
+ PyObject *globals, PyObject *locals, PyObject *name,
+ PyObject *fromlist, PyObject *level)
{
PyObject *import_func;
- if (PyMapping_GetOptionalItem(frame->f_builtins, &_Py_ID(__import__), &import_func) < 0) {
+ if (PyMapping_GetOptionalItem(builtins, &_Py_ID(__import__),
+ &import_func) < 0) {
return NULL;
}
if (import_func == NULL) {
return NULL;
}
- PyObject *locals = frame->f_locals;
+ PyObject *res = _PyEval_ImportNameWithImport(
+ tstate, import_func, globals, locals, name, fromlist, level);
+ Py_DECREF(import_func);
+ return res;
+}
+
+PyObject *
+_PyEval_ImportNameWithImport(PyThreadState *tstate, PyObject *import_func,
+ PyObject *globals, PyObject *locals,
+ PyObject *name, PyObject *fromlist, PyObject *level)
+{
if (locals == NULL) {
locals = Py_None;
}
/* Fast path for not overloaded __import__. */
if (_PyImport_IsDefaultImportFunc(tstate->interp, import_func)) {
- Py_DECREF(import_func);
int ilevel = PyLong_AsInt(level);
if (ilevel == -1 && _PyErr_Occurred(tstate)) {
return NULL;
}
return PyImport_ImportModuleLevelObject(
name,
- frame->f_globals,
+ globals,
locals,
fromlist,
ilevel);
}
- PyObject* args[5] = {name, frame->f_globals, locals, fromlist, level};
+ PyObject *args[5] = {name, globals, locals, fromlist, level};
PyObject *res = PyObject_Vectorcall(import_func, args, 5, NULL);
- Py_DECREF(import_func);
+ return res;
+}
+
+static int
+check_lazy_import_compatibility(PyThreadState *tstate, PyObject *globals,
+ PyObject *name, PyObject *level)
+{
+ // Check if this module should be imported lazily due to
+ // the compatibility mode support via __lazy_modules__.
+ PyObject *lazy_modules = NULL;
+ PyObject *abs_name = NULL;
+ int res = -1;
+
+ if (globals != NULL &&
+ PyMapping_GetOptionalItem(globals, &_Py_ID(__lazy_modules__),
+ &lazy_modules) < 0)
+ {
+ return -1;
+ }
+ if (lazy_modules == NULL) {
+ assert(!PyErr_Occurred());
+ return 0;
+ }
+
+ int ilevel = PyLong_AsInt(level);
+ if (ilevel == -1 && _PyErr_Occurred(tstate)) {
+ goto error;
+ }
+
+ abs_name = _PyImport_GetAbsName(tstate, name, globals, ilevel);
+ if (abs_name == NULL) {
+ goto error;
+ }
+
+ res = PySequence_Contains(lazy_modules, abs_name);
+error:
+ Py_XDECREF(abs_name);
+ Py_XDECREF(lazy_modules);
+ return res;
+}
+
+PyObject *
+_PyEval_LazyImportName(PyThreadState *tstate, PyObject *builtins,
+ PyObject *globals, PyObject *locals, PyObject *name,
+ PyObject *fromlist, PyObject *level, int lazy)
+{
+ PyObject *res = NULL;
+ // Check if global policy overrides the local syntax
+ switch (PyImport_GetLazyImportsMode()) {
+ case PyImport_LAZY_NONE:
+ lazy = 0;
+ break;
+ case PyImport_LAZY_ALL:
+ lazy = 1;
+ break;
+ case PyImport_LAZY_NORMAL:
+ break;
+ }
+
+ if (!lazy) {
+ // See if __lazy_modules__ forces this to be lazy.
+ lazy = check_lazy_import_compatibility(tstate, globals, name, level);
+ if (lazy < 0) {
+ return NULL;
+ }
+ }
+
+ if (!lazy) {
+ // Not a lazy import or lazy imports are disabled, fallback to the
+ // regular import.
+ return _PyEval_ImportName(tstate, builtins, globals, locals,
+ name, fromlist, level);
+ }
+
+ PyObject *lazy_import_func;
+ if (PyMapping_GetOptionalItem(builtins, &_Py_ID(__lazy_import__),
+ &lazy_import_func) < 0) {
+ goto error;
+ }
+ if (lazy_import_func == NULL) {
+ assert(!PyErr_Occurred());
+ _PyErr_SetString(tstate, PyExc_ImportError,
+ "__lazy_import__ not found");
+ goto error;
+ }
+
+ if (locals == NULL) {
+ locals = Py_None;
+ }
+
+ if (_PyImport_IsDefaultLazyImportFunc(tstate->interp, lazy_import_func)) {
+ int ilevel = PyLong_AsInt(level);
+ if (ilevel == -1 && PyErr_Occurred()) {
+ goto error;
+ }
+
+ res = _PyImport_LazyImportModuleLevelObject(
+ tstate, name, builtins, globals, locals, fromlist, ilevel
+ );
+ goto error;
+ }
+
+ PyObject *args[6] = {name, globals, locals, fromlist, level, builtins};
+ res = PyObject_Vectorcall(lazy_import_func, args, 6, NULL);
+error:
+ Py_XDECREF(lazy_import_func);
return res;
}
return NULL;
}
+PyObject *
+_PyEval_LazyImportFrom(PyThreadState *tstate, _PyInterpreterFrame *frame, PyObject *v, PyObject *name)
+{
+ assert(PyLazyImport_CheckExact(v));
+ assert(name);
+ assert(PyUnicode_Check(name));
+ PyObject *ret;
+ PyLazyImportObject *d = (PyLazyImportObject *)v;
+ PyObject *mod = PyImport_GetModule(d->lz_from);
+ if (mod != NULL) {
+ // Check if the module already has the attribute, if so, resolve it
+ // eagerly.
+ if (PyModule_Check(mod)) {
+ PyObject *mod_dict = PyModule_GetDict(mod);
+ if (mod_dict != NULL) {
+ if (PyDict_GetItemRef(mod_dict, name, &ret) < 0) {
+ Py_DECREF(mod);
+ return NULL;
+ }
+ if (ret != NULL) {
+ Py_DECREF(mod);
+ return ret;
+ }
+ }
+ }
+ Py_DECREF(mod);
+ }
+
+ if (d->lz_attr != NULL) {
+ if (PyUnicode_Check(d->lz_attr)) {
+ PyObject *from = PyUnicode_FromFormat(
+ "%U.%U", d->lz_from, d->lz_attr);
+ if (from == NULL) {
+ return NULL;
+ }
+ ret = _PyLazyImport_New(frame, d->lz_builtins, from, name);
+ Py_DECREF(from);
+ return ret;
+ }
+ }
+ else {
+ Py_ssize_t dot = PyUnicode_FindChar(
+ d->lz_from, '.', 0, PyUnicode_GET_LENGTH(d->lz_from), 1
+ );
+ if (dot >= 0) {
+ PyObject *from = PyUnicode_Substring(d->lz_from, 0, dot);
+ if (from == NULL) {
+ return NULL;
+ }
+ ret = _PyLazyImport_New(frame, d->lz_builtins, from, name);
+ Py_DECREF(from);
+ return ret;
+ }
+ }
+ ret = _PyLazyImport_New(frame, d->lz_builtins, d->lz_from, name);
+ return ret;
+}
+
#define CANNOT_CATCH_MSG "catching classes that do not inherit from "\
"BaseException is not allowed"
}
*writeto = PyStackRef_FromPyObjectSteal(res);
}
+
+ PyObject *res_o = PyStackRef_AsPyObjectBorrow(*writeto);
+ if (res_o != NULL && PyLazyImport_CheckExact(res_o)) {
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(PyThreadState_GET(), res_o);
+ PyStackRef_CLOSE(writeto[0]);
+ if (l_v == NULL) {
+ assert(PyErr_Occurred());
+ *writeto = PyStackRef_NULL;
+ return;
+ }
+ int err = PyDict_SetItem(globals, name, l_v);
+ if (err < 0) {
+ Py_DECREF(l_v);
+ *writeto = PyStackRef_NULL;
+ return;
+ }
+ *writeto = PyStackRef_FromPyObjectSteal(l_v);
+ }
}
PyObject *
#include "pycore_interpolation.h" // _PyInterpolation_Build()
#include "pycore_intrinsics.h"
#include "pycore_jit.h"
+#include "pycore_lazyimportobject.h"
#include "pycore_list.h" // _PyList_GetItemRef()
#include "pycore_long.h" // _PyLong_GetZero()
#include "pycore_moduleobject.h" // PyModuleObject
return return_value;
}
+PyDoc_STRVAR(builtin___lazy_import____doc__,
+"__lazy_import__($module, /, name, globals=None, locals=None,\n"
+" fromlist=(), level=0)\n"
+"--\n"
+"\n"
+"Lazily imports a module.\n"
+"\n"
+"Returns either the module to be imported or a imp.lazy_module object which\n"
+"indicates the module to be lazily imported.");
+
+#define BUILTIN___LAZY_IMPORT___METHODDEF \
+ {"__lazy_import__", _PyCFunction_CAST(builtin___lazy_import__), METH_FASTCALL|METH_KEYWORDS, builtin___lazy_import____doc__},
+
+static PyObject *
+builtin___lazy_import___impl(PyObject *module, PyObject *name,
+ PyObject *globals, PyObject *locals,
+ PyObject *fromlist, int level);
+
+static PyObject *
+builtin___lazy_import__(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
+{
+ PyObject *return_value = NULL;
+ #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
+
+ #define NUM_KEYWORDS 5
+ static struct {
+ PyGC_Head _this_is_not_used;
+ PyObject_VAR_HEAD
+ Py_hash_t ob_hash;
+ PyObject *ob_item[NUM_KEYWORDS];
+ } _kwtuple = {
+ .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
+ .ob_hash = -1,
+ .ob_item = { &_Py_ID(name), &_Py_ID(globals), &_Py_ID(locals), &_Py_ID(fromlist), &_Py_ID(level), },
+ };
+ #undef NUM_KEYWORDS
+ #define KWTUPLE (&_kwtuple.ob_base.ob_base)
+
+ #else // !Py_BUILD_CORE
+ # define KWTUPLE NULL
+ #endif // !Py_BUILD_CORE
+
+ static const char * const _keywords[] = {"name", "globals", "locals", "fromlist", "level", NULL};
+ static _PyArg_Parser _parser = {
+ .keywords = _keywords,
+ .fname = "__lazy_import__",
+ .kwtuple = KWTUPLE,
+ };
+ #undef KWTUPLE
+ PyObject *argsbuf[5];
+ Py_ssize_t noptargs = nargs + (kwnames ? PyTuple_GET_SIZE(kwnames) : 0) - 1;
+ PyObject *name;
+ PyObject *globals = NULL;
+ PyObject *locals = NULL;
+ PyObject *fromlist = NULL;
+ int level = 0;
+
+ args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser,
+ /*minpos*/ 1, /*maxpos*/ 5, /*minkw*/ 0, /*varpos*/ 0, argsbuf);
+ if (!args) {
+ goto exit;
+ }
+ name = args[0];
+ if (!noptargs) {
+ goto skip_optional_pos;
+ }
+ if (args[1]) {
+ globals = args[1];
+ if (!--noptargs) {
+ goto skip_optional_pos;
+ }
+ }
+ if (args[2]) {
+ locals = args[2];
+ if (!--noptargs) {
+ goto skip_optional_pos;
+ }
+ }
+ if (args[3]) {
+ fromlist = args[3];
+ if (!--noptargs) {
+ goto skip_optional_pos;
+ }
+ }
+ level = PyLong_AsInt(args[4]);
+ if (level == -1 && PyErr_Occurred()) {
+ goto exit;
+ }
+skip_optional_pos:
+ return_value = builtin___lazy_import___impl(module, name, globals, locals, fromlist, level);
+
+exit:
+ return return_value;
+}
+
PyDoc_STRVAR(builtin_abs__doc__,
"abs($module, number, /)\n"
"--\n"
exit:
return return_value;
}
-/*[clinic end generated code: output=06500bcc9a341e68 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=1c3327da8885bb8e input=a9049054013a1b77]*/
return return_value;
}
+PyDoc_STRVAR(_imp__set_lazy_attributes__doc__,
+"_set_lazy_attributes($module, modobj, name, /)\n"
+"--\n"
+"\n"
+"Sets attributes to lazy submodules on the module, as side effects.");
+
+#define _IMP__SET_LAZY_ATTRIBUTES_METHODDEF \
+ {"_set_lazy_attributes", _PyCFunction_CAST(_imp__set_lazy_attributes), METH_FASTCALL, _imp__set_lazy_attributes__doc__},
+
+static PyObject *
+_imp__set_lazy_attributes_impl(PyObject *module, PyObject *modobj,
+ PyObject *name);
+
+static PyObject *
+_imp__set_lazy_attributes(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
+{
+ PyObject *return_value = NULL;
+ PyObject *modobj;
+ PyObject *name;
+
+ if (!_PyArg_CheckPositional("_set_lazy_attributes", nargs, 2, 2)) {
+ goto exit;
+ }
+ modobj = args[0];
+ if (!PyUnicode_Check(args[1])) {
+ _PyArg_BadArgument("_set_lazy_attributes", "argument 2", "str", args[1]);
+ goto exit;
+ }
+ name = args[1];
+ return_value = _imp__set_lazy_attributes_impl(module, modobj, name);
+
+exit:
+ return return_value;
+}
+
#ifndef _IMP_CREATE_DYNAMIC_METHODDEF
#define _IMP_CREATE_DYNAMIC_METHODDEF
#endif /* !defined(_IMP_CREATE_DYNAMIC_METHODDEF) */
#ifndef _IMP_EXEC_DYNAMIC_METHODDEF
#define _IMP_EXEC_DYNAMIC_METHODDEF
#endif /* !defined(_IMP_EXEC_DYNAMIC_METHODDEF) */
-/*[clinic end generated code: output=24f597d6b0f3feed input=a9049054013a1b77]*/
+/*[clinic end generated code: output=5fa42f580441b3fa input=a9049054013a1b77]*/
return return_value;
}
+PyDoc_STRVAR(sys_set_lazy_imports_filter__doc__,
+"set_lazy_imports_filter($module, /, filter)\n"
+"--\n"
+"\n"
+"Set the lazy imports filter callback.\n"
+"\n"
+"The filter is a callable which disables lazy imports when they\n"
+"would otherwise be enabled. Returns True if the import is still enabled\n"
+"or False to disable it. The callable is called with:\n"
+"\n"
+"(importing_module_name, imported_module_name, [fromlist])\n"
+"\n"
+"Pass None to clear the filter.");
+
+#define SYS_SET_LAZY_IMPORTS_FILTER_METHODDEF \
+ {"set_lazy_imports_filter", _PyCFunction_CAST(sys_set_lazy_imports_filter), METH_FASTCALL|METH_KEYWORDS, sys_set_lazy_imports_filter__doc__},
+
+static PyObject *
+sys_set_lazy_imports_filter_impl(PyObject *module, PyObject *filter);
+
+static PyObject *
+sys_set_lazy_imports_filter(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
+{
+ PyObject *return_value = NULL;
+ #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
+
+ #define NUM_KEYWORDS 1
+ static struct {
+ PyGC_Head _this_is_not_used;
+ PyObject_VAR_HEAD
+ Py_hash_t ob_hash;
+ PyObject *ob_item[NUM_KEYWORDS];
+ } _kwtuple = {
+ .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
+ .ob_hash = -1,
+ .ob_item = { &_Py_ID(filter), },
+ };
+ #undef NUM_KEYWORDS
+ #define KWTUPLE (&_kwtuple.ob_base.ob_base)
+
+ #else // !Py_BUILD_CORE
+ # define KWTUPLE NULL
+ #endif // !Py_BUILD_CORE
+
+ static const char * const _keywords[] = {"filter", NULL};
+ static _PyArg_Parser _parser = {
+ .keywords = _keywords,
+ .fname = "set_lazy_imports_filter",
+ .kwtuple = KWTUPLE,
+ };
+ #undef KWTUPLE
+ PyObject *argsbuf[1];
+ PyObject *filter;
+
+ args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser,
+ /*minpos*/ 1, /*maxpos*/ 1, /*minkw*/ 0, /*varpos*/ 0, argsbuf);
+ if (!args) {
+ goto exit;
+ }
+ filter = args[0];
+ return_value = sys_set_lazy_imports_filter_impl(module, filter);
+
+exit:
+ return return_value;
+}
+
+PyDoc_STRVAR(sys_get_lazy_imports_filter__doc__,
+"get_lazy_imports_filter($module, /)\n"
+"--\n"
+"\n"
+"Get the current lazy imports filter callback.\n"
+"\n"
+"Returns the filter callable or None if no filter is set.");
+
+#define SYS_GET_LAZY_IMPORTS_FILTER_METHODDEF \
+ {"get_lazy_imports_filter", (PyCFunction)sys_get_lazy_imports_filter, METH_NOARGS, sys_get_lazy_imports_filter__doc__},
+
+static PyObject *
+sys_get_lazy_imports_filter_impl(PyObject *module);
+
+static PyObject *
+sys_get_lazy_imports_filter(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+ return sys_get_lazy_imports_filter_impl(module);
+}
+
+PyDoc_STRVAR(sys_set_lazy_imports__doc__,
+"set_lazy_imports($module, /, mode)\n"
+"--\n"
+"\n"
+"Sets the global lazy imports mode.\n"
+"\n"
+"The mode parameter must be one of the following strings:\n"
+"- \"all\": All top-level imports become potentially lazy\n"
+"- \"none\": All lazy imports are suppressed (even explicitly marked ones)\n"
+"- \"normal\": Only explicitly marked imports (with \'lazy\' keyword) are lazy\n"
+"\n"
+"In addition to the mode, lazy imports can be controlled via the filter\n"
+"provided to sys.set_lazy_imports_filter");
+
+#define SYS_SET_LAZY_IMPORTS_METHODDEF \
+ {"set_lazy_imports", _PyCFunction_CAST(sys_set_lazy_imports), METH_FASTCALL|METH_KEYWORDS, sys_set_lazy_imports__doc__},
+
+static PyObject *
+sys_set_lazy_imports_impl(PyObject *module, PyObject *mode);
+
+static PyObject *
+sys_set_lazy_imports(PyObject *module, PyObject *const *args, Py_ssize_t nargs, PyObject *kwnames)
+{
+ PyObject *return_value = NULL;
+ #if defined(Py_BUILD_CORE) && !defined(Py_BUILD_CORE_MODULE)
+
+ #define NUM_KEYWORDS 1
+ static struct {
+ PyGC_Head _this_is_not_used;
+ PyObject_VAR_HEAD
+ Py_hash_t ob_hash;
+ PyObject *ob_item[NUM_KEYWORDS];
+ } _kwtuple = {
+ .ob_base = PyVarObject_HEAD_INIT(&PyTuple_Type, NUM_KEYWORDS)
+ .ob_hash = -1,
+ .ob_item = { &_Py_ID(mode), },
+ };
+ #undef NUM_KEYWORDS
+ #define KWTUPLE (&_kwtuple.ob_base.ob_base)
+
+ #else // !Py_BUILD_CORE
+ # define KWTUPLE NULL
+ #endif // !Py_BUILD_CORE
+
+ static const char * const _keywords[] = {"mode", NULL};
+ static _PyArg_Parser _parser = {
+ .keywords = _keywords,
+ .fname = "set_lazy_imports",
+ .kwtuple = KWTUPLE,
+ };
+ #undef KWTUPLE
+ PyObject *argsbuf[1];
+ PyObject *mode;
+
+ args = _PyArg_UnpackKeywords(args, nargs, NULL, kwnames, &_parser,
+ /*minpos*/ 1, /*maxpos*/ 1, /*minkw*/ 0, /*varpos*/ 0, argsbuf);
+ if (!args) {
+ goto exit;
+ }
+ mode = args[0];
+ return_value = sys_set_lazy_imports_impl(module, mode);
+
+exit:
+ return return_value;
+}
+
+PyDoc_STRVAR(sys_get_lazy_imports__doc__,
+"get_lazy_imports($module, /)\n"
+"--\n"
+"\n"
+"Gets the global lazy imports mode.\n"
+"\n"
+"Returns \"all\" if all top level imports are potentially lazy.\n"
+"Returns \"none\" if all explicitly marked lazy imports are suppressed.\n"
+"Returns \"normal\" if only explicitly marked imports are lazy.");
+
+#define SYS_GET_LAZY_IMPORTS_METHODDEF \
+ {"get_lazy_imports", (PyCFunction)sys_get_lazy_imports, METH_NOARGS, sys_get_lazy_imports__doc__},
+
+static PyObject *
+sys_get_lazy_imports_impl(PyObject *module);
+
+static PyObject *
+sys_get_lazy_imports(PyObject *module, PyObject *Py_UNUSED(ignored))
+{
+ return sys_get_lazy_imports_impl(module);
+}
+
PyDoc_STRVAR(_jit_is_available__doc__,
"is_available($module, /)\n"
"--\n"
#ifndef SYS_GETANDROIDAPILEVEL_METHODDEF
#define SYS_GETANDROIDAPILEVEL_METHODDEF
#endif /* !defined(SYS_GETANDROIDAPILEVEL_METHODDEF) */
-/*[clinic end generated code: output=5f7d84c5bf00d557 input=a9049054013a1b77]*/
+/*[clinic end generated code: output=adbadb629b98eabf input=a9049054013a1b77]*/
#define LOAD_ZERO_SUPER_METHOD -4
static int
-codegen_addop_name(compiler *c, location loc,
- int opcode, PyObject *dict, PyObject *o)
+codegen_addop_name_custom(compiler *c, location loc, int opcode,
+ PyObject *dict, PyObject *o, int shift, int low)
{
PyObject *mangled = _PyCompile_MaybeMangle(c, o);
if (!mangled) {
if (arg < 0) {
return ERROR;
}
+ ADDOP_I(c, loc, opcode, (arg << shift) | low);
+ return SUCCESS;
+}
+
+static int
+codegen_addop_name(compiler *c, location loc,
+ int opcode, PyObject *dict, PyObject *o)
+{
+ int shift = 0, low = 0;
if (opcode == LOAD_ATTR) {
- arg <<= 1;
+ shift = 1;
}
if (opcode == LOAD_METHOD) {
opcode = LOAD_ATTR;
- arg <<= 1;
- arg |= 1;
+ shift = 1;
+ low = 1;
}
if (opcode == LOAD_SUPER_ATTR) {
- arg <<= 2;
- arg |= 2;
+ shift = 2;
+ low = 2;
}
if (opcode == LOAD_SUPER_METHOD) {
opcode = LOAD_SUPER_ATTR;
- arg <<= 2;
- arg |= 3;
+ shift = 2;
+ low = 3;
}
if (opcode == LOAD_ZERO_SUPER_ATTR) {
opcode = LOAD_SUPER_ATTR;
- arg <<= 2;
+ shift = 2;
}
if (opcode == LOAD_ZERO_SUPER_METHOD) {
opcode = LOAD_SUPER_ATTR;
- arg <<= 2;
- arg |= 1;
+ shift = 2;
+ low = 1;
}
- ADDOP_I(c, loc, opcode, arg);
- return SUCCESS;
+ return codegen_addop_name_custom(c, loc, opcode, dict, o, shift, low);
}
#define ADDOP_NAME(C, LOC, OP, O, TYPE) \
RETURN_IF_ERROR(codegen_addop_name((C), (LOC), (OP), METADATA(C)->u_ ## TYPE, (O)))
-static int
+#define ADDOP_NAME_CUSTOM(C, LOC, OP, O, TYPE, SHIFT, LOW) \
+ RETURN_IF_ERROR(codegen_addop_name_custom((C), (LOC), (OP), METADATA(C)->u_ ## TYPE, (O), SHIFT, LOW))
+
+ static int
codegen_addop_j(instr_sequence *seq, location loc,
int opcode, jump_target_label target)
{
return codegen_nameop(c, loc, asname, Store);
}
+static int
+codegen_validate_lazy_import(compiler *c, location loc)
+{
+ if (_PyCompile_ScopeType(c) != COMPILE_SCOPE_MODULE) {
+ return _PyCompile_Error(
+ c, loc, "lazy imports only allowed in module scope");
+ }
+
+ return SUCCESS;
+}
+
static int
codegen_import(compiler *c, stmt_ty s)
{
ADDOP_LOAD_CONST(c, loc, zero);
ADDOP_LOAD_CONST(c, loc, Py_None);
- ADDOP_NAME(c, loc, IMPORT_NAME, alias->name, names);
+ if (s->v.Import.is_lazy) {
+ RETURN_IF_ERROR(codegen_validate_lazy_import(c, loc));
+ ADDOP_NAME_CUSTOM(c, loc, IMPORT_NAME, alias->name, names, 2, 1);
+ } else {
+ if (_PyCompile_InExceptionHandler(c) ||
+ _PyCompile_ScopeType(c) != COMPILE_SCOPE_MODULE) {
+ // force eager import in try/except block
+ ADDOP_NAME_CUSTOM(c, loc, IMPORT_NAME, alias->name, names, 2, 2);
+ } else {
+ ADDOP_NAME_CUSTOM(c, loc, IMPORT_NAME, alias->name, names, 2, 0);
+ }
+ }
if (alias->asname) {
r = codegen_import_as(c, loc, alias->name, alias->asname);
ADDOP_LOAD_CONST_NEW(c, LOC(s), names);
+ identifier from = &_Py_STR(empty);
if (s->v.ImportFrom.module) {
- ADDOP_NAME(c, LOC(s), IMPORT_NAME, s->v.ImportFrom.module, names);
+ from = s->v.ImportFrom.module;
}
- else {
- _Py_DECLARE_STR(empty, "");
- ADDOP_NAME(c, LOC(s), IMPORT_NAME, &_Py_STR(empty), names);
+ if (s->v.ImportFrom.is_lazy) {
+ alias_ty alias = (alias_ty)asdl_seq_GET(s->v.ImportFrom.names, 0);
+ if (PyUnicode_READ_CHAR(alias->name, 0) == '*') {
+ return _PyCompile_Error(c, LOC(s), "cannot lazy import *");
+ }
+ RETURN_IF_ERROR(codegen_validate_lazy_import(c, LOC(s)));
+ ADDOP_NAME_CUSTOM(c, LOC(s), IMPORT_NAME, from, names, 2, 1);
+ } else {
+ alias_ty alias = (alias_ty)asdl_seq_GET(s->v.ImportFrom.names, 0);
+ if (_PyCompile_InExceptionHandler(c) ||
+ _PyCompile_ScopeType(c) != COMPILE_SCOPE_MODULE ||
+ PyUnicode_READ_CHAR(alias->name, 0) == '*') {
+ // forced non-lazy import due to try/except or import *
+ ADDOP_NAME_CUSTOM(c, LOC(s), IMPORT_NAME, from, names, 2, 2);
+ } else {
+ ADDOP_NAME_CUSTOM(c, LOC(s), IMPORT_NAME, from, names, 2, 0);
+ }
}
+
for (Py_ssize_t i = 0; i < n; i++) {
alias_ty alias = (alias_ty)asdl_seq_GET(s->v.ImportFrom.names, i);
identifier store_name;
return &c->u->u_fblock[c->u->u_nfblocks - 1];
}
+bool
+_PyCompile_InExceptionHandler(compiler *c)
+{
+ for (Py_ssize_t i = 0; i < c->u->u_nfblocks; i++) {
+ fblockinfo *block = &c->u->u_fblock[i];
+ switch (block->fb_type) {
+ case COMPILE_FBLOCK_TRY_EXCEPT:
+ case COMPILE_FBLOCK_FINALLY_TRY:
+ case COMPILE_FBLOCK_FINALLY_END:
+ case COMPILE_FBLOCK_EXCEPTION_HANDLER:
+ case COMPILE_FBLOCK_EXCEPTION_GROUP_HANDLER:
+ case COMPILE_FBLOCK_HANDLER_CLEANUP:
+ return true;
+ default:
+ break;
+ }
+ }
+ return false;
+}
+
void
_PyCompile_DeferredAnnotations(compiler *c,
PyObject **deferred_annotations,
SET_CURRENT_CACHED_VALUES(0);
JUMP_TO_ERROR();
}
+ if (PyLazyImport_CheckExact(v_o)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (l_v == NULL) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_DECREF(v_o);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ SET_CURRENT_CACHED_VALUES(0);
+ JUMP_TO_ERROR();
+ }
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ int err = PyDict_SetItem(GLOBALS(), name, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (err < 0) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_DECREF(v_o);
+ Py_DECREF(l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ SET_CURRENT_CACHED_VALUES(0);
+ JUMP_TO_ERROR();
+ }
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_SETREF(v_o, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
v = PyStackRef_FromPyObjectSteal(v_o);
_tos_cache0 = v;
_tos_cache1 = PyStackRef_ZERO_BITS;
oparg = CURRENT_OPARG();
fromlist = _stack_item_1;
level = _stack_item_0;
- PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
- stack_pointer[0] = level;
- stack_pointer[1] = fromlist;
- stack_pointer += 2;
- ASSERT_WITHIN_STACK_BOUNDS(__FILE__, __LINE__);
+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2);
+ PyObject *res_o;
+ if (!(oparg & 0x02)) {
+ stack_pointer[0] = level;
+ stack_pointer[1] = fromlist;
+ stack_pointer += 2;
+ ASSERT_WITHIN_STACK_BOUNDS(__FILE__, __LINE__);
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_LazyImportName(tstate, BUILTINS(), GLOBALS(),
+ LOCALS(), name,
+ PyStackRef_AsPyObjectBorrow(fromlist),
+ PyStackRef_AsPyObjectBorrow(level),
+ oparg & 0x01);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
+ else {
+ stack_pointer[0] = level;
+ stack_pointer[1] = fromlist;
+ stack_pointer += 2;
+ ASSERT_WITHIN_STACK_BOUNDS(__FILE__, __LINE__);
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_ImportName(tstate, BUILTINS(), GLOBALS(),
+ LOCALS(), name,
+ PyStackRef_AsPyObjectBorrow(fromlist),
+ PyStackRef_AsPyObjectBorrow(level));
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
_PyFrame_SetStackPointer(frame, stack_pointer);
- PyObject *res_o = _PyEval_ImportName(tstate, frame, name,
- PyStackRef_AsPyObjectBorrow(fromlist),
- PyStackRef_AsPyObjectBorrow(level));
_PyStackRef tmp = fromlist;
fromlist = PyStackRef_NULL;
stack_pointer[-1] = fromlist;
oparg = CURRENT_OPARG();
from = _stack_item_0;
PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
- stack_pointer[0] = from;
- stack_pointer += 1;
- ASSERT_WITHIN_STACK_BOUNDS(__FILE__, __LINE__);
- _PyFrame_SetStackPointer(frame, stack_pointer);
- PyObject *res_o = _PyEval_ImportFrom(tstate, PyStackRef_AsPyObjectBorrow(from), name);
- stack_pointer = _PyFrame_GetStackPointer(frame);
+ PyObject *res_o;
+ if (PyLazyImport_CheckExact(PyStackRef_AsPyObjectBorrow(from))) {
+ stack_pointer[0] = from;
+ stack_pointer += 1;
+ ASSERT_WITHIN_STACK_BOUNDS(__FILE__, __LINE__);
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_LazyImportFrom(
+ tstate, frame, PyStackRef_AsPyObjectBorrow(from), name);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
+ else {
+ stack_pointer[0] = from;
+ stack_pointer += 1;
+ ASSERT_WITHIN_STACK_BOUNDS(__FILE__, __LINE__);
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_ImportFrom(
+ tstate, PyStackRef_AsPyObjectBorrow(from), name);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
if (res_o == NULL) {
+ stack_pointer[-1] = from;
SET_CURRENT_CACHED_VALUES(0);
JUMP_TO_ERROR();
}
_PyStackRef res;
from = stack_pointer[-1];
PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
- _PyFrame_SetStackPointer(frame, stack_pointer);
- PyObject *res_o = _PyEval_ImportFrom(tstate, PyStackRef_AsPyObjectBorrow(from), name);
- stack_pointer = _PyFrame_GetStackPointer(frame);
+ PyObject *res_o;
+ if (PyLazyImport_CheckExact(PyStackRef_AsPyObjectBorrow(from))) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_LazyImportFrom(
+ tstate, frame, PyStackRef_AsPyObjectBorrow(from), name);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
+ else {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_ImportFrom(
+ tstate, PyStackRef_AsPyObjectBorrow(from), name);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
if (res_o == NULL) {
JUMP_TO_LABEL(error);
}
_PyStackRef res;
fromlist = stack_pointer[-1];
level = stack_pointer[-2];
- PyObject *name = GETITEM(FRAME_CO_NAMES, oparg);
+ PyObject *name = GETITEM(FRAME_CO_NAMES, oparg >> 2);
+ PyObject *res_o;
+ if (!(oparg & 0x02)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_LazyImportName(tstate, BUILTINS(), GLOBALS(),
+ LOCALS(), name,
+ PyStackRef_AsPyObjectBorrow(fromlist),
+ PyStackRef_AsPyObjectBorrow(level),
+ oparg & 0x01);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
+ else {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ res_o = _PyEval_ImportName(tstate, BUILTINS(), GLOBALS(),
+ LOCALS(), name,
+ PyStackRef_AsPyObjectBorrow(fromlist),
+ PyStackRef_AsPyObjectBorrow(level));
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
_PyFrame_SetStackPointer(frame, stack_pointer);
- PyObject *res_o = _PyEval_ImportName(tstate, frame, name,
- PyStackRef_AsPyObjectBorrow(fromlist),
- PyStackRef_AsPyObjectBorrow(level));
_PyStackRef tmp = fromlist;
fromlist = PyStackRef_NULL;
stack_pointer[-1] = fromlist;
}
JUMP_TO_LABEL(error);
}
+ if (PyLazyImport_CheckExact(v_o)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ Py_SETREF(v_o, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (v_o == NULL) {
+ JUMP_TO_LABEL(error);
+ }
+ }
}
else {
_PyFrame_SetStackPointer(frame, stack_pointer);
JUMP_TO_LABEL(error);
}
}
+ if (PyLazyImport_CheckExact(v_o)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ Py_SETREF(v_o, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (v_o == NULL) {
+ JUMP_TO_LABEL(error);
+ }
+ }
}
}
v = PyStackRef_FromPyObjectSteal(v_o);
if (v_o == NULL) {
JUMP_TO_LABEL(error);
}
+ if (PyLazyImport_CheckExact(v_o)) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ PyObject *l_v = _PyImport_LoadLazyImportTstate(tstate, v_o);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (l_v == NULL) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_DECREF(v_o);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ JUMP_TO_LABEL(error);
+ }
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ int err = PyDict_SetItem(GLOBALS(), name, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ if (err < 0) {
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_DECREF(v_o);
+ Py_DECREF(l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ JUMP_TO_LABEL(error);
+ }
+ _PyFrame_SetStackPointer(frame, stack_pointer);
+ Py_SETREF(v_o, l_v);
+ stack_pointer = _PyFrame_GetStackPointer(frame);
+ }
v = PyStackRef_FromPyObjectSteal(v_o);
stack_pointer[0] = v;
stack_pointer += 1;
#include "pycore_audit.h" // _PySys_Audit()
#include "pycore_ceval.h"
#include "pycore_critical_section.h" // Py_BEGIN_CRITICAL_SECTION()
+#include "pycore_dict.h" // _PyDict_Contains_KnownHash()
#include "pycore_hashtable.h" // _Py_hashtable_new_full()
#include "pycore_import.h" // _PyImport_BootstrapImp()
#include "pycore_initconfig.h" // _PyStatus_OK()
#include "pycore_interp.h" // struct _import_runtime_state
+#include "pycore_interpframe.h"
+#include "pycore_lazyimportobject.h"
+#include "pycore_long.h" // _PyLong_GetZero
#include "pycore_magic_number.h" // PYC_MAGIC_NUMBER_TOKEN
#include "pycore_moduleobject.h" // _PyModule_GetDef()
#include "pycore_namespace.h" // _PyNamespace_Type
#include "pycore_object.h" // _Py_SetImmortal()
+#include "pycore_pyatomic_ft_wrappers.h"
#include "pycore_pyerrors.h" // _PyErr_SetString()
#include "pycore_pyhash.h" // _Py_KeyedHash()
#include "pycore_pylifecycle.h"
#include "pycore_pymem.h" // _PyMem_DefaultRawFree()
#include "pycore_pystate.h" // _PyInterpreterState_GET()
+#include "pycore_setobject.h" // _PySet_NextEntry()
#include "pycore_sysmodule.h" // _PySys_ClearAttrString()
#include "pycore_time.h" // _PyTime_AsMicroseconds()
+#include "pycore_traceback.h"
#include "pycore_unicodeobject.h" // _PyUnicode_AsUTF8NoNUL()
#include "pycore_weakref.h" // _PyWeakref_GET_REF()
(interp)->imports.modules
#define MODULES_BY_INDEX(interp) \
(interp)->imports.modules_by_index
+#define LAZY_MODULES(interp) \
+ (interp)->imports.lazy_modules
#define IMPORTLIB(interp) \
(interp)->imports.importlib
#define OVERRIDE_MULTI_INTERP_EXTENSIONS_CHECK(interp) \
#define IMPORT_FUNC(interp) \
(interp)->imports.import_func
+#define LAZY_IMPORT_FUNC(interp) \
+ (interp)->imports.lazy_import_func
+
#define IMPORT_LOCK(interp) \
(interp)->imports.lock
#define FIND_AND_LOAD(interp) \
(interp)->imports.find_and_load
+#define LAZY_IMPORTS_MODE(interp) \
+ (interp)->imports.lazy_imports_mode
+
+#define LAZY_IMPORTS_FILTER(interp) \
+ (interp)->imports.lazy_imports_filter
+
+#ifdef Py_GIL_DISABLED
+#define LAZY_IMPORTS_LOCK(interp) PyMutex_Lock(&(interp)->imports.lazy_mutex)
+#define LAZY_IMPORTS_UNLOCK(interp) PyMutex_Unlock(&(interp)->imports.lazy_mutex)
+#else
+#define LAZY_IMPORTS_LOCK(interp)
+#define LAZY_IMPORTS_UNLOCK(interp)
+#endif
+
+
#define _IMPORT_TIME_HEADER(interp) \
do { \
if (FIND_AND_LOAD((interp)).header) { \
return m;
}
+PyObject *
+_PyImport_InitLazyModules(PyInterpreterState *interp)
+{
+ assert(LAZY_MODULES(interp) == NULL);
+ LAZY_MODULES(interp) = PyDict_New();
+ return LAZY_MODULES(interp);
+}
+
+void
+_PyImport_ClearLazyModules(PyInterpreterState *interp)
+{
+ Py_CLEAR(LAZY_MODULES(interp));
+}
+
static int
import_ensure_initialized(PyInterpreterState *interp, PyObject *mod, PyObject *name)
{
if (filename == NULL) {
return NULL;
}
- } else {
+ }
+ else {
filename = Py_NewRef(info->filename);
}
// XXX There's a refleak somewhere with the filename.
return -1;
}
IMPORT_FUNC(interp) = import_func;
+
+ // Get the __lazy_import__ function
+ if (PyDict_GetItemStringRef(interp->builtins, "__lazy_import__",
+ &import_func) <= 0) {
+ return -1;
+ }
+ LAZY_IMPORT_FUNC(interp) = import_func;
return 0;
}
return func == IMPORT_FUNC(interp);
}
+int
+_PyImport_IsDefaultLazyImportFunc(PyInterpreterState *interp, PyObject *func)
+{
+ return func == LAZY_IMPORT_FUNC(interp);
+}
/* Import a module, either built-in, frozen, or external, and return
its module object WITH INCREMENTED REFERENCE COUNT */
return NULL;
}
+PyObject *
+_PyImport_ResolveName(PyThreadState *tstate, PyObject *name,
+ PyObject *globals, int level)
+{
+ return resolve_name(tstate, name, globals, level);
+}
+
+PyObject *
+_PyImport_LoadLazyImportTstate(PyThreadState *tstate, PyObject *lazy_import)
+{
+ PyObject *obj = NULL;
+ PyObject *fromlist = Py_None;
+ PyObject *import_func = NULL;
+ assert(lazy_import != NULL);
+ assert(PyLazyImport_CheckExact(lazy_import));
+
+ PyLazyImportObject *lz = (PyLazyImportObject *)lazy_import;
+ PyInterpreterState *interp = tstate->interp;
+
+ // Acquire the global import lock to serialize reification
+ _PyImport_AcquireLock(interp);
+
+ // Check if we are already importing this module, if so, then we want to
+ // return an error that indicates we've hit a cycle which will indicate
+ // the value isn't yet available.
+ PyObject *importing = interp->imports.lazy_importing_modules;
+ if (importing == NULL) {
+ importing = interp->imports.lazy_importing_modules = PySet_New(NULL);
+ if (importing == NULL) {
+ _PyImport_ReleaseLock(interp);
+ return NULL;
+ }
+ }
+
+ assert(PyAnySet_CheckExact(importing));
+ int is_loading = _PySet_Contains((PySetObject *)importing, lazy_import);
+ if (is_loading < 0) {
+ _PyImport_ReleaseLock(interp);
+ return NULL;
+ }
+ else if (is_loading == 1) {
+ PyObject *name = _PyLazyImport_GetName(lazy_import);
+ if (name == NULL) {
+ _PyImport_ReleaseLock(interp);
+ return NULL;
+ }
+ PyObject *errmsg = PyUnicode_FromFormat(
+ "cannot import name %R (most likely due to a circular import)",
+ name);
+ if (errmsg == NULL) {
+ Py_DECREF(name);
+ _PyImport_ReleaseLock(interp);
+ return NULL;
+ }
+ PyErr_SetImportErrorSubclass(PyExc_ImportCycleError, errmsg,
+ lz->lz_from, NULL);
+ Py_DECREF(errmsg);
+ Py_DECREF(name);
+ _PyImport_ReleaseLock(interp);
+ return NULL;
+ }
+ else if (PySet_Add(importing, lazy_import) < 0) {
+ _PyImport_ReleaseLock(interp);
+ goto error;
+ }
+
+ Py_ssize_t dot = -1;
+ int full = 0;
+ if (lz->lz_attr != NULL) {
+ full = 1;
+ }
+ if (!full) {
+ dot = PyUnicode_FindChar(lz->lz_from, '.', 0,
+ PyUnicode_GET_LENGTH(lz->lz_from), 1);
+ }
+ if (dot < 0) {
+ full = 1;
+ }
+
+ if (lz->lz_attr != NULL) {
+ if (PyUnicode_Check(lz->lz_attr)) {
+ fromlist = PyTuple_New(1);
+ if (fromlist == NULL) {
+ goto error;
+ }
+ Py_INCREF(lz->lz_attr);
+ PyTuple_SET_ITEM(fromlist, 0, lz->lz_attr);
+ }
+ else {
+ Py_INCREF(lz->lz_attr);
+ fromlist = lz->lz_attr;
+ }
+ }
+
+ PyObject *globals = PyEval_GetGlobals();
+
+ if (PyMapping_GetOptionalItem(lz->lz_builtins, &_Py_ID(__import__),
+ &import_func) < 0) {
+ goto error;
+ }
+ if (import_func == NULL) {
+ PyErr_SetString(PyExc_ImportError, "__import__ not found");
+ goto error;
+ }
+ if (full) {
+ obj = _PyEval_ImportNameWithImport(
+ tstate, import_func, globals, globals,
+ lz->lz_from, fromlist, _PyLong_GetZero()
+ );
+ }
+ else {
+ PyObject *name = PyUnicode_Substring(lz->lz_from, 0, dot);
+ if (name == NULL) {
+ goto error;
+ }
+ obj = _PyEval_ImportNameWithImport(
+ tstate, import_func, globals, globals,
+ name, fromlist, _PyLong_GetZero()
+ );
+ Py_DECREF(name);
+ }
+ if (obj == NULL) {
+ goto error;
+ }
+
+ if (lz->lz_attr != NULL && PyUnicode_Check(lz->lz_attr)) {
+ PyObject *from = obj;
+ obj = _PyEval_ImportFrom(tstate, from, lz->lz_attr);
+ Py_DECREF(from);
+ if (obj == NULL) {
+ goto error;
+ }
+ }
+
+ assert(!PyLazyImport_CheckExact(obj));
+
+ goto ok;
+
+error:
+ Py_CLEAR(obj);
+
+ // If an error occurred and we have frame information, add it to the
+ // exception.
+ if (PyErr_Occurred() && lz->lz_code != NULL && lz->lz_instr_offset >= 0) {
+ // Get the current exception - this already has the full traceback
+ // from the access point.
+ PyObject *exc = _PyErr_GetRaisedException(tstate);
+
+ // Get import name - this can fail and set an exception.
+ PyObject *import_name = _PyLazyImport_GetName(lazy_import);
+ if (!import_name) {
+ // Failed to get import name, just restore original exception.
+ _PyErr_SetRaisedException(tstate, exc);
+ goto ok;
+ }
+
+ // Resolve line number from instruction offset on demand.
+ int lineno = PyCode_Addr2Line((PyCodeObject *)lz->lz_code,
+ lz->lz_instr_offset*2);
+
+ // Get strings - these can return NULL on encoding errors.
+ const char *filename_str = PyUnicode_AsUTF8(lz->lz_code->co_filename);
+ if (!filename_str) {
+ // Unicode conversion failed - clear error and restore original
+ // exception.
+ PyErr_Clear();
+ Py_DECREF(import_name);
+ _PyErr_SetRaisedException(tstate, exc);
+ goto ok;
+ }
+
+ const char *funcname_str = PyUnicode_AsUTF8(lz->lz_code->co_name);
+ if (!funcname_str) {
+ // Unicode conversion failed - clear error and restore original
+ // exception.
+ PyErr_Clear();
+ Py_DECREF(import_name);
+ _PyErr_SetRaisedException(tstate, exc);
+ goto ok;
+ }
+
+ // Create a cause exception showing where the lazy import was declared.
+ PyObject *msg = PyUnicode_FromFormat(
+ "deferred import of '%U' raised an exception during resolution",
+ import_name
+ );
+ Py_DECREF(import_name); // Done with import_name.
+
+ if (!msg) {
+ // Failed to create message - restore original exception.
+ _PyErr_SetRaisedException(tstate, exc);
+ goto ok;
+ }
+
+ PyObject *cause_exc = PyObject_CallOneArg(PyExc_ImportError, msg);
+ Py_DECREF(msg); // Done with msg.
+
+ if (!cause_exc) {
+ // Failed to create exception - restore original.
+ _PyErr_SetRaisedException(tstate, exc);
+ goto ok;
+ }
+
+ // Add traceback entry for the lazy import declaration.
+ _PyErr_SetRaisedException(tstate, cause_exc);
+ _PyTraceback_Add(funcname_str, filename_str, lineno);
+ PyObject *cause_with_tb = _PyErr_GetRaisedException(tstate);
+
+ // Set the cause on the original exception.
+ PyException_SetCause(exc, cause_with_tb); // Steals ref to cause_with_tb.
+
+ // Restore the original exception with its full traceback.
+ _PyErr_SetRaisedException(tstate, exc);
+ }
+
+ok:
+ if (PySet_Discard(importing, lazy_import) < 0) {
+ Py_CLEAR(obj);
+ }
+
+ // Release the global import lock.
+ _PyImport_ReleaseLock(interp);
+
+ Py_XDECREF(fromlist);
+ Py_XDECREF(import_func);
+ return obj;
+}
+
static PyObject *
import_find_and_load(PyThreadState *tstate, PyObject *abs_name)
{
#undef accumulated
}
+static PyObject *
+get_abs_name(PyThreadState *tstate, PyObject *name, PyObject *globals,
+ int level)
+{
+ if (level > 0) {
+ return resolve_name(tstate, name, globals, level);
+ }
+ if (PyUnicode_GET_LENGTH(name) == 0) {
+ _PyErr_SetString(tstate, PyExc_ValueError, "Empty module name");
+ return NULL;
+ }
+ return Py_NewRef(name);
+}
+
+PyObject *
+_PyImport_GetAbsName(PyThreadState *tstate, PyObject *name,
+ PyObject *globals, int level)
+{
+ return get_abs_name(tstate, name, globals, level);
+}
+
+
PyObject *
PyImport_ImportModuleLevelObject(PyObject *name, PyObject *globals,
PyObject *locals, PyObject *fromlist,
goto error;
}
- if (level > 0) {
- abs_name = resolve_name(tstate, name, globals, level);
- if (abs_name == NULL)
- goto error;
- }
- else { /* level == 0 */
- if (PyUnicode_GET_LENGTH(name) == 0) {
- _PyErr_SetString(tstate, PyExc_ValueError, "Empty module name");
- goto error;
- }
- abs_name = Py_NewRef(name);
+ abs_name = get_abs_name(tstate, name, globals, level);
+ if (abs_name == NULL) {
+ goto error;
}
mod = import_get_module(tstate, abs_name);
return final_mod;
}
+static PyObject *
+get_mod_dict(PyObject *module)
+{
+ if (PyModule_Check(module)) {
+ return Py_NewRef(_PyModule_GetDict(module));
+ }
+
+ return PyObject_GetAttr(module, &_Py_ID(__dict__));
+}
+
+// ensure we have the set for the parent module name in sys.lazy_modules.
+// Returns a new reference.
+static PyObject *
+ensure_lazy_submodules(PyDictObject *lazy_modules, PyObject *parent)
+{
+ PyObject *lazy_submodules;
+ Py_BEGIN_CRITICAL_SECTION(lazy_modules);
+ int err = _PyDict_GetItemRef_Unicode_LockHeld(lazy_modules, parent,
+ &lazy_submodules);
+ if (err == 0) {
+ // value isn't present
+ lazy_submodules = PySet_New(NULL);
+ if (lazy_submodules != NULL &&
+ _PyDict_SetItem_LockHeld(lazy_modules, parent,
+ lazy_submodules) < 0) {
+ Py_CLEAR(lazy_submodules);
+ }
+ }
+ Py_END_CRITICAL_SECTION();
+ return lazy_submodules;
+}
+
+static int
+register_lazy_on_parent(PyThreadState *tstate, PyObject *name,
+ PyObject *builtins)
+{
+ int ret = -1;
+ PyObject *parent = NULL;
+ PyObject *child = NULL;
+ PyObject *parent_module = NULL;
+ PyObject *parent_dict = NULL;
+
+ PyInterpreterState *interp = tstate->interp;
+ PyObject *lazy_modules = LAZY_MODULES(interp);
+ assert(lazy_modules != NULL);
+
+ Py_INCREF(name);
+ while (true) {
+ Py_ssize_t dot = PyUnicode_FindChar(name, '.', 0,
+ PyUnicode_GET_LENGTH(name), -1);
+ if (dot < 0) {
+ ret = 0;
+ goto done;
+ }
+ parent = PyUnicode_Substring(name, 0, dot);
+ // If `parent` is NULL then this has hit the end of the import, no
+ // more "parent.child" in the import name. The entire import will be
+ // resolved lazily.
+ if (parent == NULL) {
+ goto done;
+ }
+ Py_XDECREF(child);
+ child = PyUnicode_Substring(name, dot + 1, PyUnicode_GET_LENGTH(name));
+ if (child == NULL) {
+ goto done;
+ }
+
+ // Record the child as being lazily imported from the parent.
+ PyObject *lazy_submodules = ensure_lazy_submodules(
+ (PyDictObject *)lazy_modules, parent);
+ if (lazy_submodules == NULL) {
+ goto done;
+ }
+
+ if (PySet_Add(lazy_submodules, child) < 0) {
+ Py_DECREF(lazy_submodules);
+ goto done;
+ }
+ Py_DECREF(lazy_submodules);
+
+ // Add the lazy import for the child to the parent.
+ Py_XSETREF(parent_module, PyImport_GetModule(parent));
+ if (parent_module != NULL) {
+ Py_XSETREF(parent_dict, get_mod_dict(parent_module));
+ if (parent_dict == NULL) {
+ goto done;
+ }
+ if (PyDict_CheckExact(parent_dict)) {
+ int contains = PyDict_Contains(parent_dict, child);
+ if (contains < 0) {
+ goto done;
+ }
+ if (!contains) {
+ PyObject *lazy_module_attr = _PyLazyImport_New(
+ tstate->current_frame, builtins, parent, child
+ );
+ if (lazy_module_attr == NULL) {
+ goto done;
+ }
+ if (PyDict_SetItem(parent_dict, child,
+ lazy_module_attr) < 0) {
+ Py_DECREF(lazy_module_attr);
+ goto done;
+ }
+ Py_DECREF(lazy_module_attr);
+ }
+ }
+ ret = 0;
+ goto done;
+ }
+
+ Py_SETREF(name, parent);
+ parent = NULL;
+ }
+
+done:
+ Py_XDECREF(parent_dict);
+ Py_XDECREF(parent_module);
+ Py_XDECREF(child);
+ Py_XDECREF(parent);
+ Py_XDECREF(name);
+ return ret;
+}
+
+static int
+register_from_lazy_on_parent(PyThreadState *tstate, PyObject *abs_name,
+ PyObject *from, PyObject *builtins)
+{
+ PyObject *fromname = PyUnicode_FromFormat("%U.%U", abs_name, from);
+ if (fromname == NULL) {
+ return -1;
+ }
+ int res = register_lazy_on_parent(tstate, fromname, builtins);
+ Py_DECREF(fromname);
+ return res;
+}
+
+PyObject *
+_PyImport_LazyImportModuleLevelObject(PyThreadState *tstate,
+ PyObject *name, PyObject *builtins,
+ PyObject *globals, PyObject *locals,
+ PyObject *fromlist, int level)
+{
+ PyObject *abs_name = get_abs_name(tstate, name, globals, level);
+ if (abs_name == NULL) {
+ return NULL;
+ }
+
+ PyInterpreterState *interp = tstate->interp;
+ _PyInterpreterFrame *frame = _PyEval_GetFrame();
+ if (frame == NULL || frame->f_globals != frame->f_locals) {
+ Py_DECREF(abs_name);
+ PyErr_SetString(PyExc_SyntaxError,
+ "'lazy import' is only allowed at module level");
+ return NULL;
+ }
+
+ // Check if the filter disables the lazy import.
+ // We must hold a reference to the filter while calling it to prevent
+ // use-after-free if another thread replaces it via
+ // PyImport_SetLazyImportsFilter.
+ LAZY_IMPORTS_LOCK(interp);
+ PyObject *filter = Py_XNewRef(LAZY_IMPORTS_FILTER(interp));
+ LAZY_IMPORTS_UNLOCK(interp);
+
+ if (filter != NULL) {
+ PyObject *modname;
+ if (PyDict_GetItemRef(globals, &_Py_ID(__name__), &modname) < 0) {
+ Py_DECREF(filter);
+ Py_DECREF(abs_name);
+ return NULL;
+ }
+ if (modname == NULL) {
+ assert(!PyErr_Occurred());
+ modname = Py_NewRef(Py_None);
+ }
+ PyObject *args[] = {modname, name, fromlist};
+ PyObject *res = PyObject_Vectorcall(filter, args, 3, NULL);
+
+ Py_DECREF(modname);
+ Py_DECREF(filter);
+
+ if (res == NULL) {
+ Py_DECREF(abs_name);
+ return NULL;
+ }
+
+ int is_true = PyObject_IsTrue(res);
+ Py_DECREF(res);
+
+ if (is_true < 0) {
+ Py_DECREF(abs_name);
+ return NULL;
+ }
+ if (!is_true) {
+ Py_DECREF(abs_name);
+ return PyImport_ImportModuleLevelObject(
+ name, globals, locals, fromlist, level
+ );
+ }
+ }
+
+ // here, 'filter' is either NULL or is equivalent to a borrowed reference
+ PyObject *res = _PyLazyImport_New(frame, builtins, abs_name, fromlist);
+ if (res == NULL) {
+ Py_DECREF(abs_name);
+ return NULL;
+ }
+ if (fromlist && PyUnicode_Check(fromlist)) {
+ if (register_from_lazy_on_parent(tstate, abs_name, fromlist,
+ builtins) < 0) {
+ goto error;
+ }
+ }
+ else if (fromlist && PyTuple_Check(fromlist) &&
+ PyTuple_GET_SIZE(fromlist)) {
+ for (Py_ssize_t i = 0; i < PyTuple_GET_SIZE(fromlist); i++) {
+ if (register_from_lazy_on_parent(tstate, abs_name,
+ PyTuple_GET_ITEM(fromlist, i),
+ builtins) < 0)
+ {
+ goto error;
+ }
+ }
+ }
+ else if (register_lazy_on_parent(tstate, abs_name, builtins) < 0) {
+ goto error;
+ }
+
+ Py_DECREF(abs_name);
+ return res;
+error:
+ Py_DECREF(abs_name);
+ Py_DECREF(res);
+ return NULL;
+}
+
PyObject *
PyImport_ImportModuleLevel(const char *name, PyObject *globals, PyObject *locals,
PyObject *fromlist, int level)
Py_CLEAR(MODULES_BY_INDEX(interp));
Py_CLEAR(IMPORTLIB(interp));
Py_CLEAR(IMPORT_FUNC(interp));
+ Py_CLEAR(LAZY_IMPORT_FUNC(interp));
+ Py_CLEAR(interp->imports.lazy_modules);
+ Py_CLEAR(interp->imports.lazy_importing_modules);
+ Py_CLEAR(interp->imports.lazy_imports_filter);
}
void
return result;
}
+int
+PyImport_SetLazyImportsFilter(PyObject *filter)
+{
+ if (filter == Py_None) {
+ filter = NULL;
+ }
+ if (filter != NULL && !PyCallable_Check(filter)) {
+ PyErr_SetString(PyExc_ValueError,
+ "filter provided but is not callable");
+ return -1;
+ }
+
+ PyInterpreterState *interp = _PyInterpreterState_GET();
+ // Exchange the filter w/ the lock held. We can't use Py_XSETREF
+ // because we need to release the lock before the decref.
+ LAZY_IMPORTS_LOCK(interp);
+ PyObject *old = LAZY_IMPORTS_FILTER(interp);
+ LAZY_IMPORTS_FILTER(interp) = Py_XNewRef(filter);
+ LAZY_IMPORTS_UNLOCK(interp);
+ Py_XDECREF(old);
+ return 0;
+}
+
+/* Return a strong reference to the current lazy imports filter
+ * or NULL if none exists. This function always succeeds.
+ */
+PyObject *
+PyImport_GetLazyImportsFilter(void)
+{
+ PyInterpreterState *interp = _PyInterpreterState_GET();
+ LAZY_IMPORTS_LOCK(interp);
+ PyObject *res = Py_XNewRef(LAZY_IMPORTS_FILTER(interp));
+ LAZY_IMPORTS_UNLOCK(interp);
+ return res;
+}
+
+int
+PyImport_SetLazyImportsMode(PyImport_LazyImportsMode mode)
+{
+ PyInterpreterState *interp = _PyInterpreterState_GET();
+ FT_ATOMIC_STORE_INT_RELAXED(LAZY_IMPORTS_MODE(interp), mode);
+ return 0;
+}
+
+/* Checks if lazy imports is globally enabled or disabled. Return 1 when
+ * globally forced on, 0 when globally forced off, or -1 when not set.*/
+PyImport_LazyImportsMode
+PyImport_GetLazyImportsMode(void)
+{
+ PyInterpreterState *interp = _PyInterpreterState_GET();
+ return FT_ATOMIC_LOAD_INT_RELAXED(LAZY_IMPORTS_MODE(interp));
+}
/**************/
/* the module */
return PyBytes_FromStringAndSize(hash.data, sizeof(hash.data));
}
+static int
+publish_lazy_imports_on_module(PyThreadState *tstate,
+ PyObject *lazy_submodules,
+ PyObject *name,
+ PyObject *module_dict)
+{
+ PyObject *builtins = _PyEval_GetBuiltins(tstate);
+ PyObject *attr_name;
+ Py_ssize_t pos = 0;
+ Py_hash_t hash;
+
+ // Enumerate the set of lazy submodules which have been imported from the
+ // parent module.
+ while (_PySet_NextEntryRef(lazy_submodules, &pos, &attr_name, &hash)) {
+ if (_PyDict_Contains_KnownHash(module_dict, attr_name, hash)) {
+ Py_DECREF(attr_name);
+ continue;
+ }
+ // Create a new lazy module attr for the subpackage which was
+ // previously lazily imported.
+ PyObject *lazy_module_attr = _PyLazyImport_New(tstate->current_frame, builtins,
+ name, attr_name);
+ if (lazy_module_attr == NULL) {
+ Py_DECREF(attr_name);
+ return -1;
+ }
+
+ // Publish on the module that was just imported.
+ if (PyDict_SetItem(module_dict, attr_name,
+ lazy_module_attr) < 0) {
+ Py_DECREF(lazy_module_attr);
+ Py_DECREF(attr_name);
+ return -1;
+ }
+ Py_DECREF(lazy_module_attr);
+ Py_DECREF(attr_name);
+ }
+ return 0;
+}
+
+/*[clinic input]
+_imp._set_lazy_attributes
+ modobj: object
+ name: unicode
+ /
+Sets attributes to lazy submodules on the module, as side effects.
+[clinic start generated code]*/
+
+static PyObject *
+_imp__set_lazy_attributes_impl(PyObject *module, PyObject *modobj,
+ PyObject *name)
+/*[clinic end generated code: output=3369bb3242b1f043 input=38ea6f30956dd7d6]*/
+{
+ PyThreadState *tstate = _PyThreadState_GET();
+ PyObject *module_dict = NULL;
+ PyObject *ret = NULL;
+ PyObject *lazy_modules = LAZY_MODULES(tstate->interp);
+ assert(lazy_modules != NULL);
+
+ PyObject *lazy_submodules;
+ if (PyDict_GetItemRef(lazy_modules, name, &lazy_submodules) < 0) {
+ return NULL;
+ }
+ else if (lazy_submodules == NULL) {
+ Py_RETURN_NONE;
+ }
+
+ module_dict = get_mod_dict(modobj);
+ if (module_dict == NULL || !PyDict_CheckExact(module_dict)) {
+ goto done;
+ }
+
+ assert(PyAnySet_CheckExact(lazy_submodules));
+ Py_BEGIN_CRITICAL_SECTION(lazy_submodules);
+ publish_lazy_imports_on_module(tstate, lazy_submodules, name, module_dict);
+ Py_END_CRITICAL_SECTION();
+ Py_DECREF(lazy_submodules);
+
+ // once a module is imported it is removed from sys.lazy_modules
+ if (PyDict_DelItem(lazy_modules, name) < 0) {
+ goto error;
+ }
+
+done:
+ ret = Py_NewRef(Py_None);
+
+error:
+ Py_XDECREF(module_dict);
+ return ret;
+}
PyDoc_STRVAR(doc_imp,
"(Extremely) low-level import machinery bits as used by importlib.");
_IMP_EXEC_BUILTIN_METHODDEF
_IMP__FIX_CO_FILENAME_METHODDEF
_IMP_SOURCE_HASH_METHODDEF
+ _IMP__SET_LAZY_ATTRIBUTES_METHODDEF
{NULL, NULL} /* sentinel */
};
SPEC(base_prefix, WSTR_OPT, PUBLIC, SYS_ATTR("base_prefix")),
SPEC(bytes_warning, UINT, PUBLIC, SYS_FLAG(9)),
SPEC(cpu_count, INT, PUBLIC, NO_SYS),
+ SPEC(lazy_imports, INT, PUBLIC, NO_SYS),
SPEC(exec_prefix, WSTR_OPT, PUBLIC, SYS_ATTR("exec_prefix")),
SPEC(executable, WSTR_OPT, PUBLIC, SYS_ATTR("executable")),
SPEC(inspect, BOOL, PUBLIC, SYS_FLAG(1)),
"\
-X importtime[=2]: show how long each import takes; use -X importtime=2 to\n\
log imports of already-loaded modules; also PYTHONPROFILEIMPORTTIME\n\
+-X lazy_imports=[all|none|normal]: control global lazy imports;\n\
+ default is normal; also PYTHON_LAZY_IMPORTS\n\
-X int_max_str_digits=N: limit the size of int<->str conversions;\n\
0 disables the limit; also PYTHONINTMAXSTRDIGITS\n\
-X no_debug_ranges: don't include extra location information in code objects;\n\
"PYTHON_PRESITE: import this module before site (-X presite)\n"
#endif
"PYTHONPROFILEIMPORTTIME: show how long each import takes (-X importtime)\n"
+"PYTHON_LAZY_IMPORTS: control global lazy imports (-X lazy_imports)\n"
"PYTHONPYCACHEPREFIX: root directory for bytecode cache (pyc) files\n"
" (-X pycache_prefix)\n"
"PYTHONSAFEPATH : don't prepend a potentially unsafe path to sys.path.\n"
assert(config->int_max_str_digits >= 0);
// cpu_count can be -1 if the user doesn't override it.
assert(config->cpu_count != 0);
+ // lazy_imports can be -1 (default), 0 (off), or 1 (on).
+ assert(config->lazy_imports >= -1 && config->lazy_imports <= 1);
// config->use_frozen_modules is initialized later
// by _PyConfig_InitImportConfig().
assert(config->thread_inherit_context >= 0);
config->_is_python_build = 0;
config->code_debug_ranges = 1;
config->cpu_count = -1;
+ config->lazy_imports = -1;
#ifdef Py_GIL_DISABLED
config->thread_inherit_context = 1;
config->context_aware_warnings = 1;
return _PyStatus_OK();
}
+static PyStatus
+config_init_lazy_imports(PyConfig *config)
+{
+ int lazy_imports = -1;
+
+ const char *env = config_get_env(config, "PYTHON_LAZY_IMPORTS");
+ if (env) {
+ if (strcmp(env, "all") == 0) {
+ lazy_imports = 1;
+ }
+ else if (strcmp(env, "none") == 0) {
+ lazy_imports = 0;
+ }
+ else if (strcmp(env, "normal") == 0) {
+ lazy_imports = -1;
+ }
+ else {
+ return _PyStatus_ERR("PYTHON_LAZY_IMPORTS: invalid value; "
+ "expected 'all', 'none', or 'normal'");
+ }
+ config->lazy_imports = lazy_imports;
+ }
+
+ const wchar_t *x_value = config_get_xoption_value(config, L"lazy_imports");
+ if (x_value) {
+ if (wcscmp(x_value, L"all") == 0) {
+ lazy_imports = 1;
+ }
+ else if (wcscmp(x_value, L"none") == 0) {
+ lazy_imports = 0;
+ }
+ else if (wcscmp(x_value, L"normal") == 0) {
+ lazy_imports = -1;
+ }
+ else {
+ return _PyStatus_ERR("-X lazy_imports: invalid value; "
+ "expected 'all', 'none', or 'normal'");
+ }
+ config->lazy_imports = lazy_imports;
+ }
+ return _PyStatus_OK();
+}
+
static PyStatus
config_read_complex_options(PyConfig *config)
{
}
}
+ if (config->lazy_imports < 0) {
+ status = config_init_lazy_imports(config);
+ if (_PyStatus_EXCEPTION(status)) {
+ return status;
+ }
+ }
+
if (config->tracemalloc < 0) {
status = config_init_tracemalloc(config);
if (_PyStatus_EXCEPTION(status)) {
if (config->tracemalloc < 0) {
config->tracemalloc = 0;
}
+ if (config->lazy_imports < 0) {
+ config->lazy_imports = -1; // Default is auto/unset
+ }
if (config->perf_profiling < 0) {
config->perf_profiling = 0;
}
#include "pycore_floatobject.h"
#include "pycore_frame.h"
#include "pycore_function.h"
+#include "pycore_import.h"
#include "pycore_interpframe.h"
#include "pycore_interpolation.h"
#include "pycore_intrinsics.h"
+#include "pycore_lazyimportobject.h"
#include "pycore_list.h"
#include "pycore_long.h"
#include "pycore_mmap.h"
#include "pycore_initconfig.h" // _PyStatus_OK()
#include "pycore_interpolation.h" // _PyInterpolation_InitTypes()
#include "pycore_long.h" // _PyLong_InitTypes()
+#include "pycore_moduleobject.h" // _PyModule_InitModuleDictWatcher()
#include "pycore_object.h" // _PyDebug_PrintTotalRefs()
#include "pycore_obmalloc.h" // _PyMem_init_obmalloc()
#include "pycore_optimizer.h" // _Py_Executors_InvalidateAll
_PyInterpreterState_SetWhence(interp, _PyInterpreterState_WHENCE_RUNTIME);
interp->_ready = 1;
+ /* Initialize the module dict watcher early, before any modules are created */
+ if (_PyModule_InitModuleDictWatcher(interp) != 0) {
+ return _PyStatus_ERR("failed to initialize module dict watcher");
+ }
+
status = _PyConfig_Copy(&interp->config, src_config);
if (_PyStatus_EXCEPTION(status)) {
return status;
}
}
+ // Initialize lazy imports based on configuration. Do this after site
+ // module is imported to avoid circular imports during startup.
+ if (config->lazy_imports != -1) {
+ PyImport_LazyImportsMode lazy_mode;
+ if (config->lazy_imports == 1) {
+ lazy_mode = PyImport_LAZY_ALL;
+ }
+ else {
+ lazy_mode = PyImport_LAZY_NONE;
+ }
+ if (PyImport_SetLazyImportsMode(lazy_mode) < 0) {
+ return _PyStatus_ERR("failed to set lazy imports mode");
+ }
+ }
+ // If config->lazy_imports == -1, use the default mode, no change needed.
+
if (is_main_interp) {
#ifndef MS_WINDOWS
emit_stderr_warning_for_legacy_locale(interp->runtime);
// initialization API)
_PyImport_ClearModulesByIndex(interp);
+ // Clear the dict of lazily loaded module nname to submodule names
+ _PyImport_ClearLazyModules(interp);
+
// Clear and delete the modules directory. Actual modules will
// still be there only if imported during the execution of some
// destructor.
_PyInterpreterState_SetWhence(interp, whence);
interp->_ready = 1;
+ /* Initialize the module dict watcher early, before any modules are created */
+ if (_PyModule_InitModuleDictWatcher(interp) != 0) {
+ goto error;
+ }
+
/* From this point until the init_interp_create_gil() call,
we must not do anything that requires that the GIL be held
(or otherwise exist). That applies whether or not the new
#include "pycore_dict.h" // DICT_KEYS_UNICODE
#include "pycore_function.h" // _PyFunction_GetVersionForCurrentState()
#include "pycore_interpframe.h" // FRAME_SPECIALS_SIZE
+#include "pycore_lazyimportobject.h" // PyLazyImport_CheckExact
#include "pycore_list.h" // _PyListIterObject
#include "pycore_long.h" // _PyLong_IsNonNegativeCompact()
#include "pycore_moduleobject.h"
#define SPEC_FAIL_ATTR_BUILTIN_CLASS_METHOD 22
#define SPEC_FAIL_ATTR_CLASS_METHOD_OBJ 23
#define SPEC_FAIL_ATTR_OBJECT_SLOT 24
+#define SPEC_FAIL_ATTR_MODULE_LAZY_VALUE 25
#define SPEC_FAIL_ATTR_INSTANCE_ATTRIBUTE 26
#define SPEC_FAIL_ATTR_METACLASS_ATTRIBUTE 27
}
PyObject *value;
Py_ssize_t index = _PyDict_LookupIndexAndValue(dict, name, &value);
+ if (value != NULL && PyLazyImport_CheckExact(value)) {
+ SPECIALIZATION_FAIL(LOAD_ATTR, SPEC_FAIL_ATTR_MODULE_LAZY_VALUE);
+ return -1;
+ }
assert(index != DKIX_ERROR);
if (index != (uint16_t)index) {
SPECIALIZATION_FAIL(LOAD_ATTR,
SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_LOAD_GLOBAL_NON_STRING_OR_SPLIT);
goto fail;
}
-#ifdef Py_GIL_DISABLED
PyObject *value;
Py_ssize_t index = _PyDict_LookupIndexAndValue((PyDictObject *)globals, name, &value);
-#else
- Py_ssize_t index = _PyDictKeys_StringLookup(globals_keys, name);
-#endif
if (index == DKIX_ERROR) {
SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_EXPECTED_ERROR);
goto fail;
}
+ if (value != NULL && PyLazyImport_CheckExact(value)) {
+ SPECIALIZATION_FAIL(LOAD_GLOBAL, SPEC_FAIL_ATTR_MODULE_LAZY_VALUE);
+ Py_DECREF(value);
+ goto fail;
+ }
PyInterpreterState *interp = _PyInterpreterState_GET();
if (index != DKIX_EMPTY) {
if (index != (uint16_t)index) {
ste->ste_needs_classdict = 0;
ste->ste_has_conditional_annotations = 0;
ste->ste_in_conditional_block = 0;
+ ste->ste_in_try_block = 0;
ste->ste_in_unevaluated_annotation = 0;
ste->ste_annotation_block = NULL;
#define LEAVE_CONDITIONAL_BLOCK(ST) \
(ST)->st_cur->ste_in_conditional_block = in_conditional_block;
+#define ENTER_TRY_BLOCK(ST) \
+ int in_try_block = (ST)->st_cur->ste_in_try_block; \
+ (ST)->st_cur->ste_in_try_block = 1;
+
+#define LEAVE_TRY_BLOCK(ST) \
+ (ST)->st_cur->ste_in_try_block = in_try_block;
+
#define ENTER_RECURSIVE() \
if (Py_EnterRecursiveCall(" during compilation")) { \
return 0; \
return 1;
}
+static int
+check_lazy_import_context(struct symtable *st, stmt_ty s,
+ const char* import_type)
+{
+ // Check if inside try/except block.
+ if (st->st_cur->ste_in_try_block) {
+ PyErr_Format(PyExc_SyntaxError,
+ "lazy %s not allowed inside try/except blocks",
+ import_type);
+ SET_ERROR_LOCATION(st->st_filename, LOCATION(s));
+ return 0;
+ }
+
+ // Check if inside function scope.
+ if (st->st_cur->ste_type == FunctionBlock) {
+ PyErr_Format(PyExc_SyntaxError,
+ "lazy %s not allowed inside functions", import_type);
+ SET_ERROR_LOCATION(st->st_filename, LOCATION(s));
+ return 0;
+ }
+
+ // Check if inside class scope.
+ if (st->st_cur->ste_type == ClassBlock) {
+ PyErr_Format(PyExc_SyntaxError,
+ "lazy %s not allowed inside classes", import_type);
+ SET_ERROR_LOCATION(st->st_filename, LOCATION(s));
+ return 0;
+ }
+
+ return 1;
+}
+
static bool
allows_top_level_await(struct symtable *st)
{
break;
case Try_kind: {
ENTER_CONDITIONAL_BLOCK(st);
+ ENTER_TRY_BLOCK(st);
VISIT_SEQ(st, stmt, s->v.Try.body);
VISIT_SEQ(st, excepthandler, s->v.Try.handlers);
VISIT_SEQ(st, stmt, s->v.Try.orelse);
VISIT_SEQ(st, stmt, s->v.Try.finalbody);
+ LEAVE_TRY_BLOCK(st);
LEAVE_CONDITIONAL_BLOCK(st);
break;
}
case TryStar_kind: {
ENTER_CONDITIONAL_BLOCK(st);
+ ENTER_TRY_BLOCK(st);
VISIT_SEQ(st, stmt, s->v.TryStar.body);
VISIT_SEQ(st, excepthandler, s->v.TryStar.handlers);
VISIT_SEQ(st, stmt, s->v.TryStar.orelse);
VISIT_SEQ(st, stmt, s->v.TryStar.finalbody);
+ LEAVE_TRY_BLOCK(st);
LEAVE_CONDITIONAL_BLOCK(st);
break;
}
VISIT(st, expr, s->v.Assert.msg);
break;
case Import_kind:
+ if (s->v.Import.is_lazy) {
+ if (!check_lazy_import_context(st, s, "import")) {
+ return 0;
+ }
+ }
VISIT_SEQ(st, alias, s->v.Import.names);
break;
case ImportFrom_kind:
+ if (s->v.ImportFrom.is_lazy) {
+ if (!check_lazy_import_context(st, s, "from ... import")) {
+ return 0;
+ }
+
+ // Check for import *
+ for (Py_ssize_t i = 0; i < asdl_seq_LEN(s->v.ImportFrom.names);
+ i++) {
+ alias_ty alias = (alias_ty)asdl_seq_GET(
+ s->v.ImportFrom.names, i);
+ if (alias->name &&
+ _PyUnicode_EqualToASCIIString(alias->name, "*")) {
+ PyErr_SetString(PyExc_SyntaxError,
+ "lazy from ... import * is not allowed");
+ SET_ERROR_LOCATION(st->st_filename, LOCATION(s));
+ return 0;
+ }
+ }
+ }
VISIT_SEQ(st, alias, s->v.ImportFrom.names);
if (!check_import_from(st, s)) {
return 0;
return 0;
}
+/*[clinic input]
+sys.set_lazy_imports_filter
+
+ filter: object
+
+Set the lazy imports filter callback.
+
+The filter is a callable which disables lazy imports when they
+would otherwise be enabled. Returns True if the import is still enabled
+or False to disable it. The callable is called with:
+
+(importing_module_name, imported_module_name, [fromlist])
+
+Pass None to clear the filter.
+[clinic start generated code]*/
+
+static PyObject *
+sys_set_lazy_imports_filter_impl(PyObject *module, PyObject *filter)
+/*[clinic end generated code: output=10251d49469c278c input=2eb48786bdd4ee42]*/
+{
+ if (PyImport_SetLazyImportsFilter(filter) < 0) {
+ return NULL;
+ }
+
+ Py_RETURN_NONE;
+}
+
+/*[clinic input]
+sys.get_lazy_imports_filter
+
+Get the current lazy imports filter callback.
+
+Returns the filter callable or None if no filter is set.
+[clinic start generated code]*/
+
+static PyObject *
+sys_get_lazy_imports_filter_impl(PyObject *module)
+/*[clinic end generated code: output=3bf73022892165af input=cf1e07cb8e203c94]*/
+{
+ PyObject *filter = PyImport_GetLazyImportsFilter();
+ if (filter == NULL) {
+ assert(!PyErr_Occurred());
+ Py_RETURN_NONE;
+ }
+ return filter;
+}
+
+/*[clinic input]
+sys.set_lazy_imports
+
+ mode: object
+
+Sets the global lazy imports mode.
+
+The mode parameter must be one of the following strings:
+- "all": All top-level imports become potentially lazy
+- "none": All lazy imports are suppressed (even explicitly marked ones)
+- "normal": Only explicitly marked imports (with 'lazy' keyword) are lazy
+
+In addition to the mode, lazy imports can be controlled via the filter
+provided to sys.set_lazy_imports_filter
+
+[clinic start generated code]*/
+
+static PyObject *
+sys_set_lazy_imports_impl(PyObject *module, PyObject *mode)
+/*[clinic end generated code: output=1ff34ba6c4feaf73 input=f04e70d8bf9fe4f6]*/
+{
+ PyImport_LazyImportsMode lazy_mode;
+ if (!PyUnicode_Check(mode)) {
+ PyErr_SetString(PyExc_TypeError,
+ "mode must be a string: 'normal', 'all', or 'none'");
+ return NULL;
+ }
+ if (PyUnicode_CompareWithASCIIString(mode, "normal") == 0) {
+ lazy_mode = PyImport_LAZY_NORMAL;
+ }
+ else if (PyUnicode_CompareWithASCIIString(mode, "all") == 0) {
+ lazy_mode = PyImport_LAZY_ALL;
+ }
+ else if (PyUnicode_CompareWithASCIIString(mode, "none") == 0) {
+ lazy_mode = PyImport_LAZY_NONE;
+ }
+ else {
+ PyErr_SetString(PyExc_ValueError,
+ "mode must be 'normal', 'all', or 'none'");
+ return NULL;
+ }
+
+ if (PyImport_SetLazyImportsMode(lazy_mode)) {
+ return NULL;
+ }
+ Py_RETURN_NONE;
+}
+
+/*[clinic input]
+sys.get_lazy_imports
+
+Gets the global lazy imports mode.
+
+Returns "all" if all top level imports are potentially lazy.
+Returns "none" if all explicitly marked lazy imports are suppressed.
+Returns "normal" if only explicitly marked imports are lazy.
+
+[clinic start generated code]*/
+
+static PyObject *
+sys_get_lazy_imports_impl(PyObject *module)
+/*[clinic end generated code: output=4147dec48c51ae99 input=8cb574f1e4e3003c]*/
+{
+ switch (PyImport_GetLazyImportsMode()) {
+ case PyImport_LAZY_NORMAL:
+ return PyUnicode_FromString("normal");
+ case PyImport_LAZY_ALL:
+ return PyUnicode_FromString("all");
+ case PyImport_LAZY_NONE:
+ return PyUnicode_FromString("none");
+ default:
+ PyErr_SetString(PyExc_RuntimeError, "unknown lazy imports mode");
+ return NULL;
+ }
+}
static PyMethodDef sys_methods[] = {
/* Might as well keep this in alphabetic order */
SYS_UNRAISABLEHOOK_METHODDEF
SYS_GET_INT_MAX_STR_DIGITS_METHODDEF
SYS_SET_INT_MAX_STR_DIGITS_METHODDEF
+ SYS_GET_LAZY_IMPORTS_METHODDEF
+ SYS_SET_LAZY_IMPORTS_METHODDEF
+ SYS_GET_LAZY_IMPORTS_FILTER_METHODDEF
+ SYS_SET_LAZY_IMPORTS_FILTER_METHODDEF
SYS__BASEREPL_METHODDEF
#ifdef Py_STATS
SYS__STATS_ON_METHODDEF
{"gil", "-X gil"},
{"thread_inherit_context", "-X thread_inherit_context"},
{"context_aware_warnings", "-X context_aware_warnings"},
+ {"lazy_imports", "-X lazy_imports"},
{0}
};
"sys.flags", /* name */
flags__doc__, /* doc */
flags_fields, /* fields */
- 18
+ 19
};
static void
#endif
SetFlag(config->thread_inherit_context);
SetFlag(config->context_aware_warnings);
+ SetFlag(config->lazy_imports);
#undef SetFlagObj
#undef SetFlag
return 0;
goto error;
}
+ PyObject *lazy_modules = _PyImport_InitLazyModules(interp); // borrowed reference
+ if (lazy_modules == NULL) {
+ goto error;
+ }
+
+ if (PyDict_SetItemString(sysdict, "lazy_modules", lazy_modules) < 0) {
+ goto error;
+ }
+
PyStatus status = _PySys_SetPreliminaryStderr(sysdict);
if (_PyStatus_EXCEPTION(status)) {
return status;
Objects/iterobject.c - PyCallIter_Type -
Objects/iterobject.c - PySeqIter_Type -
Objects/iterobject.c - _PyAnextAwaitable_Type -
+Objects/lazyimportobject.c - PyLazyImport_Type -
Objects/listobject.c - PyListIter_Type -
Objects/listobject.c - PyListRevIter_Type -
Objects/listobject.c - PyList_Type -
Objects/exceptions.c - _PyExc_GeneratorExit -
Objects/exceptions.c - _PyExc_SystemExit -
Objects/exceptions.c - _PyExc_KeyboardInterrupt -
+Objects/exceptions.c - _PyExc_ImportCycleError -
Objects/exceptions.c - _PyExc_ImportError -
Objects/exceptions.c - _PyExc_ModuleNotFoundError -
Objects/exceptions.c - _PyExc_OSError -
Objects/exceptions.c - PyExc_GeneratorExit -
Objects/exceptions.c - PyExc_SystemExit -
Objects/exceptions.c - PyExc_KeyboardInterrupt -
+Objects/exceptions.c - PyExc_ImportCycleError -
Objects/exceptions.c - PyExc_ImportError -
Objects/exceptions.c - PyExc_ModuleNotFoundError -
Objects/exceptions.c - PyExc_OSError -
#include "pycore_frame.h"
#include "pycore_function.h"
#include "pycore_genobject.h"
+#include "pycore_import.h"
#include "pycore_interpframe.h"
#include "pycore_interpolation.h"
#include "pycore_intrinsics.h"
+#include "pycore_lazyimportobject.h"
#include "pycore_jit.h"
#include "pycore_list.h"
#include "pycore_long.h"
projects / thirdparty / Python / cpython.git / commitdiff
