gh-70870: Clarify dual usage of 'free variable' (#122545)

The term "free variable" has unfortunately become genuinely
ambiguous over the years (presumably due to the names of
some relevant code object instance attributes).

While we can't eliminate that ambiguity at this late date, we can
at least alert people to the potential ambiguity by describing
both the formal meaning of the term and the common
alternative use as a direct synonym for "closure variable".

---------

Co-authored-by: Carol Willing <carolcode@willingconsulting.com>

diff --git a/Doc/c-api/code.rst b/Doc/c-api/code.rst
index 6ae6bfe4aa6ab44d5695289fca0644216a4b34ff..6eae24b38fae48a4b35dedef19cf6e3ad4d3a7ff 100644 (file)
--- a/Doc/c-api/code.rst
+++ b/Doc/c-api/code.rst
@@ -32,11 +32,13 @@ bound into a function.
 
 .. c:function:: Py_ssize_t PyCode_GetNumFree(PyCodeObject *co)
 
-   Return the number of free variables in a code object.
+   Return the number of :term:`free (closure) variables <closure variable>`
+   in a code object.
 
 .. c:function:: int PyUnstable_Code_GetFirstFree(PyCodeObject *co)
 
-   Return the position of the first free variable in a code object.
+   Return the position of the first :term:`free (closure) variable <closure variable>`
+   in a code object.
 
    .. versionchanged:: 3.13
 
@@ -144,7 +146,8 @@ bound into a function.
 
    Equivalent to the Python code ``getattr(co, 'co_freevars')``.
    Returns a new reference to a :c:type:`PyTupleObject` containing the names of
-   the free variables. On error, ``NULL`` is returned and an exception is raised.
+   the :term:`free (closure) variables <closure variable>`. On error, ``NULL`` is returned
+   and an exception is raised.
 
    .. versionadded:: 3.11
 

diff --git a/Doc/glossary.rst b/Doc/glossary.rst
index e72a8d002d507db122607916e2fc2b14aa35a366..933fb0319452a6e313036ecfd4e2fc97791ba076 100644 (file)
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -231,6 +231,28 @@ Glossary
       A variable defined in a class and intended to be modified only at
       class level (i.e., not in an instance of the class).
 
+   closure variable
+      A :term:`free variable` referenced from a :term:`nested scope` that is defined in an outer
+      scope rather than being resolved at runtime from the globals or builtin namespaces.
+      May be explicitly defined with the :keyword:`nonlocal` keyword to allow write access,
+      or implicitly defined if the variable is only being read.
+
+      For example, in the ``inner`` function in the following code, both ``x`` and ``print`` are
+      :term:`free variables <free variable>`, but only ``x`` is a *closure variable*::
+
+          def outer():
+              x = 0
+              def inner():
+                  nonlocal x
+                  x += 1
+                  print(x)
+              return inner
+
+      Due to the :attr:`codeobject.co_freevars` attribute (which, despite its name, only
+      includes the names of closure variables rather than listing all referenced free
+      variables), the more general :term:`free variable` term is sometimes used even
+      when the intended meaning is to refer specifically to closure variables.
+
    complex number
       An extension of the familiar real number system in which all numbers are
       expressed as a sum of a real part and an imaginary part.  Imaginary
@@ -454,6 +476,13 @@ Glossary
       the :term:`global interpreter lock` which allows only one thread to
       execute Python bytecode at a time.  See :pep:`703`.
 
+   free variable
+      Formally, as defined in the :ref:`language execution model <bind_names>`, a free
+      variable is any variable used in a namespace which is not a local variable in that
+      namespace. See :term:`closure variable` for an example.
+      Pragmatically, due to the name of the :attr:`codeobject.co_freevars` attribute,
+      the term is also sometimes used as a synonym for :term:`closure variable`.
+
    function
       A series of statements which returns some value to a caller. It can also
       be passed zero or more :term:`arguments <argument>` which may be used in

diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst
index e3919c2ffad84cf3a80b790673f183a85823cd09..75b84a8d827bc9c032f47c876efcb50519e704bf 100644 (file)
--- a/Doc/library/dis.rst
+++ b/Doc/library/dis.rst
@@ -1434,7 +1434,7 @@ iterations of the loop.
    slot ``i`` of the "fast locals" storage in this mapping.
    If the name is not found there, loads it from the cell contained in
    slot ``i``, similar to :opcode:`LOAD_DEREF`. This is used for loading
-   free variables in class bodies (which previously used
+   :term:`closure variables <closure variable>` in class bodies (which previously used
    :opcode:`!LOAD_CLASSDEREF`) and in
    :ref:`annotation scopes <annotation-scopes>` within class bodies.
 
@@ -1463,8 +1463,8 @@ iterations of the loop.
 
 .. opcode:: COPY_FREE_VARS (n)
 
-   Copies the ``n`` free variables from the closure into the frame.
-   Removes the need for special code on the caller's side when calling
+   Copies the ``n`` :term:`free (closure) variables <closure variable>` from the closure
+   into the frame. Removes the need for special code on the caller's side when calling
    closures.
 
    .. versionadded:: 3.11
@@ -1937,10 +1937,10 @@ instructions:
 
 .. data:: hasfree
 
-   Sequence of bytecodes that access a free variable. 'free' in this
-   context refers to names in the current scope that are referenced by inner
-   scopes or names in outer scopes that are referenced from this scope.  It does
-   *not* include references to global or builtin scopes.
+   Sequence of bytecodes that access a :term:`free (closure) variable <closure variable>`.
+   'free' in this context refers to names in the current scope that are
+   referenced by inner scopes or names in outer scopes that are referenced
+   from this scope.  It does *not* include references to global or builtin scopes.
 
 
 .. data:: hasname

diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
index a96f69e6170f0066b4536677e597fc4399faa9d4..7f8df704a3332760c52d4cf6f465224c7812144e 100644 (file)
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -684,9 +684,10 @@ are always available.  They are listed here in alphabetical order.
    ``__builtins__`` dictionary into *globals* before passing it to :func:`exec`.
 
    The *closure* argument specifies a closure--a tuple of cellvars.
-   It's only valid when the *object* is a code object containing free variables.
-   The length of the tuple must exactly match the number of free variables
-   referenced by the code object.
+   It's only valid when the *object* is a code object containing
+   :term:`free (closure) variables <closure variable>`.
+   The length of the tuple must exactly match the length of the code object'S
+   :attr:`~codeobject.co_freevars` attribute.
 
    .. audit-event:: exec code_object exec
 

diff --git a/Doc/library/symtable.rst b/Doc/library/symtable.rst
index 15e0b23aa12bf0c47d861c9c516e2a2ebb000328..56cd6b8afaa73ef3eaf8feec03aed7e25c7df4b9 100644 (file)
--- a/Doc/library/symtable.rst
+++ b/Doc/library/symtable.rst
@@ -167,11 +167,12 @@ Examining Symbol Tables
 
    .. method:: get_nonlocals()
 
-      Return a tuple containing names of nonlocals in this function.
+      Return a tuple containing names of explicitly declared nonlocals in this function.
 
    .. method:: get_frees()
 
-      Return a tuple containing names of free variables in this function.
+      Return a tuple containing names of :term:`free (closure) variables <closure variable>`
+      in this function.
 
 
 .. class:: Class

diff --git a/Doc/library/types.rst b/Doc/library/types.rst
index 3c3c760c206ff2d6104a2b4195e4cfcfaafa251b..84b80ec6efd59f1731d59d85920ed143282e1be1 100644 (file)
--- a/Doc/library/types.rst
+++ b/Doc/library/types.rst
@@ -199,7 +199,7 @@ Standard names are defined for the following types:
 .. data:: CellType
 
    The type for cell objects: such objects are used as containers for
-   a function's free variables.
+   a function's :term:`closure variables <closure variable>`.
 
    .. versionadded:: 3.8
 

diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
index 513199d21456bf42e149fc778d02baee89209dc5..d059a660548c7e53a882f9cb71377c133189943d 100644 (file)
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -564,8 +564,9 @@ Special read-only attributes
        in which the function was defined.
 
    * - .. attribute:: function.__closure__
-     - ``None`` or a :class:`tuple` of cells that contain bindings for the
-       function's free variables.
+     - ``None`` or a :class:`tuple` of cells that contain bindings for the names specified
+       in the :attr:`~codeobject.co_freevars` attribute of the function's
+       :attr:`code object <function.__code__>`.
 
        A cell object has the attribute ``cell_contents``.
        This can be used to get the value of the cell, as well as set the value.
@@ -1285,10 +1286,14 @@ Special read-only attributes
 
    * - .. attribute:: codeobject.co_cellvars
      - A :class:`tuple` containing the names of :ref:`local variables <naming>`
-       that are referenced by nested functions inside the function
+       that are referenced from at least one :term:`nested scope` inside the function
 
    * - .. attribute:: codeobject.co_freevars
-     - A :class:`tuple` containing the names of free variables in the function
+     - A :class:`tuple` containing the names of
+       :term:`free (closure) variables <closure variable>` that a :term:`nested scope`
+       references in an outer scope. See also :attr:`function.__closure__`.
+
+       Note: references to global and builtin names are *not* included.
 
    * - .. attribute:: codeobject.co_code
      - A string representing the sequence of :term:`bytecode` instructions in

diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst
index 99cb09d09331d853499b4de19938edabb8b1b398..cb6c524dd97a303a8139410b1e84a119dbb8d44c 100644 (file)
--- a/Doc/reference/executionmodel.rst
+++ b/Doc/reference/executionmodel.rst
@@ -90,7 +90,7 @@ If a name is bound in a block, it is a local variable of that block, unless
 declared as :keyword:`nonlocal` or :keyword:`global`.  If a name is bound at
 the module level, it is a global variable.  (The variables of the module code
 block are local and global.)  If a variable is used in a code block but not
-defined there, it is a :dfn:`free variable`.
+defined there, it is a :term:`free variable`.
 
 Each occurrence of a name in the program text refers to the :dfn:`binding` of
 that name established by the following name resolution rules.
@@ -337,6 +337,9 @@ enclosing namespace, but in the global namespace.  [#]_ The :func:`exec` and
 :func:`eval` functions have optional arguments to override the global and local
 namespace.  If only one namespace is specified, it is used for both.
 
+.. XXX(ncoghlan) above is only accurate for string execution. When executing code objects,
+   closure cells may now be passed explicitly to resolve co_freevars references.
+   Docs issue: https://github.com/python/cpython/issues/122826
 
 .. _exceptions:
 

diff --git a/Misc/NEWS.d/next/Documentation/2024-08-01-17-18-21.gh-issue-70870.fZnBM9.rst b/Misc/NEWS.d/next/Documentation/2024-08-01-17-18-21.gh-issue-70870.fZnBM9.rst
new file mode 100644 (file)
index 0000000..ba607bf
--- /dev/null
+++ b/Misc/NEWS.d/next/Documentation/2024-08-01-17-18-21.gh-issue-70870.fZnBM9.rst
@@ -0,0 +1,3 @@
+Clarified the dual usage of the term "free variable" (both the formal
+meaning of any reference to names defined outside the local scope, and the
+narrower pragmatic meaning of nonlocal variables named in ``co_freevars``).