gh-153056: Fix a data race compiling the string.Template pattern in free-threading...

* gh-153056: Fix a data race compiling the string.Template pattern in free-threading builds

Template compiles its substitution pattern lazily and caches it on the class.  On the free-threaded build two concurrent first uses could race: a thread that observed the pattern another thread had just compiled would try to recompile it, and re.compile() rejects flags on an already-compiled pattern, raising a spurious ValueError.  Return the already-compiled pattern instead.

As a side effect, a subclass that supplies an already-compiled pattern now works too; previously it raised the same ValueError at class definition.

* Trim test comments and NEWS wording

* Document that the pattern attribute accepts a string or a compiled regex

* Comment the three states of pattern and note the documented-behavior fix in NEWS

* Update Doc/library/string.rst

---------

Co-authored-by: Barry Warsaw <barry@python.org>

diff --git a/Doc/library/string.rst b/Doc/library/string.rst
index be968a3c53d84305d25b266abc0e217c1657c604..f1abed91d20ed5149dc2ba94ca28580df1a19846 100644 (file)
--- a/Doc/library/string.rst
+++ b/Doc/library/string.rst
@@ -971,7 +971,8 @@ attributes:
 
 Alternatively, you can provide the entire regular expression pattern by
 overriding the class attribute *pattern*.  If you do this, the value must be a
-regular expression object with four named capturing groups.  The capturing
+regular expression pattern string, or a compiled regular expression
+object, with four named capturing groups.  The capturing
 groups correspond to the rules given above, along with the invalid placeholder
 rule:
 

diff --git a/Lib/string/__init__.py b/Lib/string/__init__.py
index b788d7136f1ae3097844343e2592a0bca59029d5..88f1ae1b9b023eb01a033469e8eb49a2e83216f0 100644 (file)
--- a/Lib/string/__init__.py
+++ b/Lib/string/__init__.py
@@ -83,7 +83,14 @@ class Template:
     def _compile_pattern(cls):
         import re  # deferred import, for performance
 
+        # `pattern` may be the `_TemplatePattern` sentinel (not yet compiled), an
+        # already-compiled regular expression object (as documented), or a string
+        # regular expression.  An already-compiled object is returned as-is; the
+        # other two are compiled and cached back on the class.
         pattern = cls.__dict__.get('pattern', _TemplatePattern)
+        if isinstance(pattern, re.Pattern):
+            # re.compile() rejects flags on an already-compiled pattern.
+            return pattern
         if pattern is _TemplatePattern:
             delim = re.escape(cls.delimiter)
             id = cls.idpattern

diff --git a/Lib/test/test_free_threading/test_string_template_race.py b/Lib/test/test_free_threading/test_string_template_race.py
new file mode 100644 (file)
index 0000000..dc4b4e9
--- /dev/null
+++ b/Lib/test/test_free_threading/test_string_template_race.py
@@ -0,0 +1,35 @@
+import string
+import unittest
+from string import Template
+
+from test.support import threading_helper
+
+
+@threading_helper.requires_working_threading()
+class TestTemplateCompileRace(unittest.TestCase):
+    def test_concurrent_first_use(self):
+        # Racing the lazy pattern compile must not raise a spurious ValueError
+        # from recompiling an already-compiled pattern.  A throwaway subclass,
+        # re-armed to the sentinel each round, keeps string.Template unmutated
+        # (subclasses precompile eagerly in __init_subclass__).
+        uncompiled = string._TemplatePattern
+        errors = []
+
+        def use_template(cls):
+            try:
+                cls("$x and ${y}").substitute(x=1, y=2)
+            except Exception as e:
+                errors.append(e)
+
+        for _ in range(20):
+            class T(Template):
+                pass
+            T.pattern = uncompiled
+            T.flags = None
+            threading_helper.run_concurrently(use_template, nthreads=10, args=(T,))
+
+        self.assertEqual(errors, [], msg=f"unexpected errors: {errors}")
+
+
+if __name__ == "__main__":
+    unittest.main()

diff --git a/Lib/test/test_string/test_string.py b/Lib/test/test_string/test_string.py
index 5394fe4e12cd41c29dc0d398efbfa14b9a4c7261..350784a9f00817bd59ba772fb7348fe598facdb9 100644 (file)
--- a/Lib/test/test_string/test_string.py
+++ b/Lib/test/test_string/test_string.py
@@ -299,6 +299,20 @@ class TestTemplate(unittest.TestCase):
         eq(s.safe_substitute(dict(who='tim', what='ham', meal='dinner')),
            'tim likes ham for dinner')
 
+    def test_precompiled_pattern(self):
+        # A subclass may supply an already-compiled pattern; it must be reused,
+        # not recompiled (re.compile() rejects flags on a compiled pattern).
+        import re
+        compiled = re.compile(
+            r'\$(?:(?P<escaped>\$)|(?P<named>[a-z]+)|'
+            r'\{(?P<braced>[a-z]+)\}|(?P<invalid>))')
+        class MyTemplate(Template):
+            pattern = compiled
+        self.assertIs(MyTemplate.pattern, compiled)
+        self.assertEqual(
+            MyTemplate('$who likes $what').substitute(who='tim', what='ham'),
+            'tim likes ham')
+
     def test_invalid_placeholders(self):
         raises = self.assertRaises
         s = Template('$who likes $')

diff --git a/Misc/NEWS.d/next/Library/2026-07-05-12-00-00.gh-issue-153056.tMpLat.rst b/Misc/NEWS.d/next/Library/2026-07-05-12-00-00.gh-issue-153056.tMpLat.rst
new file mode 100644 (file)
index 0000000..a2bc89f
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2026-07-05-12-00-00.gh-issue-153056.tMpLat.rst
@@ -0,0 +1,4 @@
+Fix :class:`string.Template` raising a spurious :exc:`ValueError` when the
+*pattern* attribute is a compiled regular expression object, which the
+documentation allows.  On the free-threaded build this also occurred as a data
+race on the first concurrent use.