]> git.ipfire.org Git - thirdparty/Python/cpython.git/commitdiff
gh-152384: pixi-packages: use `flags` to define variants (#152385)
authorLucas Colley <lucas.colley8@gmail.com>
Fri, 17 Jul 2026 06:32:29 +0000 (08:32 +0200)
committerGitHub <noreply@github.com>
Fri, 17 Jul 2026 06:32:29 +0000 (12:02 +0530)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Isuru Fernando <isuruf@gmail.com>
Co-authored-by: Nathan Goldbaum <nathan.goldbaum@gmail.com>
15 files changed:
.github/workflows/reusable-san.yml
Misc/NEWS.d/next/Tools-Demos/2026-07-10-14-38-31.gh-issue-152384.uaPADM.rst [new file with mode: 0644]
Tools/pixi-packages/README.md
Tools/pixi-packages/asan/variants.yaml [deleted file]
Tools/pixi-packages/build.sh
Tools/pixi-packages/clone-recipe.sh [deleted file]
Tools/pixi-packages/default/pixi.toml [deleted file]
Tools/pixi-packages/default/variants.yaml [deleted file]
Tools/pixi-packages/freethreading/pixi.toml [deleted file]
Tools/pixi-packages/freethreading/variants.yaml [deleted file]
Tools/pixi-packages/pixi.toml [moved from Tools/pixi-packages/asan/pixi.toml with 50% similarity]
Tools/pixi-packages/recipe.yaml [moved from Tools/pixi-packages/default/recipe.yaml with 85% similarity]
Tools/pixi-packages/tsan-freethreading/pixi.toml [deleted file]
Tools/pixi-packages/tsan-freethreading/variants.yaml [deleted file]
Tools/pixi-packages/variants.yaml [new file with mode: 0644]

index cae51fad2c586ca3c9b53580547f62682119b77f..40464695a281afd8a9fe63460d2ed2800552386b 100644 (file)
@@ -39,6 +39,8 @@ jobs:
       run: |
         sudo ./.github/workflows/posix-deps-apt.sh
         # On ubuntu-26.04 image, clang is clang-21 by default
+        # NOTE: when bumping to a new version of clang,
+        # please update the versions in `Tools/pixi-packages/variants.yaml` too.
         echo "CC=clang" >> "$GITHUB_ENV"
         echo "CXX=clang++" >> "$GITHUB_ENV"
     - name: TSan option setup
diff --git a/Misc/NEWS.d/next/Tools-Demos/2026-07-10-14-38-31.gh-issue-152384.uaPADM.rst b/Misc/NEWS.d/next/Tools-Demos/2026-07-10-14-38-31.gh-issue-152384.uaPADM.rst
new file mode 100644 (file)
index 0000000..08b94d5
--- /dev/null
@@ -0,0 +1,8 @@
+The CPython Pixi packages are now all accessible at the same
+``Tools/pixi-packages`` subdirectory, rather than at
+``Tools/pixi-packages/{variant}`` as before. Variants are now selected not
+via subdirectory but via ``flags``; see
+https://pixi.prefix.dev/latest/concepts/package_specifications/#extras-and-flags
+for usage instructions. The ``tsan-freethreading`` variant has been renamed
+to ``tsan_freethreading``, while the ``default``, ``asan``, and
+``freethreading`` variants retain their previous names.
index d818fddaac6a1e68557a2f031c56d06a7c1e17ef..3e9df3f56f37fadea845542502406a46d3438e88 100644 (file)
@@ -10,7 +10,8 @@ in a [Pixi workspace](https://pixi.sh/latest/first_workspace/), like:
 ```toml
 [dependencies]
 python.git = "https://github.com/python/cpython"
-python.subdirectory = "Tools/pixi-packages/asan"
+python.subdirectory = "Tools/pixi-packages"
+python.flags = ["asan"]
 ```
 
 This is particularly useful when developers need to build CPython from source
@@ -18,17 +19,18 @@ This is particularly useful when developers need to build CPython from source
 clone or build steps. Instead, Pixi will automatically handle both the build
 and installation of the package.
 
-Each package definition is contained in a subdirectory, but they share the build script
-`build.sh` in this directory. Currently defined package variants:
+Each package variant carries a 'flag' to enable selection of that variant — see
+[the Pixi docs](https://pixi.prefix.dev/latest/concepts/package_specifications/#extras-and-flags)
+for details of how to use this. Currently defined package variants:
 
 - `default`
 - `freethreading`
 - `asan`: ASan-instrumented build
-- `tsan-freethreading`: TSan-instrumented free-threading build
+- `tsan_freethreading`: TSan-instrumented free-threading build
 
 ## Maintenance
 
-- Keep the `abi_tag` and `version` fields in each `variants.yaml` up to date with the
+- Keep the `version` field in `variants.yaml` up to date with the
   Python version
 - Update `build.sh` for any breaking changes in the `configure` and `make` workflow
 
@@ -36,8 +38,6 @@ Each package definition is contained in a subdirectory, but they share the build
 
 - More package variants (such as UBSan)
 - Support for Windows
-- Using a single `pixi.toml` for all package variants is blocked on
-  [pixi#5248](https://github.com/prefix-dev/pixi/issues/5248)
 
 ## Troubleshooting
 
diff --git a/Tools/pixi-packages/asan/variants.yaml b/Tools/pixi-packages/asan/variants.yaml
deleted file mode 100644 (file)
index 2404948..0000000
+++ /dev/null
@@ -1,6 +0,0 @@
-variant:
-  - asan
-abi_tag:
-  - asan_cp315
-version:
-  - 3.15
index 7e22e6243a5f779ef16cbeb7ba119483aeb4cf08..b13e26d688e46a117eb38a725c37cb0c929d03b6 100644 (file)
@@ -7,7 +7,7 @@ if [[ "${PYTHON_VARIANT}" == "freethreading" ]]; then
 elif [[ "${PYTHON_VARIANT}" == "asan" ]]; then
     CONFIGURE_EXTRA="--with-address-sanitizer"
     export ASAN_OPTIONS="strict_init_order=true"
-elif [[ "${PYTHON_VARIANT}" == "tsan-freethreading" ]]; then
+elif [[ "${PYTHON_VARIANT}" == "tsan_freethreading" ]]; then
     CONFIGURE_EXTRA="--disable-gil --with-thread-sanitizer"
     export TSAN_OPTIONS="suppressions=${SRC_DIR}/Tools/tsan/suppressions_free_threading.txt"
 elif [[ "${PYTHON_VARIANT}" == "default" ]]; then
@@ -17,6 +17,14 @@ else
     exit 1
 fi
 
+VER_REF=$(grep "\[PYTHON_VERSION\]\, \[" configure.ac | sed -n 's/.*\[\([0-9.]*\)\].*/\1/p')
+VER=$(echo ${PKG_VERSION} | sed -E 's/^([0-9]+\.[0-9]+).*/\1/')
+
+if [[ "${VER_REF}" != "${VER}" ]]; then
+    echo "Unexpected version from conda package. Got ${VER}. Expected ${VER_REF}. Do you need to update 'version' in 'variants.yaml'?"
+    exit 1
+fi
+
 # rattler-build by default set a target of 10.9
 # override it to at least 10.12
 case ${MACOSX_DEPLOYMENT_TARGET:-10.12} in
diff --git a/Tools/pixi-packages/clone-recipe.sh b/Tools/pixi-packages/clone-recipe.sh
deleted file mode 100755 (executable)
index 25ceaf8..0000000
+++ /dev/null
@@ -1,10 +0,0 @@
-#!/bin/bash
-
-# Please always only modify default/recipe.yaml and default/pixi.toml and then run this
-# script to propagate the changes to the other variants.
-set -o errexit
-cd "$(dirname "$0")"
-
-for variant in asan freethreading tsan-freethreading; do
-    cp -av default/pixi.toml ${variant}/
-done
diff --git a/Tools/pixi-packages/default/pixi.toml b/Tools/pixi-packages/default/pixi.toml
deleted file mode 100644 (file)
index bf9841e..0000000
+++ /dev/null
@@ -1,15 +0,0 @@
-# NOTE: Please always only modify default/pixi.toml and then run clone-recipe.sh to
-# propagate the changes to the other variants.
-
-[workspace]
-channels = ["https://prefix.dev/conda-forge"]
-platforms = ["linux-64", "linux-aarch64", "osx-64", "osx-arm64"]
-preview = ["pixi-build"]
-requires-pixi = ">=0.66.0"
-
-[package.build.backend]
-name = "pixi-build-rattler-build"
-version = "*"
-
-[package.build.config]
-recipe = "../default/recipe.yaml"
diff --git a/Tools/pixi-packages/default/variants.yaml b/Tools/pixi-packages/default/variants.yaml
deleted file mode 100644 (file)
index f66e9e7..0000000
+++ /dev/null
@@ -1,6 +0,0 @@
-variant:
-  - default
-abi_tag:
-  - cp315
-version:
-  - 3.15
diff --git a/Tools/pixi-packages/freethreading/pixi.toml b/Tools/pixi-packages/freethreading/pixi.toml
deleted file mode 100644 (file)
index bf9841e..0000000
+++ /dev/null
@@ -1,15 +0,0 @@
-# NOTE: Please always only modify default/pixi.toml and then run clone-recipe.sh to
-# propagate the changes to the other variants.
-
-[workspace]
-channels = ["https://prefix.dev/conda-forge"]
-platforms = ["linux-64", "linux-aarch64", "osx-64", "osx-arm64"]
-preview = ["pixi-build"]
-requires-pixi = ">=0.66.0"
-
-[package.build.backend]
-name = "pixi-build-rattler-build"
-version = "*"
-
-[package.build.config]
-recipe = "../default/recipe.yaml"
diff --git a/Tools/pixi-packages/freethreading/variants.yaml b/Tools/pixi-packages/freethreading/variants.yaml
deleted file mode 100644 (file)
index 022833d..0000000
+++ /dev/null
@@ -1,6 +0,0 @@
-variant:
-  - freethreading
-abi_tag:
-  - cp315t
-version:
-  - 3.15
similarity index 50%
rename from Tools/pixi-packages/asan/pixi.toml
rename to Tools/pixi-packages/pixi.toml
index bf9841e18677caba73f1ed6ed2e18d8e57af425a..afbb5e1bcd56bfe906e34abc12e2b7cd52fa0a5d 100644 (file)
@@ -1,15 +1,9 @@
-# NOTE: Please always only modify default/pixi.toml and then run clone-recipe.sh to
-# propagate the changes to the other variants.
-
 [workspace]
 channels = ["https://prefix.dev/conda-forge"]
 platforms = ["linux-64", "linux-aarch64", "osx-64", "osx-arm64"]
 preview = ["pixi-build"]
-requires-pixi = ">=0.66.0"
+requires-pixi = ">=0.72.2"
 
 [package.build.backend]
 name = "pixi-build-rattler-build"
 version = "*"
-
-[package.build.config]
-recipe = "../default/recipe.yaml"
similarity index 85%
rename from Tools/pixi-packages/default/recipe.yaml
rename to Tools/pixi-packages/recipe.yaml
index 30d0d5a2ed2e04df5387a10e922ac366c7facef8..c40fe2ad2e0c79920730ef5afd435f68520c3744 100644 (file)
@@ -1,15 +1,13 @@
-# NOTE: Please always only modify default/recipe.yaml and then run clone-recipe.sh to
-# propagate the changes to the other variants.
-
 context:
-  # Keep up to date
   freethreading_tag: ${{ "t" if "freethreading" in variant else "" }}
+  abi_prefix: ${{ (variant | split("_"))[0] + "_" if "san" in variant else "" }}
+  abi_tag: ${{ abi_prefix }}cp${{ (version | split('.'))[:2] | join('') }}${{ freethreading_tag }}
 
 recipe:
   name: python
 
 source:
-  - path: ../../..
+  - path: ../..
 
 outputs:
 - package:
@@ -25,12 +23,16 @@ outputs:
     name: python
     version: ${{ version }}
   build:
+    flags:
+      - ${{ variant }}
+    variant:
+      down_prioritize_variant: ${{ 0 if variant == "default" else 1 }}
     string: "0_${{ abi_tag }}"
     files:
       exclude:
         - "*.o"
     script:
-      file: ../build.sh
+      file: build.sh
       env:
         PYTHON_VARIANT: ${{ variant }}
     python:
@@ -70,9 +72,6 @@ outputs:
           - libuuid
       - libmpdec-devel
       - expat
-      - if: linux and "san" in variant
-        then:
-          - libsanitizer
       - if: osx and "san" in variant
         then:
           - libcompiler-rt
diff --git a/Tools/pixi-packages/tsan-freethreading/pixi.toml b/Tools/pixi-packages/tsan-freethreading/pixi.toml
deleted file mode 100644 (file)
index bf9841e..0000000
+++ /dev/null
@@ -1,15 +0,0 @@
-# NOTE: Please always only modify default/pixi.toml and then run clone-recipe.sh to
-# propagate the changes to the other variants.
-
-[workspace]
-channels = ["https://prefix.dev/conda-forge"]
-platforms = ["linux-64", "linux-aarch64", "osx-64", "osx-arm64"]
-preview = ["pixi-build"]
-requires-pixi = ">=0.66.0"
-
-[package.build.backend]
-name = "pixi-build-rattler-build"
-version = "*"
-
-[package.build.config]
-recipe = "../default/recipe.yaml"
diff --git a/Tools/pixi-packages/tsan-freethreading/variants.yaml b/Tools/pixi-packages/tsan-freethreading/variants.yaml
deleted file mode 100644 (file)
index 6ed09fc..0000000
+++ /dev/null
@@ -1,6 +0,0 @@
-variant:
-  - tsan-freethreading
-abi_tag:
-  - tsan_cp315t
-version:
-  - 3.15
diff --git a/Tools/pixi-packages/variants.yaml b/Tools/pixi-packages/variants.yaml
new file mode 100644 (file)
index 0000000..2519ec5
--- /dev/null
@@ -0,0 +1,15 @@
+version: ["3.16"]
+variant: ["default", "asan", "freethreading", "tsan_freethreading"]
+
+openssl:
+  - '3.5'
+
+# match `.github/workflows/reusable-san.yml`
+c_compiler:
+  - clang
+c_compiler_version:
+  - 21
+cxx_compiler:
+  - clangxx
+cxx_compiler_version:
+  - 21