# Python for Android
-These instructions are only needed if you're planning to compile Python for
-Android yourself. Most users should *not* need to do this. Instead, use one of
-the tools listed in `Doc/using/android.rst`, which will provide a much easier
-experience.
+If you obtained this README as part of a release package, then the only
+applicable sections are "Prerequisites", "Testing", and "Using in your own app".
+If you obtained this README as part of the CPython source tree, then you can
+also follow the other sections to compile Python for Android yourself.
+
+However, most app developers should not need to do any of these things manually.
+Instead, use one of the tools listed
+[here](https://docs.python.org/3/using/android.html), which will provide a much
+easier experience.
-## Prerequisites
-First, make sure you have all the usual tools and libraries needed to build
-Python for your development machine.
+## Prerequisites
-Second, you'll need an Android SDK. If you already have the SDK installed,
-export the `ANDROID_HOME` environment variable to point at its location.
-Otherwise, here's how to install it:
+If you already have an Android SDK installed, export the `ANDROID_HOME`
+environment variable to point at its location. Otherwise, here's how to install
+it:
* Download the "Command line tools" from <https://developer.android.com/studio>.
* Create a directory `android-sdk/cmdline-tools`, and unzip the command line
* `curl`
* `java` (or set the `JAVA_HOME` environment variable)
* `tar`
-* `unzip`
## Building
Python can be built for Android on any POSIX platform supported by the Android
-development tools, which currently means Linux or macOS. This involves doing a
-cross-build where you use a "build" Python (for your development machine) to
-help produce a "host" Python for Android.
+development tools, which currently means Linux or macOS.
+
+First we'll make a "build" Python (for your development machine), then use it to
+help produce a "host" Python for Android. So make sure you have all the usual
+tools and libraries needed to build Python for your development machine.
The easiest way to do a build is to use the `android.py` script. You can either
have it perform the entire build process from start to finish in one step, or
./android.py build HOST
```
-In the end you should have a build Python in `cross-build/build`, and an Android
-build in `cross-build/HOST`.
+In the end you should have a build Python in `cross-build/build`, and a host
+Python in `cross-build/HOST`.
You can use `--` as a separator for any of the `configure`-related commands –
including `build` itself – to pass arguments to the underlying `configure`
```
+## Packaging
+
+After building an architecture as described in the section above, you can
+package it for release with this command:
+
+```sh
+./android.py package HOST
+```
+
+`HOST` is defined in the section above.
+
+This will generate a tarball in `cross-build/HOST/dist`, whose structure is
+similar to the `Android` directory of the CPython source tree.
+
+
## Testing
-The test suite can be run on Linux, macOS, or Windows:
+The Python test suite can be run on Linux, macOS, or Windows:
* On Linux, the emulator needs access to the KVM virtualization interface, and
a DISPLAY environment variable pointing at an X server.
-* On Windows, you won't be able to do the build on the same machine, so you'll
- have to copy the `cross-build/HOST` directory from somewhere else.
The test suite can usually be run on a device with 2 GB of RAM, but this is
borderline, so you may need to increase it to 4 GB. As of Android
manually to the same value, or use the Android Studio Device Manager, which will
update both files.
-Before running the test suite, follow the instructions in the previous section
-to build the architecture you want to test. Then run the test script in one of
-the following modes:
+You can run the test suite either:
+
+* Within the CPython repository, after doing a build as described above. On
+ Windows, you won't be able to do the build on the same machine, so you'll have
+ to copy the `cross-build/HOST/prefix` directory from somewhere else.
+
+* Or by taking a release package built using the `package` command, extracting
+ it wherever you want, and using its own copy of `android.py`.
+
+The test script supports the following modes:
* In `--connected` mode, it runs on a device or emulator you have already
connected to the build machine. List the available devices with
## Using in your own app
-See `Doc/using/android.rst`.
+See https://docs.python.org/3/using/android.html.
import asyncio
import argparse
-from glob import glob
import os
import re
import shlex
import sysconfig
from asyncio import wait_for
from contextlib import asynccontextmanager
+from datetime import datetime, timezone
+from glob import glob
from os.path import basename, relpath
from pathlib import Path
from subprocess import CalledProcessError
SCRIPT_NAME = Path(__file__).name
-CHECKOUT = Path(__file__).resolve().parent.parent
-ANDROID_DIR = CHECKOUT / "Android"
+ANDROID_DIR = Path(__file__).resolve().parent
+CHECKOUT = ANDROID_DIR.parent
TESTBED_DIR = ANDROID_DIR / "testbed"
CROSS_BUILD_DIR = CHECKOUT / "cross-build"
+HOSTS = ["aarch64-linux-android", "x86_64-linux-android"]
APP_ID = "org.python.testbed"
DECODE_ARGS = ("UTF-8", "backslashreplace")
path.unlink()
-def subdir(name, *, clean=None):
- path = CROSS_BUILD_DIR / name
- if clean:
- delete_glob(path)
+def subdir(*parts, create=False):
+ path = CROSS_BUILD_DIR.joinpath(*parts)
if not path.exists():
- if clean is None:
+ if not create:
sys.exit(
f"{path} does not exist. Create it by running the appropriate "
f"`configure` subcommand of {SCRIPT_NAME}.")
def configure_build_python(context):
- os.chdir(subdir("build", clean=context.clean))
+ if context.clean:
+ clean("build")
+ os.chdir(subdir("build", create=True))
command = [relpath(CHECKOUT / "configure")]
if context.args:
def configure_host_python(context):
- host_dir = subdir(context.host, clean=context.clean)
+ if context.clean:
+ clean(context.host)
+ host_dir = subdir(context.host, create=True)
prefix_dir = host_dir / "prefix"
if not prefix_dir.exists():
prefix_dir.mkdir()
os.chdir(prefix_dir)
unpack_deps(context.host)
- build_dir = host_dir / "build"
- build_dir.mkdir(exist_ok=True)
- os.chdir(build_dir)
-
+ os.chdir(host_dir)
command = [
# Basic cross-compiling configuration
relpath(CHECKOUT / "configure"),
# the build.
host_dir = subdir(context.host)
prefix_dir = host_dir / "prefix"
- delete_glob(f"{prefix_dir}/include/python*")
- delete_glob(f"{prefix_dir}/lib/libpython*")
- delete_glob(f"{prefix_dir}/lib/python*")
+ for pattern in ("include/python*", "lib/libpython*", "lib/python*"):
+ delete_glob(f"{prefix_dir}/{pattern}")
- os.chdir(host_dir / "build")
+ os.chdir(host_dir)
run(["make", "-j", str(os.cpu_count())], host=context.host)
run(["make", "install", f"prefix={prefix_dir}"], host=context.host)
step(context)
+def clean(host):
+ delete_glob(CROSS_BUILD_DIR / host)
+
+
def clean_all(context):
- delete_glob(CROSS_BUILD_DIR)
+ for host in HOSTS + ["build"]:
+ clean(host)
def setup_sdk():
# To avoid distributing compiled artifacts without corresponding source code,
# the Gradle wrapper is not included in the CPython repository. Instead, we
-# extract it from the Gradle release.
+# extract it from the Gradle GitHub repository.
def setup_testbed():
- if all((TESTBED_DIR / path).exists() for path in [
- "gradlew", "gradlew.bat", "gradle/wrapper/gradle-wrapper.jar",
- ]):
+ # The Gradle version used for the build is specified in
+ # testbed/gradle/wrapper/gradle-wrapper.properties. This wrapper version
+ # doesn't need to match, as any version of the wrapper can download any
+ # version of Gradle.
+ version = "8.9.0"
+ paths = ["gradlew", "gradlew.bat", "gradle/wrapper/gradle-wrapper.jar"]
+
+ if all((TESTBED_DIR / path).exists() for path in paths):
return
- ver_long = "8.7.0"
- ver_short = ver_long.removesuffix(".0")
-
- for filename in ["gradlew", "gradlew.bat"]:
- out_path = download(
- f"https://raw.githubusercontent.com/gradle/gradle/v{ver_long}/{filename}",
- TESTBED_DIR)
+ for path in paths:
+ out_path = TESTBED_DIR / path
+ out_path.parent.mkdir(exist_ok=True)
+ download(
+ f"https://raw.githubusercontent.com/gradle/gradle/v{version}/{path}",
+ out_path.parent,
+ )
os.chmod(out_path, 0o755)
- with TemporaryDirectory(prefix=SCRIPT_NAME) as temp_dir:
- bin_zip = download(
- f"https://services.gradle.org/distributions/gradle-{ver_short}-bin.zip",
- temp_dir)
- outer_jar = f"gradle-{ver_short}/lib/plugins/gradle-wrapper-{ver_short}.jar"
- run(["unzip", "-d", temp_dir, bin_zip, outer_jar])
- run(["unzip", "-o", "-d", f"{TESTBED_DIR}/gradle/wrapper",
- f"{temp_dir}/{outer_jar}", "gradle-wrapper.jar"])
-
# run_testbed will build the app automatically, but it's useful to have this as
# a separate command to allow running the app outside of this script.
raise e.exceptions[0]
+def package_version(prefix_dir):
+ patchlevel_glob = f"{prefix_dir}/include/python*/patchlevel.h"
+ patchlevel_paths = glob(patchlevel_glob)
+ if len(patchlevel_paths) != 1:
+ sys.exit(f"{patchlevel_glob} matched {len(patchlevel_paths)} paths.")
+
+ for line in open(patchlevel_paths[0]):
+ if match := re.fullmatch(r'\s*#define\s+PY_VERSION\s+"(.+)"\s*', line):
+ version = match[1]
+ break
+ else:
+ sys.exit(f"Failed to find Python version in {patchlevel_paths[0]}.")
+
+ # If not building against a tagged commit, add a timestamp to the version.
+ # Follow the PyPA version number rules, as this will make it easier to
+ # process with other tools.
+ if version.endswith("+"):
+ version += datetime.now(timezone.utc).strftime("%Y%m%d.%H%M%S")
+
+ return version
+
+
+def package(context):
+ prefix_dir = subdir(context.host, "prefix")
+ version = package_version(prefix_dir)
+
+ with TemporaryDirectory(prefix=SCRIPT_NAME) as temp_dir:
+ temp_dir = Path(temp_dir)
+
+ # Include all tracked files from the Android directory.
+ for line in run(
+ ["git", "ls-files"],
+ cwd=ANDROID_DIR, capture_output=True, text=True, log=False,
+ ).stdout.splitlines():
+ src = ANDROID_DIR / line
+ dst = temp_dir / line
+ dst.parent.mkdir(parents=True, exist_ok=True)
+ shutil.copy2(src, dst, follow_symlinks=False)
+
+ # Include anything from the prefix directory which could be useful
+ # either for embedding Python in an app, or building third-party
+ # packages against it.
+ for rel_dir, patterns in [
+ ("include", ["openssl*", "python*", "sqlite*"]),
+ ("lib", ["engines-3", "libcrypto*.so", "libpython*", "libsqlite*",
+ "libssl*.so", "ossl-modules", "python*"]),
+ ("lib/pkgconfig", ["*crypto*", "*ssl*", "*python*", "*sqlite*"]),
+ ]:
+ for pattern in patterns:
+ for src in glob(f"{prefix_dir}/{rel_dir}/{pattern}"):
+ dst = temp_dir / relpath(src, prefix_dir.parent)
+ dst.parent.mkdir(parents=True, exist_ok=True)
+ if Path(src).is_dir():
+ shutil.copytree(
+ src, dst, symlinks=True,
+ ignore=lambda *args: ["__pycache__"]
+ )
+ else:
+ shutil.copy2(src, dst, follow_symlinks=False)
+
+ dist_dir = subdir(context.host, "dist", create=True)
+ package_path = shutil.make_archive(
+ f"{dist_dir}/python-{version}-{context.host}", "gztar", temp_dir
+ )
+ print(f"Wrote {package_path}")
+
+
# Handle SIGTERM the same way as SIGINT. This ensures that if we're terminated
# by the buildbot worker, we'll make an attempt to clean up our subprocesses.
def install_signal_handler():
def parse_args():
parser = argparse.ArgumentParser()
subcommands = parser.add_subparsers(dest="subcommand")
+
+ # Subcommands
build = subcommands.add_parser("build", help="Build everything")
configure_build = subcommands.add_parser("configure-build",
help="Run `configure` for the "
make_host = subcommands.add_parser("make-host",
help="Run `make` for Android")
subcommands.add_parser(
- "clean", help="Delete the cross-build directory")
+ "clean", help="Delete all build and prefix directories")
+ subcommands.add_parser(
+ "build-testbed", help="Build the testbed app")
+ test = subcommands.add_parser(
+ "test", help="Run the test suite")
+ package = subcommands.add_parser("package", help="Make a release package")
+ # Common arguments
for subcommand in build, configure_build, configure_host:
subcommand.add_argument(
"--clean", action="store_true", default=False, dest="clean",
- help="Delete any relevant directories before building")
- for subcommand in build, configure_host, make_host:
+ help="Delete the relevant build and prefix directories first")
+ for subcommand in [build, configure_host, make_host, package]:
subcommand.add_argument(
- "host", metavar="HOST",
- choices=["aarch64-linux-android", "x86_64-linux-android"],
+ "host", metavar="HOST", choices=HOSTS,
help="Host triplet: choices=[%(choices)s]")
for subcommand in build, configure_build, configure_host:
subcommand.add_argument("args", nargs="*",
help="Extra arguments to pass to `configure`")
- subcommands.add_parser(
- "build-testbed", help="Build the testbed app")
- test = subcommands.add_parser(
- "test", help="Run the test suite")
+ # Test arguments
test.add_argument(
"-v", "--verbose", action="count", default=0,
help="Show Gradle output, and non-Python logcat messages. "
stream.reconfigure(line_buffering=True)
context = parse_args()
- dispatch = {"configure-build": configure_build_python,
- "make-build": make_build_python,
- "configure-host": configure_host_python,
- "make-host": make_host_python,
- "build": build_all,
- "clean": clean_all,
- "build-testbed": build_testbed,
- "test": run_testbed}
+ dispatch = {
+ "configure-build": configure_build_python,
+ "make-build": make_build_python,
+ "configure-host": configure_host_python,
+ "make-host": make_host_python,
+ "build": build_all,
+ "clean": clean_all,
+ "build-testbed": build_testbed,
+ "test": run_testbed,
+ "package": package,
+ }
try:
result = dispatch[context.subcommand](context)
-# The Gradle wrapper should be downloaded by running `../android.py setup-testbed`.
+# The Gradle wrapper can be downloaded by running the `test` or `build-testbed`
+# commands of android.py.
/gradlew
/gradlew.bat
/gradle/wrapper/gradle-wrapper.jar
+# The repository's top-level .gitignore file ignores all .idea directories, but
+# we want to keep any files which can't be regenerated from the Gradle
+# configuration.
+!.idea/
+/.idea/*
+!/.idea/inspectionProfiles
+
*.iml
.gradle
/local.properties
-/.idea/caches
-/.idea/deploymentTargetDropdown.xml
-/.idea/libraries
-/.idea/modules.xml
-/.idea/workspace.xml
-/.idea/navEditor.xml
-/.idea/assetWizardSettings.xml
.DS_Store
/build
/captures
--- /dev/null
+<component name="InspectionProjectProfileManager">
+ <profile version="1.0">
+ <option name="myName" value="Project Default" />
+ <inspection_tool class="AndroidLintGradleDependency" enabled="true" level="WEAK WARNING" enabled_by_default="true" editorAttributes="INFO_ATTRIBUTES" />
+ <inspection_tool class="AndroidLintOldTargetApi" enabled="true" level="WEAK WARNING" enabled_by_default="true" editorAttributes="INFO_ATTRIBUTES" />
+ <inspection_tool class="UnstableApiUsage" enabled="true" level="WEAK WARNING" enabled_by_default="true" editorAttributes="INFO_ATTRIBUTES" />
+ </profile>
+</component>
\ No newline at end of file
id("org.jetbrains.kotlin.android")
}
-val PYTHON_DIR = file("../../..").canonicalPath
-val PYTHON_CROSS_DIR = "$PYTHON_DIR/cross-build"
-
-val ABIS = mapOf(
- "arm64-v8a" to "aarch64-linux-android",
- "x86_64" to "x86_64-linux-android",
-).filter { file("$PYTHON_CROSS_DIR/${it.value}").exists() }
-if (ABIS.isEmpty()) {
+val ANDROID_DIR = file("../..")
+val PYTHON_DIR = ANDROID_DIR.parentFile!!
+val PYTHON_CROSS_DIR = file("$PYTHON_DIR/cross-build")
+val inSourceTree = (
+ ANDROID_DIR.name == "Android" && file("$PYTHON_DIR/pyconfig.h.in").exists()
+)
+
+val KNOWN_ABIS = mapOf(
+ "aarch64-linux-android" to "arm64-v8a",
+ "x86_64-linux-android" to "x86_64",
+)
+
+// Discover prefixes.
+val prefixes = ArrayList<File>()
+if (inSourceTree) {
+ for ((triplet, _) in KNOWN_ABIS.entries) {
+ val prefix = file("$PYTHON_CROSS_DIR/$triplet/prefix")
+ if (prefix.exists()) {
+ prefixes.add(prefix)
+ }
+ }
+} else {
+ // Testbed is inside a release package.
+ val prefix = file("$ANDROID_DIR/prefix")
+ if (prefix.exists()) {
+ prefixes.add(prefix)
+ }
+}
+if (prefixes.isEmpty()) {
throw GradleException(
- "No Android ABIs found in $PYTHON_CROSS_DIR: see Android/README.md " +
- "for building instructions."
+ "No Android prefixes found: see README.md for testing instructions"
)
}
-val PYTHON_VERSION = file("$PYTHON_DIR/Include/patchlevel.h").useLines {
- for (line in it) {
- val match = """#define PY_VERSION\s+"(\d+\.\d+)""".toRegex().find(line)
- if (match != null) {
- return@useLines match.groupValues[1]
+// Detect Python versions and ABIs.
+lateinit var pythonVersion: String
+var abis = HashMap<File, String>()
+for ((i, prefix) in prefixes.withIndex()) {
+ val libDir = file("$prefix/lib")
+ val version = run {
+ for (filename in libDir.list()!!) {
+ """python(\d+\.\d+)""".toRegex().matchEntire(filename)?.let {
+ return@run it.groupValues[1]
+ }
}
+ throw GradleException("Failed to find Python version in $libDir")
+ }
+ if (i == 0) {
+ pythonVersion = version
+ } else if (pythonVersion != version) {
+ throw GradleException(
+ "${prefixes[0]} is Python $pythonVersion, but $prefix is Python $version"
+ )
}
- throw GradleException("Failed to find Python version")
+
+ val libPythonDir = file("$libDir/python$pythonVersion")
+ val triplet = run {
+ for (filename in libPythonDir.list()!!) {
+ """_sysconfigdata__android_(.+).py""".toRegex().matchEntire(filename)?.let {
+ return@run it.groupValues[1]
+ }
+ }
+ throw GradleException("Failed to find Python triplet in $libPythonDir")
+ }
+ abis[prefix] = KNOWN_ABIS[triplet]!!
}
versionCode = 1
versionName = "1.0"
- ndk.abiFilters.addAll(ABIS.keys)
+ ndk.abiFilters.addAll(abis.values)
externalNativeBuild.cmake.arguments(
- "-DPYTHON_CROSS_DIR=$PYTHON_CROSS_DIR",
- "-DPYTHON_VERSION=$PYTHON_VERSION",
+ "-DPYTHON_PREFIX_DIR=" + if (inSourceTree) {
+ // AGP uses the ${} syntax for its own purposes, so use a Jinja style
+ // placeholder.
+ "$PYTHON_CROSS_DIR/{{triplet}}/prefix"
+ } else {
+ prefixes[0]
+ },
+ "-DPYTHON_VERSION=$pythonVersion",
"-DANDROID_SUPPORT_FLEXIBLE_PAGE_SIZES=ON",
)
// Create some custom tasks to copy Python and its standard library from
// elsewhere in the repository.
androidComponents.onVariants { variant ->
- val pyPlusVer = "python$PYTHON_VERSION"
+ val pyPlusVer = "python$pythonVersion"
generateTask(variant, variant.sources.assets!!) {
into("python") {
+ // Include files such as pyconfig.h are used by some of the tests.
into("include/$pyPlusVer") {
- for (triplet in ABIS.values) {
- from("$PYTHON_CROSS_DIR/$triplet/prefix/include/$pyPlusVer")
+ for (prefix in prefixes) {
+ from("$prefix/include/$pyPlusVer")
}
duplicatesStrategy = DuplicatesStrategy.EXCLUDE
}
into("lib/$pyPlusVer") {
- // To aid debugging, the source directory takes priority.
- from("$PYTHON_DIR/Lib")
-
- // The cross-build directory provides ABI-specific files such as
- // sysconfigdata.
- for (triplet in ABIS.values) {
- from("$PYTHON_CROSS_DIR/$triplet/prefix/lib/$pyPlusVer")
+ // To aid debugging, the source directory takes priority when
+ // running inside a CPython source tree.
+ if (inSourceTree) {
+ from("$PYTHON_DIR/Lib")
+ }
+ for (prefix in prefixes) {
+ from("$prefix/lib/$pyPlusVer")
}
into("site-packages") {
}
generateTask(variant, variant.sources.jniLibs!!) {
- for ((abi, triplet) in ABIS.entries) {
+ for ((prefix, abi) in abis.entries) {
into(abi) {
- from("$PYTHON_CROSS_DIR/$triplet/prefix/lib")
+ from("$prefix/lib")
include("libpython*.*.so")
include("lib*_python.so")
}
cmake_minimum_required(VERSION 3.4.1)
project(testbed)
-set(PREFIX_DIR ${PYTHON_CROSS_DIR}/${CMAKE_LIBRARY_ARCHITECTURE}/prefix)
-include_directories(${PREFIX_DIR}/include/python${PYTHON_VERSION})
-link_directories(${PREFIX_DIR}/lib)
+# Resolve variables from the command line.
+string(
+ REPLACE {{triplet}} ${CMAKE_LIBRARY_ARCHITECTURE}
+ PYTHON_PREFIX_DIR ${PYTHON_PREFIX_DIR}
+)
+
+include_directories(${PYTHON_PREFIX_DIR}/include/python${PYTHON_VERSION})
+link_directories(${PYTHON_PREFIX_DIR}/lib)
link_libraries(log python${PYTHON_VERSION})
add_library(main_activity SHARED main_activity.c)
Adding Python to an Android app
-------------------------------
-These instructions are only needed if you're planning to compile Python for
-Android yourself. Most users should *not* need to do this. Instead, use one of
-the following tools, which will provide a much easier experience:
+Most app developers should use one of the following tools, which will provide a
+much easier experience:
* `Briefcase <https://briefcase.readthedocs.io>`__, from the BeeWare project
* `Buildozer <https://buildozer.readthedocs.io>`__, from the Kivy project
link to the relevant file.
* Build Python by following the instructions in :source:`Android/README.md`.
+ This will create the directory ``cross-build/HOST/prefix``.
* Add code to your :source:`build.gradle <Android/testbed/app/build.gradle.kts>`
file to copy the following items into your project. All except your own Python
- code can be copied from ``cross-build/HOST/prefix/lib``:
+ code can be copied from ``prefix/lib``:
* In your JNI libraries:
projects / thirdparty / Python / cpython.git / commitdiff
