From b301dd2d20f2da66f4eb69527ba2642da34c630d Mon Sep 17 00:00:00 2001 From: Mike Bayer Date: Sat, 18 Dec 2021 12:36:21 -0500 Subject: [PATCH] update changelog / migration / intro for cython also detail pep-517 changes Change-Id: Ia8b3ffbe5ad222d22fe199077daa49e3f9d8e2a0 --- doc/build/changelog/migration_20.rst | 49 +++++++++++++ doc/build/changelog/unreleased_20/7256.rst | 27 +++++++ doc/build/intro.rst | 82 ++++++++++++++++------ 3 files changed, 137 insertions(+), 21 deletions(-) create mode 100644 doc/build/changelog/unreleased_20/7256.rst diff --git a/doc/build/changelog/migration_20.rst b/doc/build/changelog/migration_20.rst index 2ec9bc858f..8e969cd57f 100644 --- a/doc/build/changelog/migration_20.rst +++ b/doc/build/changelog/migration_20.rst @@ -109,6 +109,55 @@ This section covers behavioral changes made in SQLAlchemy 2.0 which are not otherwise part of the major 1.4->2.0 migration path; changes here are not expected to have significant effects on backwards compatibility. +.. _change_7311: + +Installation is now fully pep-517 enabled +------------------------------------------ + +The source distribution now includes a ``pyproject.toml`` file to allow for +complete :pep:`517` support. In particular this allows a local source build +using ``pip`` to automatically install the Cython_ optional dependency. + +:ticket:`7311` + +.. _change_7256: + +C Extensions now ported to Cython +--------------------------------- + +The SQLAlchemy C extensions have been replaced with all new extensions written +in Cython_. The move to Cython provides dramatic new advantages with +literally no downsides: + +* The Cython extensions that replace specific C extensions have all benchmarked + as **faster** than literally **all** the C code that SQLAlchemy previously + included. While this seems amazing, it appears to be a product of how highly + optimized Cython's routines are compared to a naive C implementation of a + function. + +* Cython extensions are much easier to write, maintain and debug compared to + raw C code, and in most cases are line-per-line equivalent to the Python + code. It is expected that many more elements of SQLAlchemy will be + ported to Cython in the coming releases which should open many new doors + to performance improvements that were previously out of reach. + +* Cython is very mature and widely used, including being the basis of some + of the prominent database drivers supported by SQLAlchemy including + ``asyncpg``, ``psycopg3`` and ``asyncmy``. + +Like the previous C extensions, the Cython extensions are pre-built within +SQLAlchemy's wheel distributions which are automatically available to ``pip`` +from PyPi. Manual build instructions are also unchanged with the exception +of the Cython requirement. + +.. seealso:: + + :ref:`c_extensions` + + +:ticket:`7256` + +.. _Cython: https://cython.org/ .. _migration_20_overview: diff --git a/doc/build/changelog/unreleased_20/7256.rst b/doc/build/changelog/unreleased_20/7256.rst new file mode 100644 index 0000000000..84eae4f938 --- /dev/null +++ b/doc/build/changelog/unreleased_20/7256.rst @@ -0,0 +1,27 @@ +.. change:: + :tags: feature, platform + :tickets: 7256 + + The SQLAlchemy C extensions have been replaced with all new implementations + written in Cython. Like the C extensions before, pre-built wheel files + for a wide range of platforms are available on pypi so that building + is not an issue for common platforms. For custom builds, ``python setup.py build_ext`` + works as before, needing only the additional Cython install. ``pyproject.toml`` + is also part of the source now which will establish the proper build dependencies + when using pip. + + + .. seealso:: + + :ref:`change_7256` + +.. change:: + :tags: change, platform + :tickets: 7311 + + SQLAlchemy's source build and installation now includes a ``pyproject.toml`` file + for full :pep:`517` support. + + .. seealso:: + + :ref:`change_7311` diff --git a/doc/build/intro.rst b/doc/build/intro.rst index 686d5f20f5..a4f68a2a88 100644 --- a/doc/build/intro.rst +++ b/doc/build/intro.rst @@ -131,8 +131,10 @@ downloaded from PyPI and installed in one step:: pip install SQLAlchemy -This command will download the latest **released** version of SQLAlchemy from the `Python -Cheese Shop `_ and install it to your system. +This command will download the latest **released** version of SQLAlchemy from +the `Python Cheese Shop `_ and install it +to your system. For most common platforms, a Python Wheel file will be +downloaded which provides native Cython / C extensions prebuilt. In order to install the latest **prerelease** version, such as ``2.0.0b1``, pip requires that the ``--pre`` flag be used:: @@ -143,38 +145,76 @@ Where above, if the most recent version is a prerelease, it will be installed instead of the latest released version. -Installing using setup.py ----------------------------------- +Installing manually from the source distribution +------------------------------------------------- -Otherwise, you can install from the distribution using the ``setup.py`` script:: +When not installing from pip, the source distribution may be installed +using the ``setup.py`` script:: python setup.py install +The source install is platform agnostic and will install on any platform +regardless of whether or not Cython / C build tools are installed. As the next +section :ref:`c_extensions` details, ``setup.py`` will attempt to build using +Cython / C if possible but will fall back to a pure Python installation +otherwise. + .. _c_extensions: -Installing the C Extensions +Building the Cython Extensions ---------------------------------- -SQLAlchemy includes C extensions which provide an extra speed boost for -dealing with result sets. The extensions are supported on both the 2.xx -and 3.xx series of cPython. +SQLAlchemy includes Cython_ extensions which provide an extra speed boost +within various areas, with a current emphasis on the speed of Core result sets. + +.. versionchanged:: 2.0 The SQLAlchemy C extensions have been rewritten + using Cython. + +``setup.py`` will automatically build the extensions if an appropriate platform +is detected, assuming the Cython package is installed. A complete manual +build looks like:: -``setup.py`` will automatically build the extensions if an appropriate platform is -detected. If the build of the C extensions fails due to a missing compiler or -other issue, the setup process will output a warning message and re-run the -build without the C extensions upon completion, reporting final status. + # cd into SQLAlchemy source distribution + cd path/to/sqlalchemy + + # install cython + pip install cython + + # optionally build Cython extensions ahead of install + python setup.py build_ext + + # run the install + python setup.py install -To run the build/install without even attempting to compile the C extensions, -the ``DISABLE_SQLALCHEMY_CEXT`` environment variable may be specified. The -use case for this is either for special testing circumstances, or in the rare -case of compatibility/build issues not overcome by the usual "rebuild" -mechanism:: +Source builds may also be performed using :pep:`517` techniques, such as +using build_:: + + # cd into SQLAlchemy source distribution + cd path/to/sqlalchemy + + # install build + pip install build + + # build source / wheel dists + python -m build + +If the build of the Cython extensions fails due to Cython not being installed, +a missing compiler or other issue, the setup process will output a warning +message and re-run the build without the Cython extensions upon completion, +reporting final status. + +To run the build/install without even attempting to compile the Cython +extensions, the ``DISABLE_SQLALCHEMY_CEXT`` environment variable may be +specified. The use case for this is either for special testing circumstances, +or in the rare case of compatibility/build issues not overcome by the usual +"rebuild" mechanism:: export DISABLE_SQLALCHEMY_CEXT=1; python setup.py install -.. versionchanged:: 1.1 The legacy ``--without-cextensions`` flag has been - removed from the installer as it relies on deprecated features of - setuptools. + +.. _Cython: https://cython.org/ + +.. _build: https://pypi.org/project/build/ Installing a Database API -- 2.47.2