Mike Bayer [Sat, 18 Oct 2025 20:43:22 +0000 (16:43 -0400)]
improve sqlite reflection regex for unusual names/formats
A series of improvements have been made for reflection of CHECK constraints
on SQLite. The reflection logic now correctly handles table names
containing the strings "CHECK" or "CONSTRAINT", properly supports all four
SQLite identifier quoting styles (double quotes, single quotes, brackets,
and backticks) for constraint names, and accurately parses CHECK constraint
expressions containing parentheses within string literals using balanced
parenthesis matching with string context tracking. Big thanks to
GruzdevAV for new test cases and implementation ideas.
krave1986 [Wed, 15 Oct 2025 19:37:10 +0000 (03:37 +0800)]
Update collection_api.rst (#12912)
In the Dictionary Collections section, the example code incorrectly calls .items() on the dictionary but shows dictionary output instead of the items() method's actual return value.
Mike Bayer [Wed, 15 Oct 2025 18:47:38 +0000 (14:47 -0400)]
fully copy_internals for AnnotatedFromClause for straight cloned traverse
Fixed issue where using :meth:`_sql.Select.params` to replace bound
parameters in a query could fail for some cases where the parameters
were embedded in subqueries or CTEs when ORM classes were involved,
due to issues with internal query traversal for these cases.
Mike Bayer [Mon, 29 Sep 2025 03:44:41 +0000 (23:44 -0400)]
use nox
The top-level test runner has been changed to use ``nox``, adding a
``noxfile.py`` as well as some included modules. The ``tox.ini`` file
remains in place so that ``tox`` runs will continue to function in the near
term, however it will be eventually removed and improvements and
maintenance going forward will be only towards ``noxfile.py``.
Support for ``VIRTUAL`` computed columns on PostgreSQL 18 and later has
been added. The default behavior when :paramref:`.Computed.persisted` is
not specified has been changed to align with PostgreSQL 18's default of
``VIRTUAL``. When :paramref:`.Computed.persisted` is not specified, no
keyword is rendered on PostgreSQL 18 and later; on older versions a
warning is emitted and ``STORED`` is used as the default. To explicitly
request ``STORED`` behavior on all PostgreSQL versions, specify
``persisted=True``.
Mike Bayer [Tue, 14 Oct 2025 21:19:26 +0000 (17:19 -0400)]
Support warnings in exclusions
this adds a new feature to exclusions ``warns_if()`` which applies
the expect_warnings() context manager to a test method. Additionally,
at the class level these requirements can be extracted from a
``__requirements__`` directive and also added to global Python warnings
filter using catch_warnings().
Federico Caselli [Wed, 18 Jun 2025 20:22:59 +0000 (16:22 -0400)]
postgresql dialect table options
Support for storage parameters in ``CREATE TABLE`` using the ``WITH``
clause has been added. The ``postgresql_with`` dialect option of
:class:`_schema.Table` accepts a mapping of key/value options.
The PostgreSQL dialect now support reflection of table options, including
the storage parameters, table access method and table spaces. These options
are automatically reflected when autoloading a table, and are also
available via the :meth:`_engine.Inspector.get_table_options` and
:meth:`_engine.Inspector.get_multi_table_optionsmethod` methods.
Allen Chen [Fri, 3 Oct 2025 02:53:34 +0000 (22:53 -0400)]
Fix mssql index column order
Fixed issue where the index reflection for SQL Server would
not correctly return the order of the column inside an index
when the order of the columns in the index did not match the
order of the columns in the table.
Pull request courtesy of Allen Chen.
Mike Bayer [Thu, 9 Oct 2025 16:43:16 +0000 (12:43 -0400)]
dont generate skipped suite names
For a suite that is ``__backend__``, if no tests will run, then
don't include that suite in the result at all, rather than
keeping them there and having them skip. the skip suites are
producing lots of noise in junit files
Mike Bayer [Wed, 8 Oct 2025 15:09:59 +0000 (11:09 -0400)]
correct for FK reflecting on PK / index col at the same time
Fixed issue in the MSSQL dialect's foreign key reflection query where
duplicate rows could be returned when a foreign key column and its
referenced primary key column have the same name, and both the referencing
and referenced tables have indexes with the same name. This resulted in an
"ForeignKeyConstraint with duplicate source column references are not
supported" error when attempting to reflect such tables. The query has been
corrected to exclude indexes on the child table when looking for unique
indexes referenced by foreign keys.
Mike Bayer [Tue, 7 Oct 2025 18:55:44 +0000 (14:55 -0400)]
Add Executable traverse internals to Executable subclasses and turn tests on
Fixed a caching issue where :func:`_orm.with_loader_criteria` would
incorrectly reuse cached bound parameter values when used with
:class:`_sql.CompoundSelect` constructs such as :func:`_sql.union`. The
issue was caused by the cache key for compound selects not including the
execution options that are part of the :class:`_sql.Executable` base class,
which :func:`_orm.with_loader_criteria` uses to apply its criteria
dynamically. The fix ensures that compound selects and other executable
constructs properly include execution options in their cache key traversal.
Denis Laxalde [Fri, 3 Oct 2025 19:10:08 +0000 (15:10 -0400)]
Add type annotations to indexable extension code
A typing test case (plain_files/ext/indexable.py) is also added.
In order to make the methods of index_property conform with type
definitions of `fget`, `fset` and `fdel` arguments of hybrid_property,
we need to make the signature of protocols
(e.g. `_HybridGetterType`) `__call__`) method positional only.
Add explicit multi-threaded tests and support free-threaded build
Implemented initial support for free-threaded Python by adding new tests
and reworking the test harness and GitHub Actions to include Python 3.13t
and Python 3.14t in test runs. Two concurrency issues have been identified
and fixed: the first involves initialization of the ``.c`` collection on a
``FromClause``, a continuation of :ticket:`12302`, where an optional mutex
under free-threading is added; the second involves synchronization of the
pool "first_connect" event, which first received thread synchronization in
:ticket:`2964`, however under free-threading the creation of the mutex
itself runs under the same free-threading mutex. Initial pull request and
test suite courtesy Lysandros Nikolaou.
Pablo Estevez [Sat, 23 Aug 2025 12:33:47 +0000 (08:33 -0400)]
type pysqlite
<!-- Provide a general summary of your proposed changes in the Title field above -->
type pysqlite from dialects.
type some related code on pysqlite.py
related to #6810
### Description
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
fix: pass dialect-specific kwargs to MetaData.reflect
Fixed issue where :meth:`_schema.MetaData.reflect` did not forward
dialect-specific keyword arguments to the :class:`_engine.Inspector`
methods, causing options like ``oracle_resolve_synonyms`` to be ignored
during reflection. The method now ensures that all extra kwargs passed to
:meth:`_schema.MetaData.reflect` are forwarded to
:meth:`_engine.Inspector.get_table_names` and related reflection methods.
Pull request courtesy Lukáš Kožušník.
Named types such as :class:`_postgresql.ENUM` and
:class:`_postgresql.DOMAIN` (as well as the dialect-agnostic
:class:`_types.Enum` version) are now more strongly associated with the
:class:`_schema.MetaData` at the top of the table hierarchy and are
de-associated with any particular :class:`_schema.Table` they may be a part
of. This better represents how PostgreSQL named types exist independently
of any particular table, and that they may be used across many tables
simultaneously. The change impacts the behavior of the "default schema"
for a named type, as well as the CREATE/DROP behavior in relationship to
the :class:`.MetaData` and :class:`.Table` construct. The change also
includes a new :class:`.CheckFirst` enumeration which allows fine grained
control over "check" queries during DDL operations, as well as that the
:paramref:`_types.SchemaType.inherit_schema` parameter is deprecated and
will emit a deprecation warning when used. See the migration notes for
full details.
Mike Bayer [Sun, 21 Sep 2025 17:54:13 +0000 (13:54 -0400)]
use standard path for asyncio create w/ exception handler
Refactored all asyncio dialects so that exceptions which occur on failed
connection attempts are appropriately wrapped with SQLAlchemy exception
objects, allowing for consistent error handling.
Mike Bayer [Sat, 20 Sep 2025 20:22:28 +0000 (16:22 -0400)]
Improve asyncpg exception hierarchy and asyncio hierarchies overall
The "emulated" exception hierarchies for the asyncio
drivers such as asyncpg, aiomysql, aioodbc, etc. have been standardized
on a common base :class:`.EmulatedDBAPIException`, which is now what's
available from the :attr:`.StatementException.orig` attribute on a
SQLAlchemy :class:`.DBAPIException` object. Within :class:`.EmulatedDBAPIException`
and the subclasses in its hiearchy, the original driver-level exception is
also now avaliable via the :attr:`.EmulatedDBAPIException.orig` attribute,
and is also available from :class:`.DBAPIException` directly using the
:attr:`.DBAPIException.driver_exception` attribute.
Added additional emulated error classes for the subclasses of
``asyncpg.exception.IntegrityError`` including ``RestrictViolationError``,
``NotNullViolationError``, ``ForeignKeyViolationError``,
``UniqueViolationError`` ``CheckViolationError``,
``ExclusionViolationError``. These exceptions are not directly thrown by
SQLAlchemy's asyncio emulation, however are available from the
newly added :attr:`.DBAPIException.driver_exception` attribute when a
:class:`.IntegrityError` is caught.
Mike Bayer [Sun, 21 Sep 2025 16:13:21 +0000 (12:13 -0400)]
add create_type to Enum
Added new parameter :paramref:`.Enum.create_type` to the Core
:class:`.Enum` class. This parameter is automatically passed to the
corresponding :class:`_postgresql.ENUM` native type during DDL operations,
allowing control over whether the PostgreSQL ENUM type is implicitly
created or dropped within DDL operations that are otherwise targeting
tables only. This provides control over the
:paramref:`_postgresql.ENUM.create_type` behavior without requiring
explicit creation of a :class:`_postgresql.ENUM` object.
Mike Bayer [Sun, 21 Sep 2025 14:59:51 +0000 (10:59 -0400)]
Apply nested for joined eager m2o w/ GROUP BY, DISTINCT
Fixed issue where joined eager loading would fail to use the "nested" form
of the query when GROUP BY or DISTINCT were present if the eager joins
being added were many-to-ones, leading to additional columns in the columns
clause which would then cause errors. The check for "nested" is tuned to
be enabled for these queries even for many-to-one joined eager loaders, and
the "only do nested if it's one to many" aspect is now localized to when
the query only has LIMIT or OFFSET added.
Mike Bayer [Sat, 20 Sep 2025 22:31:25 +0000 (18:31 -0400)]
remove _py3k suffixes from test files
on the theme of 2.0/2.1 are relatively similar right now, do
some more cleanup removing the py3k suffixes that we dont need anymore.
I'm not sure that the logic that would search for these suffixes is
even present anymore and I would guess we removed it when we removed
py2k support. However, I am planning on possibly bringing it back
for some of the py314 stuff, so I still want to be able to do
suffix stuff. that will be for another patch.
The :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods for the :class:`_types.ARRAY`
type are now deprecated for removal; these two methods along with
:func:`_postgresql.Any` and :func:`_postgresql.All` have been legacy for
some time as they are superseded by the :func:`_sql.any_` and
:func:`_sql.all_` functions, which feature more intutive use.
Mike Bayer [Sat, 20 Sep 2025 18:08:55 +0000 (14:08 -0400)]
Use ARRAY type for any_(), all_() coercion
Fixed issue where the :func:`_sql.any_` and :func:`_sql.all_` aggregation
operators would not correctly coerce the datatype of the compared value, in
those cases where the compared value were not a simple int/str etc., such
as a Python ``Enum`` or other custom value. This would lead to execution
time errors for these values. This issue is essentially the same as
:ticket:`6515` which was for the now-legacy :meth:`.ARRAY.any` and
:meth:`.ARRAY.all` methods.
raise for ENUM/DOMAIN with reserved type name and no schema
A :class:`.CompileError` is raised if attempting to create a PostgreSQL
:class:`_postgresql.ENUM` or :class:`_postgresql.DOMAIN` datatype using a
name that matches a known pg_catalog datatype name, and a default schema is
not specified. These types must be explicit within a schema in order to
be differentiated from the built-in pg_catalog type. The "public" or
otherwise default schema is not chosen by default here since the type can
only be reflected back using the explicit schema name as well (it is
otherwise not visible due to the pg_catalog name). Pull request courtesy
Kapil Dagur.
We originally thought we were going to do some default logic for the
default / "public" schema however this produces a type that is not
symmetric to its reflection, since the schema name must be explicit
for our current reflection queries.
So since it's an extremely bad idea to make an ENUM/DOMAIN with a
reserved type name anyway, we raise a compileerror if the type
has a known name. this is not robust against other names that
might exist in pg_catalog or other schemas that are in the search
path with these names. People just have to know what they're doing
here, the error here only covers a small subset of real world cases.
Mike Bayer [Sun, 31 Aug 2025 22:53:01 +0000 (18:53 -0400)]
add RegistryEvents
Added :class:`_orm.RegistryEvents` event class that allows event listeners
to be established on a :class:`_orm.registry` object. The new class
provides three events: :meth:`_orm.RegistryEvents.resolve_type_annotation`
which allows customization of type annotation resolution that can
supplement or replace the use of the
:paramref:`.registry.type_annotation_map` dictionary, including that it can
be helpful with custom resolution for complex types such as those of
:pep:`695`, as well as :meth:`_orm.RegistryEvents.before_configured` and
:meth:`_orm.RegistryEvents.after_configured`, which are registry-local
forms of the mapper-wide version of these hooks.
note this change was prematurely merged in 7111dc0cbaae and reverted
for re-review and additional changes.
Added new generalized aggregate function ordering to functions via the
:func:`_functions.FunctionElement.aggregate_order_by` method, which
receives an expression and generates the appropriate embedded "ORDER BY" or
"WITHIN GROUP (ORDER BY)" phrase depending on backend database. This new
function supersedes the use of the PostgreSQL
:func:`_postgresql.aggregate_order_by` function, which remains present for
backward compatibility. To complement the new parameter, the
:paramref:`_functions.aggregate_strings.order_by` which adds ORDER BY
capability to the :class:`_functions.aggregate_strings` dialect-agnostic
function which works for all included backends. Thanks much to Reuven
Starodubski with help on this patch.
Mike Bayer [Mon, 14 Jul 2025 17:50:09 +0000 (13:50 -0400)]
unmapped_dataclass
Reworked the handling of classes which extend from MappedAsDataclass but
are not themselves mapped, i.e. the declarative base as well as any
mixins or abstract classes. These classes as before are turned into
real dataclasses, however a scan now takes place across the mapped
elements such as mapped_column(), relationship(), etc. so that we may
also take into account dataclasses.field-specific parameters like
init=False, repr, etc. The main use case for this is so that mixin
dataclasses may make use of "default" in fields while not being rejected
by the dataclasses constructor. The generated classes are more
functional as dataclasses in a standalone fashion as well, even though
this is not their intended use. As a standalone dataclass, the one
feature that does not work is a field that has
a default with init=False, because we still need to have a
mapped_column() or similar present at the class level for the class
to work as a superclass.
The change also addes the :func:`_orm.unmapped_dataclass` decorator
function, which may be used
to create unmapped superclasses in a mapped hierarchy that is using the
:func:`_orm.mapped_dataclass` decorator to create mapped dataclasses.
Previously there was no way to use unmapped dataclass mixins with
the decorator approach.
Finally, the warning added in 2.0 for :ticket:`9350` is turned into
an error as mentioned for 2.1, since we're deep into dataclass hierarchy
changes here.
Tip ten Brink [Mon, 15 Sep 2025 12:58:40 +0000 (08:58 -0400)]
Fix get_columns sqlite reflection rejecting tables with WITHOUT_ROWID and/or STRICT for generated column case
Fixed issue where SQLite table reflection would fail for tables using
``WITHOUT ROWID`` and/or ``STRICT`` table options when the table contained
generated columns. The regular expression used to parse ``CREATE TABLE``
statements for generated column detection has been updated to properly
handle these SQLite table options that appear after the column definitions.
Pull request courtesy Tip ten Brink.
Mike Bayer [Tue, 9 Sep 2025 19:16:45 +0000 (15:16 -0400)]
Add function mapped_as_dataclass
Added new decorator :func:`_orm.mapped_as_dataclass`, which is a function
based form of :meth:`_orm.registry.mapped_as_dataclass`; the method form
:meth:`_orm.registry.mapped_as_dataclass` does not seem to be correctly
recognized within the scope of :pep:`681` in recent mypy versions.
The new function is tested and mentioned in the docs, in 2.1 in a
subsequent patch (probably the one that adds unmapped_dataclass also)
we'll switch this new decorator to be the prominent one.
also alphabetize mapping_api.rst. while the summary box at the top
auto-sorts, have the sidebar alpha also, it's kind of weird how
these were in no order at all
Mike Bayer [Sun, 31 Aug 2025 22:53:01 +0000 (18:53 -0400)]
add RegistryEvents
Added :class:`_orm.RegistryEvents` event class that allows event listeners
to be established on a :class:`_orm.registry` object. The new class
provides three events: :meth:`_orm.RegistryEvents.resolve_type_annotation`
which allows customization of type annotation resolution that can
supplement or replace the use of the
:paramref:`.registry.type_annotation_map` dictionary, including that it can
be helpful with custom resolution for complex types such as those of
:pep:`695`, as well as :meth:`_orm.RegistryEvents.before_configured` and
:meth:`_orm.RegistryEvents.after_configured`, which are registry-local
forms of the mapper-wide version of these hooks.
Mike Bayer [Sun, 7 Sep 2025 00:20:00 +0000 (20:20 -0400)]
doc updates, localize test fixtures
testing with types is inherently awkward and subject
to changes in python interpreters (such as all the recent python 3.14
stuff we had them fix), but in this suite we already have a lot of
types that are defined inline inside of test methods. so since that's
how many of the tests work anyway, organize the big series of pep-695
and pep-593 structures into fixtures or individual tests to make
the whole suite easier to follow. pyright complains quite a lot
about this, so if this becomes a bigger issue for say mypy /pep484
target, we may have to revisit (which I'd likely do with more ignores)
or if function/method-local type declarations with global becomes a runtime
issue in py3.15 or something, we can revisit then where we would in
theory need to convert the entire suite, which I'd do with a more
consistent naming style for everything.
but for now try to go with fixtures / local type declarations so that
we dont have to wonder where all these types are used.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
