Mike Bayer [Wed, 7 Jan 2026 01:03:10 +0000 (20:03 -0500)]
apply Grouping on left side of JSONB subscript in compiler
Fixed regression in PostgreSQL dialect where JSONB subscription syntax
would generate incorrect SQL for :func:`.cast` expressions returning JSONB,
causing syntax errors. The dialect now properly wraps cast expressions in
parentheses when using the ``[]`` subscription syntax, generating
``(CAST(...))[index]`` instead of ``CAST(...)[index]`` to comply with
PostgreSQL syntax requirements. This extends the fix from :ticket:`12778`
which addressed the same issue for function calls.
This reverts how we did the fix for #12778 in Function.self_group()
and instead moves to a direct Grouping() applied in the PG compiler
based on isinstance of the left side.
in retrospect, when we first did #10927, we **definitely** made
the completely wrong choice in how to do this, the original idea
to detect when we were in an UPDATE and use [] only then was
by **far** what we should have done, given the fact that PG indexes
are based on exact syntax matches. but since we've made everyone
switch to [] format for their indexes now we can't keep going
back and forth. even though PG would like [] to be the defacto
syntax it simply is not. We should potentially pursue a dialect/
create_engine option to switch the use of [] back to -> for
all cases except UPDATE.
Federico Caselli [Wed, 24 Dec 2025 13:02:17 +0000 (14:02 +0100)]
Support aiosqlite 0.22.0+
Fixed issue in the aiosqlite driver where SQLAlchemy's setting of
aiosqlite's worker thread to "daemon" stopped working because the aiosqlite
architecture moved the location of the worker thread in version 0.22.0.
This "daemon" flag is necessary so that a program is able to exit if the
SQLite connection itself was not explicitly closed, which is particularly
likely with SQLAlchemy as it maintains SQLite connections in a connection
pool. While it's perfectly fine to call :meth:`.AsyncEngine.dispose`
before program exit, this is not historically or technically necessary for
any driver of any known backend, since a primary feature of relational
databases is durability. The change also implements support for
"terminate" with aiosqlite when using version version 0.22.1 or greater,
which implements a sync ``.stop()`` method.
Mike Bayer [Wed, 31 Dec 2025 20:48:44 +0000 (15:48 -0500)]
Fixed JSONB path_match and path_exists operators to use correct type coercion
Fixed issue where PostgreSQL JSONB operators
:meth:`_postgresql.JSONB.Comparator.path_match` and
:meth:`_postgresql.JSONB.Comparator.path_exists` were applying incorrect
``VARCHAR`` casts to the right-hand side operand when used with newer
PostgreSQL drivers such as psycopg. The operators now indicate the
right-hand type as ``JSONPATH``, which currently results in no casting
taking place, but is also compatible with explicit casts if the
implementation were require it at a later point.
Support for `IF EXISTS` in SQL Server 2016 (13.x) and later versions
Added support for the ``IF EXISTS`` clause when dropping indexes on SQL
Server 2016 (13.x) and later versions. The :paramref:`.DropIndex.if_exists`
parameter is now honored by the SQL Server dialect, allowing conditional
index drops that will not raise an error if the index does not exist.
Pull request courtesy Edgar Ramírez Mondragón.
<!-- Provide a general summary of your proposed changes in the Title field above -->
<!-- Describe your changes in detail -->
`Query[_T].get(...)` should return `Optional[_T]` instead of `Optional[Any]`. This is typed correctly when migrating to `Session.get(_T, ...)`. By typing the legacy `Query.get(...)` call first, it should make migrations easier on developers, as it splits up the type checking improvements (and subsequent errors which may be discovered) from the `Query.get()`
to `Session.get()` migration.
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
G Allajmi [Tue, 9 Dec 2025 19:13:52 +0000 (14:13 -0500)]
Factor out constraints into separate methods
Fixed issue where PostgreSQL dialect options such as ``postgresql_include``
on :class:`.PrimaryKeyConstraint` and :class:`.UniqueConstraint` were
rendered in the wrong position when combined with constraint deferrability
options like ``deferrable=True``. Pull request courtesy G Allajmi.
G Allajmi [Mon, 8 Dec 2025 13:03:49 +0000 (08:03 -0500)]
Fix adding property to mapper before mapping is complete
Fixed issue where calling :meth:`.Mapper.add_property` within mapper event
hooks such as :meth:`.MapperEvents.instrument_class`,
:meth:`.MapperEvents.after_mapper_constructed`, or
:meth:`.MapperEvents.before_mapper_configured` would raise an
``AttributeError`` because the mapper's internal property collections were
not yet initialized. The :meth:`.Mapper.add_property` method now handles
early-stage property additions correctly, allowing properties including
column properties, deferred columns, and relationships to be added during
mapper initialization events. Pull request courtesy G Allajmi.
Mike Bayer [Fri, 5 Dec 2025 22:41:56 +0000 (17:41 -0500)]
fix / modernize short_selects example
Fixed the "short_selects" performance example where the cache was being
used in all the examples, making it impossible to compare performance with
and without the cache. Less important comparisons like "lambdas" and
"baked queries" have been removed.
Mike Bayer [Wed, 3 Dec 2025 19:48:08 +0000 (14:48 -0500)]
Add a test for #13021
Confirmed the upstream fix for [1] given at [2] solves the issue
illustrated here, this patch adds a test for this case as our
existing tests did not catch this error in python 3.14.1.
Mike Bayer [Mon, 1 Dec 2025 20:11:50 +0000 (15:11 -0500)]
run sentinel server side fns outside of VALUES
Fixed the structure of the SQL string used for the
:ref:`engine_insertmanyvalues` feature when an explicit sequence with
``nextval()`` is used. The SQL function invocation for the sequence has
been moved from being rendered inline within each tuple inside of VALUES to
being rendered once in the SELECT that reads from VALUES. This change
ensures the function is invoked in the correct order as rows are processed,
rather than assuming PostgreSQL will execute inline function calls within
VALUES in a particular order. While current PostgreSQL versions appear to
handle the previous approach correctly, the database does not guarantee
this behavior for future versions.
Yossi [Mon, 1 Dec 2025 17:06:12 +0000 (12:06 -0500)]
[typing] Fix type error when passing Mapped columns to values()
This adjusts the _DMLOnlyColumnArgument type to be a more
focused _OnlyColumnArgument type where we also add a more tightly
focused coercion, while still allowing ORM attributes to be used
as arguments.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #13012
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13012
Pull-request-sha: 5ebb402c686abf1090e5b83e3489dfca4908efdf
Mike Bayer [Thu, 27 Nov 2025 06:22:38 +0000 (01:22 -0500)]
drop a 400 ton anvil on oracle 23c
this DB is extremely erratic in being able to connect. Add
a brute force connection retrier to all engines everywhere
(which for oracledb we can fortunately use their built-in feature
that also works).
This actually works and I can see it pausing under load, reconnecting,
and succeeding. the problem is that absolutely every engine everywhere
needs this routine otherwise an engine without a retrier in it will
crash. That then necessitates digging into testing_engine(),
making sure testing_engine() is used everywhere an engine that's going
to connect is used, then dealing with the fallout from that.
We also simplify some older workarounds for cx_oracle and
hack into config/provision to make oracledb seem like the primary
DBAPI for most tests.
testing_engine has been completely overhauled, making use of a new
post_configure_testing_engine() hook which moves and refines
the SQLite pool sharing and savepoint logic all into sqlite/provision.py
and also allows for cx_oracle to apply a retry event handler.
Mike Bayer [Wed, 26 Nov 2025 06:36:57 +0000 (01:36 -0500)]
stop using MyISAM; more oracle struggles
getting some fails on mariadb12 and likely 11 which appear to
be related to calling in MyISAM, which is not used in
modern mysql/mariadb. see if we can just remove this whole
thing and rely on default engines for mariadb/mysql.
this change also removes the "ignore errors" part of the
run deletes for the TablesTest fixture, which was resulting
in compound failures, and apparently a lot of tests were relying
on it skipping nonexistent tables. rather than check for that
we should just improve the tests and probably increase use of
pytest style fixtures overall.
this change also identifies and fixes that memusage_w_backend
tests were running for all backends with a tag like
py314_mysql_backendonly; the memusage tests should basically
never be run as part of the whole suite since they are entirely
unreliable within a full scale test run.
dialect suite tests are also further broken out into those where
every driver should be exercised (i.e. __backend__, for tests that
test datatypes going out and coming back from the database as well
as identity/autoincrement kinds of tests) vs. those where only
one driver per backend is needed (i.e. __sparse_driver_backend__,
for tests like reflection, DDL, CTEs, etc.).
we are also trying to get a --low-connections option that actually
works. changed this so that the testing reaper aggressively disposes
the "global" engines (one per backend / driver) after test classes
are done and before any testing_engine() call. This definitely
works, however some monitoring with PG shows the number of connections
still has brief bursts for some reason. it should be much more
effective than before though as oracle 23/26 really does not handle
more than a few connections.
this change reverts oracle to oracle18c for now in setup.cfg;
further work will be needed to determine if oracle23c can be
run with this test suite
Mike Bayer [Tue, 25 Nov 2025 00:33:18 +0000 (19:33 -0500)]
test oracle 23c, mariadb12; reduce backend use
one particular vector test wont run on oracle 23c free, so
just disable it.
added better skips for the rest of the vector tests and
fixed a deprecation issue.
this will be the first run on the new oracle23 on CI so we'll have to
see how this goes.
Also adjust for mariabdb12 being overly helpful with regards
to stale row updates.
as we are having trouble getting 23c to pass throug transaction
tests, i noted we have an explosion of tests due to the multiple
drivers, so this patch introduces __sparse_driver_backend__
for all tests where we want variety of
database server but there's no need to test every driver.
This should dramatically reduce the size of the test suite run
Shamil [Wed, 19 Nov 2025 13:02:28 +0000 (08:02 -0500)]
Fix type hint for with_for_update() to support tuples of table classes
Fixed typing issue where :meth:`.Select.with_for_update` would not support
lists of ORM entities in the :paramref:`.Select.with_for_update.of`
parameter. Pull request courtesy Shamil.
Mike Bayer [Tue, 18 Nov 2025 20:10:05 +0000 (15:10 -0500)]
propagate _scalar_type() for SelectStatementGrouping
Fixed issue where using the :meth:`.ColumnOperators.in_` operator with a
nested :class:`.CompoundSelect` statement (e.g. an ``INTERSECT`` of
``UNION`` queries) would raise a :class:`NotImplementedError` when the
nested compound select was the first argument to the outer compound select.
The ``_scalar_type()`` internal method now properly handles nested compound
selects.
Mike Bayer [Mon, 17 Nov 2025 21:32:54 +0000 (16:32 -0500)]
update lint setup
We are stuck on flake8 because we rely on many plugins with
specific behaviors. The situation has calcified where:
1. the whole world uses ruff
2. nobody cares about import order linting or all the other stuff
we do, and/or similar but not quite the same things are embedded
deeply into ruff which would require us giving up a lot of our
standards (like isort)
3. flake8 is absolutely never going to support pyproject.
4. flake8-pyproject works for this
beyond that, for t string support we want to make it easy
to get onto py3.14, so here we update black to the latest which
appears to fix some missing symbols for py3.14 t strings.
we should also migrate the remaining sqlalchemy test config
from setup.cfg to pyproject.toml but that should likely be
2.1 only
Mike Bayer [Sat, 15 Nov 2025 16:41:34 +0000 (11:41 -0500)]
bump nox to latest main + fix
apparently 2.0 was behind so this copies
the files with 2.0-specific modifications from the below change id
add pyv to file template; use = for all custom args
adding "test/" to pytest doesnt work because then we can't indicate
a specific set of test files. use = for all sqlalchemy-custom
parameters instead to avoid [1]
Yannick PÉROUX [Tue, 4 Nov 2025 17:58:03 +0000 (12:58 -0500)]
Typing: fix type of func.coalesce when used with hybrid properties
Fixed typing issue where :class:`.coalesce` would not return the correct
return type when a nullable form of that argument were passed, even though
this function is meant to select the non-null entry among possibly null
arguments. Pull request courtesy Yannick PÉROUX.
JetDrag [Wed, 5 Nov 2025 03:47:02 +0000 (22:47 -0500)]
feat: Support MySQL FOR SHARE locking syntax.
Added support for MySQL 8.0.1 + ``FOR SHARE`` to be emitted for the
:meth:`.Select.with_for_uddate` method, which offers compatibility with
``NOWAIT`` and ``SKIP LOCKED``. The new syntax is used only for MySQL when
version 8.0.1 or higher is detected. Pull request courtesy JetDrag.
Mike Bayer [Tue, 4 Nov 2025 14:13:45 +0000 (09:13 -0500)]
clarify Core / ORM insert parameter behaviors
it seems to have gotten lost in our newer docs that we're looking
at the first dict only for core insert. add sections to both
INSERT tutorials explaining this difference
Pat Buxton [Mon, 3 Nov 2025 14:49:33 +0000 (09:49 -0500)]
Add order by clause to dialect tests to ensure expected result order
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Failing for Starrocks dialect currently without overrides, the amended dialect tests require an order by clause to ensure the expected result.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
Mike Bayer [Sat, 1 Nov 2025 13:56:18 +0000 (09:56 -0400)]
use the default driver for sparse backend
the memusage tests use sparse_backend but should use the
main driver in a set; they were using the pysqlite_numeric
dialect which is not a real dialect and apparently runs
dramatically slower for the memusage tests since it generates
more memory artifacts.
Mike Bayer [Sat, 1 Nov 2025 02:57:28 +0000 (22:57 -0400)]
fix sqlite regex for quoted fk, pk names
Fixed issue where SQLite dialect would fail to reflect constraint names
that contained uppercase letters or other characters requiring quoting. The
regular expressions used to parse primary key, foreign key, and unique
constraint names from the ``CREATE TABLE`` statement have been updated to
properly handle both quoted and unquoted constraint names.
Mike Bayer [Fri, 31 Oct 2025 15:51:37 +0000 (11:51 -0400)]
ensure util.get_annotations() is used
Fixed issue in Python 3.14 where dataclass transformation would fail when
a mapped class using :class:`.MappedAsDataclass` included a
:func:`.relationship` referencing a class that was not available at
runtime (e.g., within a ``TYPE_CHECKING`` block). This occurred when using
Python 3.14's :pep:`649` deferred annotations feature, which is the
default behavior without a ``from __future__ import annotations``
directive.
Mike Bayer [Tue, 28 Oct 2025 20:20:54 +0000 (16:20 -0400)]
try not to rely on GC working at all for tests
GC seems to be working in fewer cases across different interpreters,
so add a new step to force an InstanceState to act as though
it got dereferenced, for those cases where we have del'ed the object,
done a gc collect, and we know it should not be reachable.
Mike Bayer [Tue, 28 Oct 2025 19:22:45 +0000 (15:22 -0400)]
set nox min version only for newer sys python
The min version here fails on github actions because the runners
there use the single version of python for the install, and the newer
nox is not available on older pythons like 3.7.
We probably dont need the min version anyway, as since we have just
switched to venv in 5e62ea65167c, the misinterpretation of
"python3.14t" doesn't seem to happen anyway. however, all versions
of nox seem willing to install a 3.14t interpreter when 3.14 is
requested, and a 3.14t is sooner in the path. does not seem to
happen the other way around though (e.g. when 3.14t is requested,
it does not use a 3.14)
Anyway, for better predictability leave a min nox version in place
where we can do so for now.
Mike Bayer [Tue, 28 Oct 2025 14:03:09 +0000 (10:03 -0400)]
rewrite pool subsecond test to use mocking
the timing here is too sensitive to play out reliably
on CI machines particularly free threaded, so mock the time()
callable instead and ensure with subsecond clock intervals
we do the right math
Mike Bayer [Tue, 28 Oct 2025 18:06:14 +0000 (14:06 -0400)]
use the nox venv backend
This backend seems to run from the start against the resolved
interpreter, which we want. The virtualenv option, which is
the default, has the problem indicated at [1].
Mike Bayer [Sat, 18 Oct 2025 20:43:22 +0000 (16:43 -0400)]
improve sqlite reflection regex for unusual names/formats
A series of improvements have been made for reflection of CHECK constraints
on SQLite. The reflection logic now correctly handles table names
containing the strings "CHECK" or "CONSTRAINT", properly supports all four
SQLite identifier quoting styles (double quotes, single quotes, brackets,
and backticks) for constraint names, and accurately parses CHECK constraint
expressions containing parentheses within string literals using balanced
parenthesis matching with string context tracking. Big thanks to
GruzdevAV for new test cases and implementation ideas.
Mike Bayer [Mon, 29 Sep 2025 03:44:41 +0000 (23:44 -0400)]
add nox support (but dont switch out fully)
This backports the nox change from main/2.1 so that we have full
nox support available for the 2.0 series, however does not modify
any existing docs or test systems.
A noxfile.py has been added to allow testing with nox. This is a direct
port of 2.1's move to nox, however leaves the tox.ini file in place and
retains all test documentation in terms of tox. Version 2.1 will move to
nox fully, including deprecation warnings for tox and new testing
documentation.
krave1986 [Mon, 20 Oct 2025 20:06:31 +0000 (04:06 +0800)]
Fix missing back_populates in Note.item relationship in example code (#12925)
Without back_populates, the subsequent code would not automatically populate Item.notes or generate the key, which contradicts what the documentation intends to demonstrate.
krave1986 [Wed, 15 Oct 2025 19:37:10 +0000 (03:37 +0800)]
Update collection_api.rst (#12912)
In the Dictionary Collections section, the example code incorrectly calls .items() on the dictionary but shows dictionary output instead of the items() method's actual return value.
Mike Bayer [Wed, 15 Oct 2025 18:47:38 +0000 (14:47 -0400)]
fully copy_internals for AnnotatedFromClause for straight cloned traverse
Fixed issue where using :meth:`_sql.Select.params` to replace bound
parameters in a query could fail for some cases where the parameters
were embedded in subqueries or CTEs when ORM classes were involved,
due to issues with internal query traversal for these cases.
Mike Bayer [Tue, 14 Oct 2025 21:19:26 +0000 (17:19 -0400)]
Support warnings in exclusions
this adds a new feature to exclusions ``warns_if()`` which applies
the expect_warnings() context manager to a test method. Additionally,
at the class level these requirements can be extracted from a
``__requirements__`` directive and also added to global Python warnings
filter using catch_warnings().
Allen Chen [Fri, 3 Oct 2025 02:53:34 +0000 (22:53 -0400)]
Fix mssql index column order
Fixed issue where the index reflection for SQL Server would
not correctly return the order of the column inside an index
when the order of the columns in the index did not match the
order of the columns in the table.
Pull request courtesy of Allen Chen.
Mike Bayer [Thu, 9 Oct 2025 16:43:16 +0000 (12:43 -0400)]
dont generate skipped suite names
For a suite that is ``__backend__``, if no tests will run, then
don't include that suite in the result at all, rather than
keeping them there and having them skip. the skip suites are
producing lots of noise in junit files
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
