Mike Bayer [Wed, 3 Dec 2025 19:48:08 +0000 (14:48 -0500)]
Add a test for #13021
Confirmed the upstream fix for [1] given at [2] solves the issue
illustrated here, this patch adds a test for this case as our
existing tests did not catch this error in python 3.14.1.
Mike Bayer [Wed, 3 Dec 2025 04:10:10 +0000 (23:10 -0500)]
send same sub_stmt to after_cursor_execute as before
Fixed issue in the :meth:`.ConnectionEvents.after_cursor_execute` method
where the SQL statement and parameter list for an "insertmanyvalues"
operation sent to the event would not be the actual SQL / parameters just
emitted on the cursor, instead being the non-batched form of the statement
that's used as a template to generate the batched statements.
Mike Bayer [Mon, 1 Dec 2025 20:11:50 +0000 (15:11 -0500)]
run sentinel server side fns outside of VALUES
Fixed the structure of the SQL string used for the
:ref:`engine_insertmanyvalues` feature when an explicit sequence with
``nextval()`` is used. The SQL function invocation for the sequence has
been moved from being rendered inline within each tuple inside of VALUES to
being rendered once in the SELECT that reads from VALUES. This change
ensures the function is invoked in the correct order as rows are processed,
rather than assuming PostgreSQL will execute inline function calls within
VALUES in a particular order. While current PostgreSQL versions appear to
handle the previous approach correctly, the database does not guarantee
this behavior for future versions.
Yossi [Mon, 1 Dec 2025 17:06:12 +0000 (12:06 -0500)]
[typing] Fix type error when passing Mapped columns to values()
This adjusts the _DMLOnlyColumnArgument type to be a more
focused _OnlyColumnArgument type where we also add a more tightly
focused coercion, while still allowing ORM attributes to be used
as arguments.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #13012
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13012
Pull-request-sha: 5ebb402c686abf1090e5b83e3489dfca4908efdf
Yeongbae Jeon [Sun, 23 Nov 2025 11:44:43 +0000 (06:44 -0500)]
Add native BOOLEAN type support for Oracle 23c and later versions
Added support for native BOOLEAN support in Oracle Database 23c and above.
The Oracle dialect now renders ``BOOLEAN`` automatically when
:class:`.Boolean` is used in DDL, and also now supports direct use of the
:class:`.BOOLEAN` datatype, when 23c and above is in use. For Oracle
versions prior to 23c, boolean values continue to be emulated using
SMALLINT as before. Special case handling is also present to ensure a
SMALLINT that's interpreted with the :class:`.Boolean` datatype on Oracle
Database 23c and above continues to return bool values. Pull request
courtesy Yeongbae Jeon.
Mike Bayer [Mon, 17 Nov 2025 18:11:24 +0000 (13:11 -0500)]
add TString support
Added support for Python 3.14+ template strings (t-strings) via the new
:func:`_sql.tstring` construct, as defined in :pep:`750`. This feature
allows for ergonomic SQL statement construction by automatically
interpolating Python values and SQLAlchemy expressions within template
strings.
Part of the challenge here is the syntax only works on py314, so we have
to exclude the test file at many levels when py314 is not used. not
sure yet how i want to adjust pep8 tests and rules for this.
Mike Bayer [Sun, 30 Nov 2025 16:39:04 +0000 (11:39 -0500)]
use oracledb as default in setup, update URLs
part of :ticket:`13010`, as CI is now oracledb centric
move setup.cfg / pyproject installs / URLs to be oracledb first.
use 23c URLs now that we are on this platform.
Mike Bayer [Thu, 27 Nov 2025 06:22:38 +0000 (01:22 -0500)]
drop a 400 ton anvil on oracle 23c
this DB is extremely erratic in being able to connect. Add
a brute force connection retrier to all engines everywhere
(which for oracledb we can fortunately use their built-in feature
that also works).
This actually works and I can see it pausing under load, reconnecting,
and succeeding. the problem is that absolutely every engine everywhere
needs this routine otherwise an engine without a retrier in it will
crash. That then necessitates digging into testing_engine(),
making sure testing_engine() is used everywhere an engine that's going
to connect is used, then dealing with the fallout from that.
We also simplify some older workarounds for cx_oracle and
hack into config/provision to make oracledb seem like the primary
DBAPI for most tests.
testing_engine has been completely overhauled, making use of a new
post_configure_testing_engine() hook which moves and refines
the SQLite pool sharing and savepoint logic all into sqlite/provision.py
and also allows for cx_oracle to apply a retry event handler.
Mike Bayer [Wed, 26 Nov 2025 06:36:57 +0000 (01:36 -0500)]
stop using MyISAM; more oracle struggles
getting some fails on mariadb12 and likely 11 which appear to
be related to calling in MyISAM, which is not used in
modern mysql/mariadb. see if we can just remove this whole
thing and rely on default engines for mariadb/mysql.
this change also removes the "ignore errors" part of the
run deletes for the TablesTest fixture, which was resulting
in compound failures, and apparently a lot of tests were relying
on it skipping nonexistent tables. rather than check for that
we should just improve the tests and probably increase use of
pytest style fixtures overall.
this change also identifies and fixes that memusage_w_backend
tests were running for all backends with a tag like
py314_mysql_backendonly; the memusage tests should basically
never be run as part of the whole suite since they are entirely
unreliable within a full scale test run.
dialect suite tests are also further broken out into those where
every driver should be exercised (i.e. __backend__, for tests that
test datatypes going out and coming back from the database as well
as identity/autoincrement kinds of tests) vs. those where only
one driver per backend is needed (i.e. __sparse_driver_backend__,
for tests like reflection, DDL, CTEs, etc.).
we are also trying to get a --low-connections option that actually
works. changed this so that the testing reaper aggressively disposes
the "global" engines (one per backend / driver) after test classes
are done and before any testing_engine() call. This definitely
works, however some monitoring with PG shows the number of connections
still has brief bursts for some reason. it should be much more
effective than before though as oracle 23/26 really does not handle
more than a few connections.
this change reverts oracle to oracle18c for now in setup.cfg;
further work will be needed to determine if oracle23c can be
run with this test suite
Mike Bayer [Tue, 25 Nov 2025 00:33:18 +0000 (19:33 -0500)]
test oracle 23c, mariadb12; reduce backend use
one particular vector test wont run on oracle 23c free, so
just disable it.
added better skips for the rest of the vector tests and
fixed a deprecation issue.
this will be the first run on the new oracle23 on CI so we'll have to
see how this goes.
Also adjust for mariabdb12 being overly helpful with regards
to stale row updates.
as we are having trouble getting 23c to pass throug transaction
tests, i noted we have an explosion of tests due to the multiple
drivers, so this patch introduces __sparse_driver_backend__
for all tests where we want variety of
database server but there's no need to test every driver.
This should dramatically reduce the size of the test suite run
Shamil [Thu, 20 Nov 2025 12:44:34 +0000 (07:44 -0500)]
Fix parameter mutation in orm_pre_session_exec()
The :meth:`_events.SessionEvents.do_orm_execute` event now allows direct
mutation or replacement of the :attr:`.ORMExecuteState.parameters`
dictionary or list, which will take effect when the the statement is
executed. Previously, changes to this collection were not accommodated by
the event hook. Pull request courtesy Shamil.
Shamil [Wed, 19 Nov 2025 13:02:28 +0000 (08:02 -0500)]
Fix type hint for with_for_update() to support tuples of table classes
Fixed typing issue where :meth:`.Select.with_for_update` would not support
lists of ORM entities in the :paramref:`.Select.with_for_update.of`
parameter. Pull request courtesy Shamil.
Mike Bayer [Tue, 18 Nov 2025 20:10:05 +0000 (15:10 -0500)]
propagate _scalar_type() for SelectStatementGrouping
Fixed issue where using the :meth:`.ColumnOperators.in_` operator with a
nested :class:`.CompoundSelect` statement (e.g. an ``INTERSECT`` of
``UNION`` queries) would raise a :class:`NotImplementedError` when the
nested compound select was the first argument to the outer compound select.
The ``_scalar_type()`` internal method now properly handles nested compound
selects.
Mike Bayer [Mon, 17 Nov 2025 21:32:54 +0000 (16:32 -0500)]
update lint setup
We are stuck on flake8 because we rely on many plugins with
specific behaviors. The situation has calcified where:
1. the whole world uses ruff
2. nobody cares about import order linting or all the other stuff
we do, and/or similar but not quite the same things are embedded
deeply into ruff which would require us giving up a lot of our
standards (like isort)
3. flake8 is absolutely never going to support pyproject.
4. flake8-pyproject works for this
beyond that, for t string support we want to make it easy
to get onto py3.14, so here we update black to the latest which
appears to fix some missing symbols for py3.14 t strings.
we should also migrate the remaining sqlalchemy test config
from setup.cfg to pyproject.toml but that should likely be
2.1 only
Mike Bayer [Sat, 15 Nov 2025 16:36:56 +0000 (11:36 -0500)]
add pyv to file template; use = for all custom args
adding "test/" to pytest doesnt work because then we can't indicate
a specific set of test files. use = for all sqlalchemy-custom
parameters instead to avoid [1]
Yannick PÉROUX [Tue, 4 Nov 2025 17:58:03 +0000 (12:58 -0500)]
Typing: fix type of func.coalesce when used with hybrid properties
Fixed typing issue where :class:`.coalesce` would not return the correct
return type when a nullable form of that argument were passed, even though
this function is meant to select the non-null entry among possibly null
arguments. Pull request courtesy Yannick PÉROUX.
JetDrag [Wed, 5 Nov 2025 03:47:02 +0000 (22:47 -0500)]
feat: Support MySQL FOR SHARE locking syntax.
Added support for MySQL 8.0.1 + ``FOR SHARE`` to be emitted for the
:meth:`.Select.with_for_uddate` method, which offers compatibility with
``NOWAIT`` and ``SKIP LOCKED``. The new syntax is used only for MySQL when
version 8.0.1 or higher is detected. Pull request courtesy JetDrag.
Federico Caselli [Thu, 16 Oct 2025 18:59:30 +0000 (20:59 +0200)]
Improve params implementation
Added new implementation for the :meth:`.Select.params` method and that of
similar statements, via a new statement-only
:meth:`.ExecutableStatement.params` method which works more efficiently and
correctly than the previous implementations available from
:class:`.ClauseElement`, by assocating the given parameter dictionary with
the statement overall rather than cloning the statement and rewriting its
bound parameters. The :meth:`_sql.ClauseElement.params` and
:meth:`_sql.ClauseElement.unique_params` methods, when called on an object
that does not implement :class:`.ExecutableStatement`, will continue to
work the old way of cloning the object, and will emit a deprecation
warning. This issue both resolves the architectural / performance
concerns of :ticket:`7066` and also provides correct ORM compatibility for
functions like :func:`_orm.aliased`, reported by :ticket:`12915`.
Inada Naoki [Wed, 29 Oct 2025 08:09:14 +0000 (04:09 -0400)]
remove use of `LABEL_STYLE_TABLENAME_PLUS_COL` outside of Query
Changed the query style for ORM queries emitted by :meth:`.Session.get` as
well as many-to-one lazy load queries to use the default labeling style,
:attr:`_sql.SelectLabelStyle.LABEL_STYLE_DISAMBIGUATE_ONLY`, which normally
does not apply labels to columns in a SELECT statement. Previously, the
older style :attr:`_sql.SelectLabelStyle.LABEL_STYLE_TABLENAME_PLUS_COL`
that labels columns as `<tablename>_<columname>` was used for
:meth:`.Session.get` to maintain compatibility with :class:`_orm.Query`.
The change allows the string representation of ORM queries to be less
verbose in all cases outside of legacy :class:`_orm.Query` use. Pull
request courtesy Inada Naoki.
Mike Bayer [Sun, 9 Nov 2025 18:14:48 +0000 (13:14 -0500)]
be more aggressive in removing all references
python3.14t is showing connections being referenced and not finalized.
explicit closes/disposes and such dont seem to be working but
ensuring globs are cleared out is working better
Mike Bayer [Sat, 25 Oct 2025 17:19:48 +0000 (13:19 -0400)]
Add DDL association to Table, CreateView support
Added support for the SQL ``CREATE VIEW`` statement via the new
:class:`.CreateView` DDL class. The new class allows creating database
views from SELECT statements, with support for options such as
``TEMPORARY``, ``IF NOT EXISTS``, and ``MATERIALIZED`` where supported by
the target database. Views defined with :class:`.CreateView` integrate with
:class:`.MetaData` for automated DDL generation and provide a
:class:`.Table` object for querying.
this alters the CreateTableAs commit:
* adds the ability for Table to be associated with Create and Drop
DDL constructs
* Adds CreateView variant of CreateTableAs
* Both associate themselves with Table so they take place in
create_all/create/drop_all/drop
Mike Bayer [Tue, 4 Nov 2025 14:13:45 +0000 (09:13 -0500)]
clarify Core / ORM insert parameter behaviors
it seems to have gotten lost in our newer docs that we're looking
at the first dict only for core insert. add sections to both
INSERT tutorials explaining this difference
Pat Buxton [Mon, 3 Nov 2025 14:49:33 +0000 (09:49 -0500)]
Add order by clause to dialect tests to ensure expected result order
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Failing for Starrocks dialect currently without overrides, the amended dialect tests require an order by clause to ensure the expected result.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
Mike Bayer [Sat, 1 Nov 2025 13:56:18 +0000 (09:56 -0400)]
use the default driver for sparse backend
the memusage tests use sparse_backend but should use the
main driver in a set; they were using the pysqlite_numeric
dialect which is not a real dialect and apparently runs
dramatically slower for the memusage tests since it generates
more memory artifacts.
Mike Bayer [Sat, 1 Nov 2025 02:57:28 +0000 (22:57 -0400)]
fix sqlite regex for quoted fk, pk names
Fixed issue where SQLite dialect would fail to reflect constraint names
that contained uppercase letters or other characters requiring quoting. The
regular expressions used to parse primary key, foreign key, and unique
constraint names from the ``CREATE TABLE`` statement have been updated to
properly handle both quoted and unquoted constraint names.
Mike Bayer [Fri, 31 Oct 2025 18:08:05 +0000 (14:08 -0400)]
support subscript for hstore
Added support for PostgreSQL 14+ HSTORE subscripting syntax. When connected
to PostgreSQL 14 or later, HSTORE columns now automatically use the native
subscript notation ``hstore_col['key']`` instead of the arrow operator
``hstore_col -> 'key'`` for both read and write operations. This provides
better compatibility with PostgreSQL's native HSTORE subscripting feature
while maintaining backward compatibility with older PostgreSQL versions.
as part of this change we add a new parameter to custom_op "visit_name"
which allows a custom op to refer to a specific visit method in a
dialect's compiler.
Mike Bayer [Fri, 31 Oct 2025 15:51:37 +0000 (11:51 -0400)]
ensure util.get_annotations() is used
Fixed issue in Python 3.14 where dataclass transformation would fail when
a mapped class using :class:`.MappedAsDataclass` included a
:func:`.relationship` referencing a class that was not available at
runtime (e.g., within a ``TYPE_CHECKING`` block). This occurred when using
Python 3.14's :pep:`649` deferred annotations feature, which is the
default behavior without a ``from __future__ import annotations``
directive.
Mike Bayer [Tue, 28 Oct 2025 20:20:54 +0000 (16:20 -0400)]
try not to rely on GC working at all for tests
GC seems to be working in fewer cases across different interpreters,
so add a new step to force an InstanceState to act as though
it got dereferenced, for those cases where we have del'ed the object,
done a gc collect, and we know it should not be reachable.
Mike Bayer [Tue, 28 Oct 2025 19:22:45 +0000 (15:22 -0400)]
set nox min version only for newer sys python
The min version here fails on github actions because the runners
there use the single version of python for the install, and the newer
nox is not available on older pythons like 3.7.
We probably dont need the min version anyway, as since we have just
switched to venv in 5e62ea65167c, the misinterpretation of
"python3.14t" doesn't seem to happen anyway. however, all versions
of nox seem willing to install a 3.14t interpreter when 3.14 is
requested, and a 3.14t is sooner in the path. does not seem to
happen the other way around though (e.g. when 3.14t is requested,
it does not use a 3.14)
Anyway, for better predictability leave a min nox version in place
where we can do so for now.
Mike Bayer [Tue, 28 Oct 2025 18:06:14 +0000 (14:06 -0400)]
use the nox venv backend
This backend seems to run from the start against the resolved
interpreter, which we want. The virtualenv option, which is
the default, has the problem indicated at [1].
Mike Bayer [Tue, 28 Oct 2025 14:03:09 +0000 (10:03 -0400)]
rewrite pool subsecond test to use mocking
the timing here is too sensitive to play out reliably
on CI machines particularly free threaded, so mock the time()
callable instead and ensure with subsecond clock intervals
we do the right math
Added support for the SQL ``CREATE TABLE ... AS SELECT`` construct via the
new :class:`_sql.CreateTableAs` DDL construct and the
:meth:`_sql.SelectBase.into` method. The new construct allows creating a
table directly from the results of a SELECT statement, with support for
options such as ``TEMPORARY`` and ``IF NOT EXISTS`` where supported by the
target database. Pull request courtesy Greg Jarzab.
Updated the :func:`_sql.over` clause to allow non-integer values in
:paramref:`_sql.over.range_` clause. Previously, only integer values
were allowed and any other values would lead to a failure.
To specify a non-integer value, use the new :class:`_sql.FrameClause`
construct along with the new :class:`_sql.FrameClauseType` enum to specify
the frame boundaries.
Mike Bayer [Sat, 18 Oct 2025 20:43:22 +0000 (16:43 -0400)]
improve sqlite reflection regex for unusual names/formats
A series of improvements have been made for reflection of CHECK constraints
on SQLite. The reflection logic now correctly handles table names
containing the strings "CHECK" or "CONSTRAINT", properly supports all four
SQLite identifier quoting styles (double quotes, single quotes, brackets,
and backticks) for constraint names, and accurately parses CHECK constraint
expressions containing parentheses within string literals using balanced
parenthesis matching with string context tracking. Big thanks to
GruzdevAV for new test cases and implementation ideas.
Mike Bayer [Tue, 5 Mar 2024 21:28:51 +0000 (16:28 -0500)]
proposal: remove _execute_compiled()
this isn't used internally and only allows execute(stmt.compile())
to work, which is not something I want to support.
Since people are definitely using this [1] we might want to think about
either deprectation or having a recipe for people who really want
to still do this
Mike Bayer [Tue, 21 Oct 2025 12:48:12 +0000 (08:48 -0400)]
ensure explicit left side considered for ON clause in join_from
Fixed an issue in :meth:`_sql.Select.join_from` where the join condition
between the left and right tables specified in the method call could be
incorrectly determined based on an intermediate table already present in
the FROM clause, rather than matching the foreign keys between the
immediate left and right arguments. The join condition is now determined by
matching primary keys between the two tables explicitly passed to
:meth:`_sql.Select.join_from`, ensuring consistent and predictable join
behavior regardless of the order of join operations or other tables present
in the query. The fix is applied to both the Core and ORM implementations
of :meth:`_sql.Select.join_from`.
krave1986 [Mon, 20 Oct 2025 20:06:31 +0000 (04:06 +0800)]
Fix missing back_populates in Note.item relationship in example code (#12925)
Without back_populates, the subsequent code would not automatically populate Item.notes or generate the key, which contradicts what the documentation intends to demonstrate.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
