Added ``dataclass_metadata`` argument to a all column functions
used in the ORM that accept dataclasses parameters.
It's passed to the underlying dataclass ``metadata`` attribute
of the dataclass field.
Pull request courtesy Sigmund Lahn.
Mike Bayer [Mon, 14 Jul 2025 20:13:03 +0000 (16:13 -0400)]
extend Values from HasCTE
The :func:`_sql.values` construct gains a new method :meth:`_sql.Values.cte`,
which allows creation of a named, explicit-columns :class:`.CTE` against an
unnamed ``VALUES`` expression, producing a syntax that allows column-oriented
selection from a ``VALUES`` construct on modern versions of PostgreSQL, SQLite,
and MariaDB.
Mike Bayer [Fri, 11 Jul 2025 16:51:44 +0000 (12:51 -0400)]
support JSONB subscripting syntax
Added support for PostgreSQL 14+ JSONB subscripting syntax. When connected
to PostgreSQL 14 or later, JSONB columns now automatically use the native
subscript notation ``jsonb_col['key']`` instead of the arrow operator
``jsonb_col -> 'key'`` for both read and write operations. This provides
better compatibility with PostgreSQL's native JSONB subscripting feature
while maintaining backward compatibility with older PostgreSQL versions.
JSON columns continue to use the traditional arrow syntax regardless of
PostgreSQL version.
Fix the code examples and the grammatic in `declarative_tables.rst` (#12721)
* Fix the code example in `declarative_tables.rst`
Add an import of a `Optional` class from the `typing` library in the
code example of the "Union types inside the Type Map" chapter.
* Fix the code example in `declarative_tables.rst`
Replace the import of the `deferred` function to `column_property` from
the `sqlalchemy.orm` package in first code example of the "Applying
Load, Persistence and Mapping Options for Imperative Table Columns"
chapter.
* Fix the grammatic in `declarative_tables.rst`
Remove the unnecessary article in the second paragraph of the
"Mapping to an Explicit Set of Primary Key Columns" chapter in
`declarative_tables.rst`.
remove support for list of tuples in the normal execute
The function that validates the arguments in the normal execute flow
allowed by mistake list of tuples, that are not supported by the
code since the 2.0 series.
Mike Bayer [Tue, 24 Jun 2025 17:57:17 +0000 (13:57 -0400)]
try flake8-import-order 0.19.2
the big new thang is that it is doing import order checks inside of
TYPE_CHECKING blocks. Introduces some new codes that we
enthusiastically add to our ignore list. update to the latest
and greatest zimports 0.6.2 as well
Mike Bayer [Mon, 23 Jun 2025 13:21:59 +0000 (09:21 -0400)]
hardening against inappropriate multi-table updates
Hardening of the compiler's actions for UPDATE statements that access
multiple tables to report more specifically when tables or aliases are
referenced in the SET clause; on cases where the backend does not support
secondary tables in the SET clause, an explicit error is raised, and on the
MySQL or similar backends that support such a SET clause, more specific
checking for not-properly-included tables is performed. Overall the change
is preventing these erroneous forms of UPDATE statements from being
compiled, whereas previously it was relied on the database to raise an
error, which was not always guaranteed to happen, or to be non-ambiguous,
due to cases where the parent table included the same column name as the
secondary table column being updated.
Fixed bug where the ORM would pull in the wrong column into an UPDATE when
a key name inside of the :meth:`.ValuesBase.values` method could be located
from an ORM entity mentioned in the statement, but where that ORM entity
was not the actual table that the statement was inserting or updating. An
extra check for this edge case is added to avoid this problem.
Mike Fiedler [Fri, 20 Jun 2025 21:49:26 +0000 (17:49 -0400)]
Update `table_per_related` example for Declarative API
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
When reading more of the Generic Associations, I found that the examples state "Annotated example", and wondered what that meant, since in other parts of the docs that means they are using the new 2.0 style.
I tried to update this example to be more in line with the new style, including a little f-string update.
I completely understand this is unlikely to be merged as-is - but wanted to understand more about the right way to use modern styles to properly code well-hinted, more "exotic" implementations.
Outstanding questions:
- Should examples pass pass `mypy --strict` ? It doesn't right now.
- Are there better ways to apply `Mapped` within the `type(...)` definition, so we could skip importing `Integer`?
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
(Do non-running examples count as code fixes/features?)
- [x] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
Mike Bayer [Fri, 20 Jun 2025 17:04:54 +0000 (13:04 -0400)]
really pin flake8-import-order to <0.19
In 8a287bf5c5635daf99217e I told @caselit to just omit 0.19.0,
as I wa sure 0.19.1 would fix the issue. Unfortunately in [1]
we see that this release continues to have problems, which seems
to be related not to the importlib.metadata fix but to an unrelated
improvement for TYPE_CHECKING blocks.
So we really have to wait for them to get a working release before
we can update.
Mike Bayer [Mon, 16 Jun 2025 23:53:30 +0000 (19:53 -0400)]
rework wraps_column_expression logic to be purely compile time checking
Fixed issue where :func:`.select` of a free-standing, unnamed scalar expression that
has a unary operator applied, such as negation, would not apply result
processors to the selected column even though the correct type remains in
place for the unary expression.
This change opened up a typing rabbithole where we were led to also
improve and harden the typing for the Exists element, in particular
in that the Exists now always refers to a ScalarSelect object, and
no longer a SelectStatementGrouping within the _regroup() cases; there
did not seem to be any reason for this inconsistency.
Mike Bayer [Wed, 11 Jun 2025 18:55:14 +0000 (14:55 -0400)]
update pickle tests
Since I want to get rid of util.portable_instancemethod, first
make sure we are testing pickle extensively including going through
all protocols for all metadata-oriented tests.
Pablo Estevez [Mon, 9 Jun 2025 12:49:13 +0000 (08:49 -0400)]
update tox mypy
<!-- Provide a general summary of your proposed changes in the Title field above -->
After this commit https://github.com/sqlalchemy/sqlalchemy/commit/68cd3e8ec7098d4bb4b2102ad247f84cd89dfd8c
tox will fail with mypy below 1.16, at least locally.
<!-- Describe your changes in detail -->
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Sun, 8 Jun 2025 17:01:45 +0000 (13:01 -0400)]
use sys.columns to allow accurate joining to other SYS tables
Reworked SQL Server column reflection to be based on the ``sys.columns``
table rather than ``information_schema.columns`` view. By correctly using
the SQL Server ``object_id()`` function as a lead and joining to related
tables on object_id rather than names, this repairs a variety of issues in
SQL Server reflection, including:
* Issue where reflected column comments would not correctly line up
with the columns themselves in the case that the table had been ALTERed
* Correctly targets tables with awkward names such as names with brackets,
when reflecting not just the basic table / columns but also extended
information including IDENTITY, computed columns, comments which
did not work previously
* Correctly targets IDENTITY, computed status from temporary tables
which did not work previously
Mike Bayer [Thu, 5 Jun 2025 12:58:49 +0000 (08:58 -0400)]
hardcode now(), current_timstamp() into the MySQL regex
Fixed yet another regression caused by by the DEFAULT rendering changes in
2.0.40 :ticket:`12425`, similar to :ticket:`12488`, this time where using a
CURRENT_TIMESTAMP function with a fractional seconds portion inside a
textual default value would also fail to be recognized as a
non-parenthesized server default.
There's no way to do this other than start hardcoding a list
of MySQL functions that demand that parenthesis are not added around
them, I can think of no other heuristic that will work here.
Suggestions welcome
Denis Laxalde [Wed, 28 May 2025 19:37:36 +0000 (15:37 -0400)]
Reflect index's column operator class on PostgreSQL
Fill the `postgresql_ops` key of PostgreSQL's `dialect_options` returned by get_multi_indexes() with a mapping from column names to the operator class, if it's not the default for respective data type.
As we need to join on ``pg_catalog.pg_opclass``, the table definition is added to ``postgresql.pg_catalog``.
Denis Laxalde [Tue, 20 May 2025 14:26:14 +0000 (10:26 -0400)]
Use pg_index's indnatts when indnkeyatts is not available
Using NULL when this column is not available does not work with old PostgreSQL (tested on version 9.6, as reported in #12600).
Instead, use `indnatts` which should be equal to what `indnkeyatts` would be as there is no "included attributes" in the index on these old versions (but only "key columns").
From https://www.postgresql.org/docs/17/catalog-pg-index.html:
* `indnatts`, "The total number of columns in the index [...]; this number includes both key and included attributes"
* `indnkeyatts`, "The number of key columns in the index, not counting any included columns [...]"
Mike Bayer [Sun, 18 May 2025 17:54:09 +0000 (13:54 -0400)]
backport mysql / base portions of #10415 to 2.0
For a backport of the mysql typing change in [1] to be useful, we need
to have most of the asyncio typing installed as well. To make this
easier include that we will backport aiomysql / asyncmy over to the
connectors/asyncio.py connector which is already in use.
Mike Bayer [Thu, 15 May 2025 17:39:36 +0000 (13:39 -0400)]
expand column options for composites up front at the attribute level
Implemented the :func:`_orm.defer`, :func:`_orm.undefer` and
:func:`_orm.load_only` loader options to work for composite attributes, a
use case that had never been supported previously.
Mike Bayer [Wed, 14 May 2025 12:24:44 +0000 (08:24 -0400)]
use pep639 license
Removed the "license classifier" from setup.cfg for SQLAlchemy 2.0, which
eliminates loud deprecation warnings when building the package. SQLAlchemy
2.1 will use a full :pep:`639` configuration in pyproject.toml while
SQLAlchemy 2.0 remains using ``setup.cfg`` for setup.
Mike Bayer [Tue, 13 May 2025 15:28:25 +0000 (11:28 -0400)]
remove __getattr__ from root
Removed ``__getattr__()`` rule from ``sqlalchemy/__init__.py`` that
appeared to be trying to correct for a previous typographical error in the
imports. This rule interferes with type checking and is removed.
Mike Bayer [Mon, 12 May 2025 19:25:07 +0000 (15:25 -0400)]
rewrite the docs on SQLite transaction handling
SQLite has added the new "connection.autocommit" mode and
associated fixes for pep-249 as of python 3.12. they plan to
default to using this attribute as of python 3.16. Get
on top of things by rewriting the whole doc section here, removing
old cruft about sqlalchemy isolation levels that was not correct
in any case, update recipes in a more succinct and unified way.
Mike Bayer [Thu, 6 Mar 2025 14:12:43 +0000 (09:12 -0500)]
implement pep-649 workarounds, test suite passing for python 3.14
Changes to the test suite to accommodate Python 3.14 as of version
3.14.0b1
Originally this included a major breaking change to how python 3.14
implemented :pep:`649`, however this was resolved by [1].
As of a7, greenlet is skipped due to issues in a7 and later b1
in [2].
1. the change to rewrite all conditionals in annotation related tests
is reverted.
2. test_memusage needed an explicit set_start_method() call so that
it can continue to use plain fork
3. unfortunately at the moment greenlet has to be re-disabled for 3.14.
4. Changes to tox overall, remove pysqlcipher which hasn't worked
in years, etc.
5. we need to support upcoming typing-extensions also, install the beta
6. 3.14.0a7 introduces major regressions to our runtime typing
utilities, unfortunately, it's not clear if these can be resolved
7. for 3.14.0b1, we have to vendor get_annotations to work around [3]
Justine Krejcha [Tue, 6 May 2025 19:18:02 +0000 (15:18 -0400)]
typing: pg: type NamedType create/drops (fixes #12557)
Type the `create` and `drop` functions for `NamedType`s
Also partially type the SchemaType create/drop functions more generally
One change to this is that the default parameter of `None` is removed. It doesn't work and will fail with a `AttributeError` at runtime since it immediately tries to access a property of `None` which doesn't exist.
Fixes #12557
This pull request is:
- [X] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 6 May 2025 22:06:15 +0000 (18:06 -0400)]
reorganize ORM Annotated Declarative into its own section
The ORM Annotated Declarative section is now very large but has been
indented under the "Declarative Table with mapped_column()" section
where it does not show up well on top level TOCs and is too deeply
nested. Break it out into its own section following the
entire "Declarative Table" section, but also maintain a short intro
section inside of "Declarative Table" to ensure this use is still
prominent.
suraj [Mon, 5 May 2025 15:14:35 +0000 (11:14 -0400)]
Added vector datatype support in Oracle dialect
Added new datatype :class:`_oracle.VECTOR` and accompanying DDL and DQL
support to fully support this type for Oracle Database. This change
includes the base :class:`_oracle.VECTOR` type that adds new type-specific
methods ``l2_distance``, ``cosine_distance``, ``inner_product`` as well as
new parameters ``oracle_vector`` for the :class:`.Index` construct,
allowing vector indexes to be configured, and ``oracle_fetch_approximate``
for the :meth:`.Select.fetch` clause. Pull request courtesy Suraj Shaw.
Mike Bayer [Thu, 1 May 2025 13:43:29 +0000 (09:43 -0400)]
fix sqlite localtimestamp function
Fixed and added test support for a few SQLite SQL functions hardcoded into
the compiler most notably the "localtimestamp" function which rendered with
incorrect internal quoting.
refactor (orm): remove unused variables and simplify key lookups
Redundant variables and unnecessary conditions were removed across several modules. Improved readability and reduced code complexity without changing functionality.
refactor(testing-and-utils): Remove unused code and fix style issues
This PR includes several small refactorings and style fixes aimed at improving code cleanliness, primarily within the test suite and tooling.
Key changes:
* Removed assignments to unused variables in various test files (`test_dialect.py`, `test_reflection.py`, `test_select.py`).
* Removed an unused variable in the pytest plugin (`pytestplugin.py`).
* Removed an unused variable in the topological sort utility (`topological.py`).
* Fixed a minor style issue (removed an extra blank line) in the `cython_imports.py` script.
refactor: clean up unused variables in engine module
Removed unused variables to improve code clarity and maintainability. This change simplifies logic in `base.py`, `default.py`, and `result.py`. No functionality was altered.
refactor (sql): simplify and optimize internal SQL handling
Replaced redundant variable assignments with direct operations. Used `dict.get()` for safer dictionary lookups to streamline logic. Improves code readability and reduces unnecessary lines.
refactor: simplify and clean up dialect-specific code
**Title:** Removed unused variables and redundant functions across multiple dialects. Improves code readability and reduces maintenance complexity without altering functionality.
### Description
This pull request introduces several minor refactorings across different dialect modules:
- **MSSQL:**
- Simplified the initialization of the `fkeys` dictionary in `_get_foreign_keys` using `util.defaultdict` directly.
- **MySQL:** Removed the unused variable in `_get_table_comment`. `rp`
- **PostgreSQL (_psycopg_common):** Removed the unused variable `cursor` in `do_ping`.
- **PostgreSQL (base):** Removed the unused variable `args` in `_get_column_info`.
- **SQLite:** Removed the unused variable `new_filename` in `generate_driver_url`.
These changes focus purely on code cleanup and simplification, removing dead code and improving clarity. They do not alter the existing logic or functionality of the dialects.
### Checklist
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- [x] A short code fix
- _Note: This is a general cleanup refactor rather than a fix for a specific reported issue._
Mike Bayer [Mon, 21 Apr 2025 13:44:40 +0000 (09:44 -0400)]
disable mysql/connector-python, again
Just as we got this driver "working", a new regression is introduced
in version 9.3.0 which prevents basic binary string persistence [1].
I would say we need to leave this driver off for another few years
until something changes with its upstream maintenance.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
