Denis Laxalde [Wed, 28 May 2025 19:37:36 +0000 (15:37 -0400)]
Reflect index's column operator class on PostgreSQL
Fill the `postgresql_ops` key of PostgreSQL's `dialect_options` returned by get_multi_indexes() with a mapping from column names to the operator class, if it's not the default for respective data type.
As we need to join on ``pg_catalog.pg_opclass``, the table definition is added to ``postgresql.pg_catalog``.
Denis Laxalde [Tue, 20 May 2025 14:26:14 +0000 (10:26 -0400)]
Use pg_index's indnatts when indnkeyatts is not available
Using NULL when this column is not available does not work with old PostgreSQL (tested on version 9.6, as reported in #12600).
Instead, use `indnatts` which should be equal to what `indnkeyatts` would be as there is no "included attributes" in the index on these old versions (but only "key columns").
From https://www.postgresql.org/docs/17/catalog-pg-index.html:
* `indnatts`, "The total number of columns in the index [...]; this number includes both key and included attributes"
* `indnkeyatts`, "The number of key columns in the index, not counting any included columns [...]"
Mike Bayer [Sun, 18 May 2025 17:54:09 +0000 (13:54 -0400)]
backport mysql / base portions of #10415 to 2.0
For a backport of the mysql typing change in [1] to be useful, we need
to have most of the asyncio typing installed as well. To make this
easier include that we will backport aiomysql / asyncmy over to the
connectors/asyncio.py connector which is already in use.
Mike Bayer [Thu, 15 May 2025 17:39:36 +0000 (13:39 -0400)]
expand column options for composites up front at the attribute level
Implemented the :func:`_orm.defer`, :func:`_orm.undefer` and
:func:`_orm.load_only` loader options to work for composite attributes, a
use case that had never been supported previously.
Mike Bayer [Wed, 14 May 2025 12:24:44 +0000 (08:24 -0400)]
use pep639 license
Removed the "license classifier" from setup.cfg for SQLAlchemy 2.0, which
eliminates loud deprecation warnings when building the package. SQLAlchemy
2.1 will use a full :pep:`639` configuration in pyproject.toml while
SQLAlchemy 2.0 remains using ``setup.cfg`` for setup.
Mike Bayer [Tue, 13 May 2025 15:28:25 +0000 (11:28 -0400)]
remove __getattr__ from root
Removed ``__getattr__()`` rule from ``sqlalchemy/__init__.py`` that
appeared to be trying to correct for a previous typographical error in the
imports. This rule interferes with type checking and is removed.
Mike Bayer [Mon, 12 May 2025 19:25:07 +0000 (15:25 -0400)]
rewrite the docs on SQLite transaction handling
SQLite has added the new "connection.autocommit" mode and
associated fixes for pep-249 as of python 3.12. they plan to
default to using this attribute as of python 3.16. Get
on top of things by rewriting the whole doc section here, removing
old cruft about sqlalchemy isolation levels that was not correct
in any case, update recipes in a more succinct and unified way.
Mike Bayer [Thu, 6 Mar 2025 14:12:43 +0000 (09:12 -0500)]
implement pep-649 workarounds, test suite passing for python 3.14
Changes to the test suite to accommodate Python 3.14 as of version
3.14.0b1
Originally this included a major breaking change to how python 3.14
implemented :pep:`649`, however this was resolved by [1].
As of a7, greenlet is skipped due to issues in a7 and later b1
in [2].
1. the change to rewrite all conditionals in annotation related tests
is reverted.
2. test_memusage needed an explicit set_start_method() call so that
it can continue to use plain fork
3. unfortunately at the moment greenlet has to be re-disabled for 3.14.
4. Changes to tox overall, remove pysqlcipher which hasn't worked
in years, etc.
5. we need to support upcoming typing-extensions also, install the beta
6. 3.14.0a7 introduces major regressions to our runtime typing
utilities, unfortunately, it's not clear if these can be resolved
7. for 3.14.0b1, we have to vendor get_annotations to work around [3]
Justine Krejcha [Tue, 6 May 2025 19:18:02 +0000 (15:18 -0400)]
typing: pg: type NamedType create/drops (fixes #12557)
Type the `create` and `drop` functions for `NamedType`s
Also partially type the SchemaType create/drop functions more generally
One change to this is that the default parameter of `None` is removed. It doesn't work and will fail with a `AttributeError` at runtime since it immediately tries to access a property of `None` which doesn't exist.
Fixes #12557
This pull request is:
- [X] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 6 May 2025 22:06:15 +0000 (18:06 -0400)]
reorganize ORM Annotated Declarative into its own section
The ORM Annotated Declarative section is now very large but has been
indented under the "Declarative Table with mapped_column()" section
where it does not show up well on top level TOCs and is too deeply
nested. Break it out into its own section following the
entire "Declarative Table" section, but also maintain a short intro
section inside of "Declarative Table" to ensure this use is still
prominent.
suraj [Mon, 5 May 2025 15:14:35 +0000 (11:14 -0400)]
Added vector datatype support in Oracle dialect
Added new datatype :class:`_oracle.VECTOR` and accompanying DDL and DQL
support to fully support this type for Oracle Database. This change
includes the base :class:`_oracle.VECTOR` type that adds new type-specific
methods ``l2_distance``, ``cosine_distance``, ``inner_product`` as well as
new parameters ``oracle_vector`` for the :class:`.Index` construct,
allowing vector indexes to be configured, and ``oracle_fetch_approximate``
for the :meth:`.Select.fetch` clause. Pull request courtesy Suraj Shaw.
Mike Bayer [Thu, 1 May 2025 13:43:29 +0000 (09:43 -0400)]
fix sqlite localtimestamp function
Fixed and added test support for a few SQLite SQL functions hardcoded into
the compiler most notably the "localtimestamp" function which rendered with
incorrect internal quoting.
refactor (orm): remove unused variables and simplify key lookups
Redundant variables and unnecessary conditions were removed across several modules. Improved readability and reduced code complexity without changing functionality.
refactor(testing-and-utils): Remove unused code and fix style issues
This PR includes several small refactorings and style fixes aimed at improving code cleanliness, primarily within the test suite and tooling.
Key changes:
* Removed assignments to unused variables in various test files (`test_dialect.py`, `test_reflection.py`, `test_select.py`).
* Removed an unused variable in the pytest plugin (`pytestplugin.py`).
* Removed an unused variable in the topological sort utility (`topological.py`).
* Fixed a minor style issue (removed an extra blank line) in the `cython_imports.py` script.
refactor: clean up unused variables in engine module
Removed unused variables to improve code clarity and maintainability. This change simplifies logic in `base.py`, `default.py`, and `result.py`. No functionality was altered.
refactor (sql): simplify and optimize internal SQL handling
Replaced redundant variable assignments with direct operations. Used `dict.get()` for safer dictionary lookups to streamline logic. Improves code readability and reduces unnecessary lines.
refactor: simplify and clean up dialect-specific code
**Title:** Removed unused variables and redundant functions across multiple dialects. Improves code readability and reduces maintenance complexity without altering functionality.
### Description
This pull request introduces several minor refactorings across different dialect modules:
- **MSSQL:**
- Simplified the initialization of the `fkeys` dictionary in `_get_foreign_keys` using `util.defaultdict` directly.
- **MySQL:** Removed the unused variable in `_get_table_comment`. `rp`
- **PostgreSQL (_psycopg_common):** Removed the unused variable `cursor` in `do_ping`.
- **PostgreSQL (base):** Removed the unused variable `args` in `_get_column_info`.
- **SQLite:** Removed the unused variable `new_filename` in `generate_driver_url`.
These changes focus purely on code cleanup and simplification, removing dead code and improving clarity. They do not alter the existing logic or functionality of the dialects.
### Checklist
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- [x] A short code fix
- _Note: This is a general cleanup refactor rather than a fix for a specific reported issue._
Mike Bayer [Mon, 21 Apr 2025 13:44:40 +0000 (09:44 -0400)]
disable mysql/connector-python, again
Just as we got this driver "working", a new regression is introduced
in version 9.3.0 which prevents basic binary string persistence [1].
I would say we need to leave this driver off for another few years
until something changes with its upstream maintenance.
$ .venv/bin/python sample.py
add(*(1,), **{'y': 3.14}) => 4.140000000000001
{'x': int | float, 'y': int | float, 'return': int | float}
35.93937499681488 us
mem=9252896, peak=9300808
$ git switch -
Switched to branch 'opt-decorator'
$ .venv/bin/python sample.py
add(*(1,), **{'y': 3.14}) => 4.140000000000001
{'x': int | float, 'y': int | float, 'return': int | float}
23.32574996398762 us
mem=1439032, peak=1476423
```
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Federico Caselli [Mon, 24 Mar 2025 20:50:45 +0000 (21:50 +0100)]
improve overloads applied to generic functions
try again to remove the overloads to the generic functionn
generator (like coalesce, array_agg, etc).
As of mypy 1.15 it still does now work, but a simpler version
is added in this change
Alexander Ruehe [Tue, 1 Apr 2025 21:52:12 +0000 (17:52 -0400)]
ensure ON UPDATE test is case insensitive
Fixed regression caused by the DEFAULT rendering changes in 2.0.40
:ticket:`12425` where using lowercase `on update` in a MySQL server default
would incorrectly apply parenthesis, leading to errors when MySQL
interpreted the rendered DDL. Pull request courtesy Alexander Ruehe.
Fixed issue where :meth:`.AsyncSession.get_transaction` and
:meth:`.AsyncSession.get_nested_transaction` would fail with
``NotImplementedError`` if the "proxy transaction" used by
:class:`.AsyncSession` were garbage collected and needed regeneration.
Kaan [Wed, 19 Mar 2025 15:58:30 +0000 (11:58 -0400)]
Implement GROUPS frame spec for window functions
Implemented support for the GROUPS frame specification in window functions
by adding :paramref:`_sql.over.groups` option to :func:`_sql.over`
and :meth:`.FunctionElement.over`. Pull request courtesy Kaan Dikmen.
Daraan [Wed, 26 Mar 2025 18:27:46 +0000 (14:27 -0400)]
compatibility with typing_extensions 4.13 and type statement
Fixed regression caused by ``typing_extension==4.13.0`` that introduced
a different implementation for ``TypeAliasType`` while SQLAlchemy assumed
that it would be equivalent to the ``typing`` version.
Denis Laxalde [Mon, 24 Mar 2025 20:35:07 +0000 (16:35 -0400)]
Type array_agg()
The return type of `array_agg()` is declared as a `Sequence[T]` where `T` is bound to the type of input argument.
This is implemented by making `array_agg()` inheriting from `ReturnTypeFromArgs` which provides appropriate overloads of `__init__()` to support this.
This usage of ReturnTypeFromArgs is a bit different from previous ones as the return type of the function is not exactly the same as that of its arguments, but a "collection" (a generic, namely a Sequence here) of the argument types. Accordingly, we adjust the code of `tools/generate_sql_functions.py` to retrieve the "collection" type from 'fn_class' annotation and generate expected return type.
Also add a couple of hand-written typing tests for PostgreSQL.
Denis Laxalde [Wed, 19 Mar 2025 08:17:27 +0000 (04:17 -0400)]
Cast empty PostgreSQL ARRAY from the type specified to array()
When building a PostgreSQL ``ARRAY`` literal using
:class:`_postgresql.array` with an empty ``clauses`` argument, the
:paramref:`_postgresql.array.type_` parameter is now significant in that it
will be used to render the resulting ``ARRAY[]`` SQL expression with a
cast, such as ``ARRAY[]::INTEGER``. Pull request courtesy Denis Laxalde.
Mike Bayer [Wed, 19 Mar 2025 22:30:21 +0000 (18:30 -0400)]
skip FROM disambiguation for immediate alias of table
Fixed regression caused by :ticket:`7471` leading to a SQL compilation
issue where name disambiguation for two same-named FROM clauses with table
aliasing in use at the same time would produce invalid SQL in the FROM
clause with two "AS" clauses for the aliased table, due to double aliasing.
Mike Bayer [Wed, 19 Mar 2025 12:59:54 +0000 (08:59 -0400)]
remove attrs w/ orm annotated declarative example
as pointed out at
https://github.com/sqlalchemy/sqlalchemy/discussions/12449, ORM
annotated declarative is not compatible with attrs, declarative
cannot be used with attrs.
Denis Laxalde [Tue, 18 Mar 2025 16:23:01 +0000 (12:23 -0400)]
Make ARRAY generic on the item_type
Now `Column(type_=ARRAY(Integer)` is inferred as `Column[Sequence[int]]` instead as `Column[Sequence[Any]]` previously. This only works with the `type_` argument to Column, but that's not new.
This follows from a suggestion at
https://github.com/sqlalchemy/sqlalchemy/pull/12386#issuecomment-2694056069.
Mike Bayer [Mon, 17 Mar 2025 20:46:12 +0000 (16:46 -0400)]
ensure SQL expressions w/o bool pass through to correct typing error
Fixed regression which occurred as of 2.0.37 where the checked
:class:`.ArgumentError` that's raised when an inappropriate type or object
is used inside of a :class:`.Mapped` annotation would raise ``TypeError``
with "boolean value of this clause is not defined" if the object resolved
into a SQL expression in a boolean context, for programs where future
annotations mode was not enabled. This case is now handled explicitly and
a new error message has also been tailored for this case. In addition, as
there are at least half a dozen distinct error scenarios for intepretation
of the :class:`.Mapped` construct, these scenarios have all been unified
under a new subclass of :class:`.ArgumentError` called
:class:`.MappedAnnotationError`, to provide some continuity between these
different scenarios, even though specific messaging remains distinct.
Mike Bayer [Wed, 12 Mar 2025 20:25:48 +0000 (16:25 -0400)]
expand paren rules for default rendering, sqlite/mysql
Expanded the rules for when to apply parenthesis to a server default in DDL
to suit the general case of a default string that contains non-word
characters such as spaces or operators and is not a string literal.
Fixed issue in MySQL server default reflection where a default that has
spaces would not be correctly reflected. Additionally, expanded the rules
for when to apply parenthesis to a server default in DDL to suit the
general case of a default string that contains non-word characters such as
spaces or operators and is not a string literal.
Denis Laxalde [Thu, 13 Mar 2025 12:43:53 +0000 (08:43 -0400)]
Support column list for foreign key ON DELETE SET actions on PostgreSQL
Added support for specifying a list of columns for ``SET NULL`` and ``SET
DEFAULT`` actions of ``ON DELETE`` clause of foreign key definition on
PostgreSQL. Pull request courtesy Denis Laxalde.
Denis Laxalde [Fri, 14 Mar 2025 21:01:50 +0000 (17:01 -0400)]
Add type annotations to `postgresql.array`
Improved static typing for `postgresql.array()` by making the type parameter (the type of array's elements) inferred from the `clauses` and `type_` arguments while also ensuring they are consistent.
Also completed type annotations of `postgresql.ARRAY` following commit 0bf7e02afbec557eb3a5607db407f27deb7aac77 and added type annotations for functions `postgresql.Any()` and `postgresql.All()`.
Finally, fixed shadowing `typing.Any` by the `Any()` function through aliasing as `typing_Any`.
Mike Bayer [Fri, 14 Mar 2025 14:33:22 +0000 (10:33 -0400)]
anonymize CRUD params if visiting_cte is present
Fixed issue in :class:`.CTE` constructs involving multiple DDL
:class:`.Insert` statements with multiple VALUES parameter sets where the
bound parameter names generated for these parameter sets would conflict,
generating a compile time error.
Denis Laxalde [Wed, 5 Mar 2025 20:59:39 +0000 (15:59 -0500)]
Complement type annotations for ARRAY
This complements the type annotations of the `ARRAY` class, in preparation of #12384.
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Related to https://github.com/sqlalchemy/sqlalchemy/issues/6810
Mike Bayer [Sun, 23 Feb 2025 16:20:18 +0000 (11:20 -0500)]
re-support mysql-connector python
Support has been re-added for the MySQL-Connector/Python DBAPI using the
``mysql+mysqlconnector://`` URL scheme. The DBAPI now works against
modern MySQL versions as well as MariaDB versions (in the latter case it's
required to pass charset/collation explicitly). Note however that
server side cursor support is disabled due to unresolved issues with this
driver.
note the 2.0 backport here necessarily needs to also backport some
of 49ce245998 to handle the mariadb database working under mysql
connector.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
