Mike Bayer [Wed, 11 Dec 2024 20:54:23 +0000 (15:54 -0500)]
fix test due to merge of 11370 with 5252
Numeric and Float are split out in main so a type cant be both
at the same time. Also there's no reason to do isinstance(Float)
and isintance(Numeric) even if they are in the same hierarchy.
Frazer McLean [Thu, 3 Oct 2024 22:21:12 +0000 (18:21 -0400)]
dont match partial types in type_annotation_map
Fixed issue regarding ``Union`` types that would be present in the
:paramref:`_orm.registry.type_annotation_map` of a :class:`_orm.registry`
or declarative base class, where a ``Mapped[]`` element that included one
of the subtypes present in that ``Union`` would be matched to that entry,
potentially ignoring other entries that matched exactly. The correct
behavior now takes place such that an entry should only match in
``type_annotation_map`` exactly, as a ``Union`` type is a self-contained
type. For example, an attribute with ``Mapped[float]`` would previously
match to a ``type_annotation_map`` entry ``Union[float, Decimal]``; this
will no longer match and will now only match to an entry that states
``float``. Pull request courtesy Frazer McLean.
Nick Wilkinson [Fri, 6 Dec 2024 06:59:22 +0000 (01:59 -0500)]
Fixes: #11724 - PGDialect `get_multi_indexes` PGVecto.rs Bug
When attempting to generate an auto-revision using Alembic, the `get_multi_indexes` method fails with the error:
```python
dialect_options["postgresql_with"] = dict(
ValueError: dictionary update sequence element #0 has length 4; 2 is required
```
### Description
The cause of this error is that when creating a vector index in PGVecto.rs, the index is:
```sql
CREATE INDEX vector_embedding_idx ON public.vector_embeddings USING vectors (embedding vector_cos_ops) WITH (options='
[indexing.hnsw]
m = 16
ef_construction = 64
')
```
However, in PostgreSQL the index seems to be generated as:
```sql
CREATE INDEX vector_embedding_idx ON public.vector_embeddings USING hnsw (embedding vector_cos_ops) WITH (m='16', ef_construction='64');
```
To fix this, we need to modify:
```diff
if row["reloptions"]:
- dialect_options["postgresql_with"] = dict([option.split("=") for option in row["reloptions"]])
+ dialect_options["postgresql_with"] = dict([option.split("=", 1) for option in row["reloptions"]])
```
For more details on this error and a reproducible example, refer to #11724
### Testing
I couldn't really think of an easy way to add the potential test suggested in the issue thread [here](https://github.com/sqlalchemy/sqlalchemy/issues/11724#issuecomment-2518501318). However, this code is already tested in [`test_get_multi_view_indexes`](https://github.com/sqlalchemy/sqlalchemy/blob/5ded16fae8abfc31d43430cb25757fb434c37ba2/test/dialect/postgresql/test_reflection.py#L378), so assuming that test still passes and nothing breaks I believe we should be fine.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Mon, 2 Dec 2024 23:59:19 +0000 (18:59 -0500)]
use VARCHAR for CLOB outputtypehandler
Fixed issue in oracledb / cx_oracle dialects where output type handlers for
``CLOB`` were being routed to ``NVARCHAR`` rather than ``VARCHAR``, causing
a double conversion to take place.
Frazer McLean [Fri, 15 Nov 2024 18:07:00 +0000 (13:07 -0500)]
Add Range.__contains__
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Fixes #12093
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 12 Nov 2024 19:50:50 +0000 (14:50 -0500)]
dont leak mutating bindparams list into AnalyzedFunction
Fixed issue in "lambda SQL" feature where the tracking of bound parameters
could be corrupted if the same lambda were evaluated across multiple
compile phases, including when using the same lambda across multiple engine
instances or with statement caching disabled.
Federico Caselli [Sat, 26 Oct 2024 19:50:36 +0000 (21:50 +0200)]
Improve Oracle identifier length detection
Use the connection attribute ``max_identifier_length`` available
in oracledb since version 2.5 when determining the identifier length
in the Oracle dialect.
Miguel Grillo [Thu, 24 Oct 2024 18:32:33 +0000 (14:32 -0400)]
Fixed: 12012: Add Support for `TABLESPACE` Specification in Table Definitions for Oracle
Fixes: #12016
**Description**
This PR adds support for specifying the `TABLESPACE` in table definitions in SQLAlchemy, specifically for Oracle. This feature is particularly useful for Oracle users who need to specify the tablespace where the table data will be stored.
**Changes Made**
1. Updated `construct_arguments` in `OracleDialect`:
- The `construct_arguments` list in the `OracleDialect` class has been updated to include the `tablespace` argument for the `Table` class.
2. Modified OracleDDLCompiler to Include TABLESPACE in post_create_table:
- The OracleDDLCompiler class has been modified to include the TABLESPACE clause at the end of the CREATE TABLE statement if the tablespace option is provided.
3. Added tablespace Argument to the Table Class:
- A new tablespace argument has been added to the Table class to allow specifying the tablespace in the table definition.
4. Documentation Update:
- The documentation has been updated to reflect the new feature and provide usage examples.
**Usage Example**
```Python
from sqlalchemy import create_engine, Table, Column, Integer, String, MetaData
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Mon, 21 Oct 2024 14:03:01 +0000 (10:03 -0400)]
refine in_() check to use proper duck-typing for __clause_element__
Fixed regression caused by an internal code change in response to recent
Mypy releases that caused the very unusual case of a list of ORM-mapped
attribute expressions passed to :meth:`.ColumnOperators.in_` to no longer
be accepted.
in this commit we had to revisit d8dd28c42e where mypy typing
didn't accept ColumnOperartors. the type here is the _HasClauseElement[_T]
protocol which means we need to use a duck type for a runtime check.
Mike Bayer [Mon, 21 Oct 2024 17:10:54 +0000 (13:10 -0400)]
pin older pip for tox under py37
this is suddenly failing due to a non-py37 syntax in
pip/_vendor/typing_extensions.py. Not clear why the failure has just
started however as there is no new version of pip recently.
Mike Bayer [Tue, 15 Oct 2024 12:20:25 +0000 (08:20 -0400)]
add tests for pickling types inside an expression, some reduce methods
Fixed regression from 1.4 where some datatypes such as those derived from
:class:`.TypeDecorator` could not be pickled when they were part of a
larger SQL expression composition due to internal supporting structures
themselves not being pickleable.
Mike Bayer [Mon, 14 Oct 2024 15:15:21 +0000 (11:15 -0400)]
match ORM mapped cols to PK in interpret_returning_rows
Fixed bug in ORM "update with WHERE clause" feature where an explicit
``.returning()`` would interfere with the "fetch" synchronize strategy due
to an assumption that the ORM mapped class featured the primary key columns
in a specific position within the RETURNING. This has been fixed to use
appropriate ORM column targeting.
the _interpret_returning_rows method looked to be mostly not used as far
as its joined inheritance features, which appear to have never been
used as joined inheritance mappers are skipped.
Federico Caselli [Sun, 13 Oct 2024 16:32:46 +0000 (18:32 +0200)]
Render bind cast in json and jsonb in PG
Render bind cast for ``JSON`` and ``JSONB`` datatype on every dialect.
Previously this was only enabled in a subset of dialects. Fixes: #11994
Change-Id: Ib085deb3e84034dac9e4f4057d32f055d5533e52
(cherry picked from commit b2648e69f2375f7257cbe04b16f663d97795db19)
Mike Bayer [Sun, 13 Oct 2024 14:04:23 +0000 (10:04 -0400)]
consult allow_partial_pks for NULL check in lazyload
Refined the check which the ORM lazy loader uses to detect "this would be
loading by primary key and the primary key is NULL, skip loading" to take
into account the current setting for the
:paramref:`.orm.Mapper.allow_partial_pks` parameter. If this parameter is
False, then a composite PK value that has partial NULL elements should also
be skipped. This can apply to some composite overlapping foreign key
configurations.
Federico Caselli [Sat, 12 Oct 2024 12:58:26 +0000 (14:58 +0200)]
Optimize MySQL foreign key reflection
Improved foreign keys reflection logic in MySQL 8+ to use a better
optimized query. The previous query could be quite slow in databases
with a large number of columns.
Added a better error when trying to map as dataclass a class while also
manually providing the ``__table__`` attribute.
This usage is currently not supported.
Mike Bayer [Thu, 10 Oct 2024 02:05:05 +0000 (22:05 -0400)]
_Binary as generic to LargeBinary
Datatypes that are binary based such as :class:`.VARBINARY` will resolve to
:class:`.LargeBinary` when the :meth:`.TypeEngine.as_generic()` method is
called.
Mike Bayer [Tue, 8 Oct 2024 14:29:34 +0000 (10:29 -0400)]
re-apply right memo for nested ORMJoin when splicing
Fixed regression caused by fixes to joined eager loading in
:ticket:`11449`, where a particular joinedload case could not be asserted
correctly. We now have an example of that case so the assertion has been
repaired to allow for it.
Mike Bayer [Wed, 25 Sep 2024 18:19:02 +0000 (14:19 -0400)]
honor prefetch_cols and postfetch_cols in ORM update w/ WHERE criteria
Continuing from :ticket:`11912`, columns marked with
:paramref:`.mapped_column.onupdate`,
:paramref:`.mapped_column.server_onupdate`, or :class:`.Computed` are now
refreshed in ORM instances when running an ORM enabled UPDATE with WHERE
criteria, even if the statement does not use RETURNING or
populate_existing.
this moves the test we added in #11912 to be in
test_update_delete_where, since this behavior is not related to bulk
statements. For bulk statements, we're building onto the "many rows
fast" use case and we as yet intentionally don't do any "bookkeeping",
which means none of the expiration or any of that. would need to rethink
"bulk update" a bit to get onupdates to refresh.
Mike Bayer [Tue, 8 Oct 2024 18:01:40 +0000 (14:01 -0400)]
correct mis-cherry-picked commit for #11449 in 2.0.31
The main review for #11449 at Ie8f0e8d9bb7958baac33c7c2231e4afae15cf5b1
had three revisions but the cherry pick to 2.0 somehow missed the
second two cherry picks. it's not clear if this was intentional.
however, as we need to continue on with correcting this behavior
it seems like we really should have the full "main" behavior in
the release.
Mike Bayer [Sun, 22 Sep 2024 15:34:48 +0000 (11:34 -0400)]
propagate populate_existing for ORM bulk update
Similar to #9742
Fixed bug in ORM bulk update/delete where using RETURNING with bulk
update/delete in combination with populate existing would fail to
accommodate the populate_existing option.
MAINT: pytest doesn't have any more python 3.12 deprecations
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
I believe this workaround is not needed any more, pytest doesn't have any more python deprecations. (Arguably neither 3.13)
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] Tweek to the (testing) infrastructure
- New category as I didn't think this PR fit in any of the 3 options offered.
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Merge url query args to opts in mariadbconnector like mysqldb
Fixed issue in mariadbconnector dialect where query string arguments that
weren't checked integer or boolean arguments would be ignored, such as
string arguments like ``unix_socket``, etc. As part of this change, the
argument parsing for particular elements such as ``client_flags``,
``compress``, ``local_infile`` has been made more consistent across all
MySQL / MariaDB dialect which accept each argument. Pull request courtesy
Tobias Alex-Petersen.
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
I'm chasing some loose datetime.datetime.utcnow() deprecation warning in some test suites, and one of these was seemingly coming from sqlalchemy. It wasn't, but nevertheless these minor cleanup changes may still be found useful.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
Mike Bayer [Mon, 9 Sep 2024 13:21:20 +0000 (09:21 -0400)]
deprecate joinedload, subqueryload with DML; use correct statement
An ORM exception is raised if :func:`_orm.joinedload` or
:func:`_orm.subqueryload` are used as a top level option against a
statement that is not a SELECT statement, such as with an
``insert().returning()``. There are no JOINs in INSERT statements nor is
there a "subquery" that can be repurposed for subquery eager loading, and
for UPDATE/DELETE joinedload does not support these either, so it is never
appropriate for this use to pass silently.
Fixed issue where using eager loaders such as :func:`_orm.selectinload`
with additional criteria in combination with ORM DML such as
:func:`_sql.insert` with RETURNING would not correctly set up internal
contexts required for caching to work correctly, leading to incorrect
results.
Mike Bayer [Sat, 7 Sep 2024 21:41:16 +0000 (17:41 -0400)]
test for Concatenable in ORM evaluator for concat_op
Fixed issue in ORM evaluator where two datatypes being evaluated with the
SQL concatenator operator would not be checked for
:class:`.UnevaluatableError` based on their datatype; this missed the case
of :class:`_postgresql.JSONB` values being used in a concatenate operation
which is supported by PostgreSQL as well as how SQLAlchemy renders the SQL
for this operation, but does not work at the Python level. By implementing
:class:`.UnevaluatableError` for this combination, ORM update statements
will now fall back to "expire" when a concatenated JSON value used in a SET
clause is to be synchronized to a Python object.
Mike Bayer [Thu, 5 Sep 2024 17:21:17 +0000 (13:21 -0400)]
revert SQLite CHECK constraint changes and add new tests
The changes made for SQLite CHECK constraint reflection in versions 2.0.33
and 2.0.34 , :ticket:`11832` and :ticket:`11677`, have now been fully
reverted, as users continued to identify existing use cases that stopped
working after this change. For the moment, because SQLite does not
provide any consistent way of delivering information about CHECK
constraints, SQLAlchemy is limited in what CHECK constraint syntaxes can be
reflected, including that a CHECK constraint must be stated all on a
single, independent line (or inline on a column definition) without
newlines, tabs in the constraint definition or unusual characters in the
constraint name. Overall, reflection for SQLite is tailored towards being
able to reflect CREATE TABLE statements that were originally created by
SQLAlchemy DDL constructs. Long term work on a DDL parser that does not
rely upon regular expressions may eventually improve upon this situation.
A wide range of additional cross-dialect CHECK constraint reflection tests
have been added as it was also a bug that these changes did not trip any
existing tests.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
