Mike Bayer [Mon, 30 Dec 2024 18:17:29 +0000 (13:17 -0500)]
further fixes for _cleanup_mapped_str_annotation
Fixed issues in type handling within the ``type_annotation_map`` feature
which prevented the use of unions, using either pep-604 or ``Union``
syntaxes under future annotations mode, which contained multiple generic
types as elements from being correctly resolvable.
also adds some further tests to assert that None added into the type
map for pep695, typing.NewType etc. sets up nullability on the column
Mike Bayer [Thu, 19 Dec 2024 22:50:21 +0000 (17:50 -0500)]
fix SQLite on conflict tests
in I4d66ec1473321616a1707da324a7dfe7a61ec94e we added new tests
in the sqlite suite but we forgot to extend from fixtures.TestBase,
so these tests did not get run at all. repair tests
Mike Bayer [Wed, 18 Dec 2024 16:24:58 +0000 (11:24 -0500)]
harden typing / coercion for on conflict/on duplicate key
in 2.1 we want these structures to be cacheable, so start
by cleaning up types and adding coercions to enforce those types.
these will be more locked down in 2.1 as we will need to move
bound parameter coercion outside of compilation, but here
do some small starts and introduce in 2.0.
in one interest of cachability, a "literal_binds" that found
its way into SQLite's compiler is replaced with "literal_execute",
the difference being that the latter is cacheable. This literal
is apparently necessary to suit SQLite's query planner for
the "index criteria" portion of the on conflict clause that otherwise
can't work with a real bound parameter.
Mike Bayer [Mon, 16 Dec 2024 15:58:01 +0000 (10:58 -0500)]
harden HSTORE registration
* use the driver_connection when we register on the connection
* assert targets passed to register_hstore assert as boolean true;
psycopg docs say "if None, register globally" but looking in the
source it's actually registering globally if any false-evaluating
object is passed.
add delete limit to mysql; ensure int for update/delete limits
Added support for the ``LIMIT`` clause with ``DELETE`` for the MySQL and
MariaDB dialects, to complement the already present option for
``UPDATE``. The :meth:`.delete.with_dialect_options` method of the
`:func:`.delete` construct accepts parameters for ``mysql_limit`` and
``mariadb_limit``, allowing users to specify a limit on the number of rows
deleted. Pull request courtesy of Pablo Nicolás Estevez.
Added logic to ensure that the ``mysql_limit`` and ``mariadb_limit``
parameters of :meth:`.update.with_dialect_options` and
:meth:`.delete.with_dialect_options` when compiled to string will only
compile if the parameter is passed as an integer; a ``ValueError`` is
raised otherwise.
corrected mysql documentation for update/delete options which
must be specified using the ``with_dialect_options()`` method.
Mike Bayer [Wed, 11 Dec 2024 20:54:23 +0000 (15:54 -0500)]
fix test due to merge of 11370 with 5252
Numeric and Float are split out in main so a type cant be both
at the same time. Also there's no reason to do isinstance(Float)
and isintance(Numeric) even if they are in the same hierarchy.
Frazer McLean [Thu, 3 Oct 2024 22:21:12 +0000 (18:21 -0400)]
dont match partial types in type_annotation_map
Fixed issue regarding ``Union`` types that would be present in the
:paramref:`_orm.registry.type_annotation_map` of a :class:`_orm.registry`
or declarative base class, where a ``Mapped[]`` element that included one
of the subtypes present in that ``Union`` would be matched to that entry,
potentially ignoring other entries that matched exactly. The correct
behavior now takes place such that an entry should only match in
``type_annotation_map`` exactly, as a ``Union`` type is a self-contained
type. For example, an attribute with ``Mapped[float]`` would previously
match to a ``type_annotation_map`` entry ``Union[float, Decimal]``; this
will no longer match and will now only match to an entry that states
``float``. Pull request courtesy Frazer McLean.
Mike Bayer [Mon, 13 Apr 2020 14:25:27 +0000 (10:25 -0400)]
Separate Numeric and Float
the :class:`.Numeric` and :class:`.Float` SQL types have been separated out
so that :class:`.Float` no longer inherits from :class:`.Numeric`; instead,
they both extend from a common mixin :class:`.NumericCommon`. This
corrects for some architectural shortcomings where numeric and float types
are typically separate, and establishes more consistency with
:class:`.Integer` also being a distinct type. The change should not have
any end-user implications except for code that may be using
``isinstance()`` to test for the :class:`.Numeric` datatype; third party
dialects which rely upon specific implementation types for numeric and/or
float may also require adjustment to maintain compatibility.
Nick Wilkinson [Fri, 6 Dec 2024 06:59:22 +0000 (01:59 -0500)]
Fixes: #11724 - PGDialect `get_multi_indexes` PGVecto.rs Bug
When attempting to generate an auto-revision using Alembic, the `get_multi_indexes` method fails with the error:
```python
dialect_options["postgresql_with"] = dict(
ValueError: dictionary update sequence element #0 has length 4; 2 is required
```
### Description
The cause of this error is that when creating a vector index in PGVecto.rs, the index is:
```sql
CREATE INDEX vector_embedding_idx ON public.vector_embeddings USING vectors (embedding vector_cos_ops) WITH (options='
[indexing.hnsw]
m = 16
ef_construction = 64
')
```
However, in PostgreSQL the index seems to be generated as:
```sql
CREATE INDEX vector_embedding_idx ON public.vector_embeddings USING hnsw (embedding vector_cos_ops) WITH (m='16', ef_construction='64');
```
To fix this, we need to modify:
```diff
if row["reloptions"]:
- dialect_options["postgresql_with"] = dict([option.split("=") for option in row["reloptions"]])
+ dialect_options["postgresql_with"] = dict([option.split("=", 1) for option in row["reloptions"]])
```
For more details on this error and a reproducible example, refer to #11724
### Testing
I couldn't really think of an easy way to add the potential test suggested in the issue thread [here](https://github.com/sqlalchemy/sqlalchemy/issues/11724#issuecomment-2518501318). However, this code is already tested in [`test_get_multi_view_indexes`](https://github.com/sqlalchemy/sqlalchemy/blob/5ded16fae8abfc31d43430cb25757fb434c37ba2/test/dialect/postgresql/test_reflection.py#L378), so assuming that test still passes and nothing breaks I believe we should be fine.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Mon, 2 Dec 2024 23:59:19 +0000 (18:59 -0500)]
use VARCHAR for CLOB outputtypehandler
Fixed issue in oracledb / cx_oracle dialects where output type handlers for
``CLOB`` were being routed to ``NVARCHAR`` rather than ``VARCHAR``, causing
a double conversion to take place.
Mike Bayer [Mon, 18 Nov 2024 18:43:08 +0000 (13:43 -0500)]
apply underscores to ORM class and def names
criteria used here is:
* The class or def should definitely not be used directly by
a third party
* The class would never be the subject of an `isinstance()` check
* The class is not exposed as the type of a return value for a public
function
A sweep through class and function names in the ORM renames many classes
and functions that have no intent of public visibility to be underscored.
This is to reduce ambiguity as to which APIs are intended to be targeted by
third party applications and extensions. Third parties are encouraged to
propose new public APIs in Discussions to the extent they are needed to
replace those that have been clarified as private.
Mike Bayer [Sat, 16 Nov 2024 20:41:04 +0000 (15:41 -0500)]
remove _implicit_subquery and all derived functions
The ``.c`` and ``.columns`` attributes on the :class:`.Select` and
:class:`.TextualSelect` constructs, which are not instances of
:class:`.FromClause`, have been removed completely, in addition to the
``.select()`` method as well as other codepaths which would implicitly
generate a subquery from a :class:`.Select` without the need to explicitly
call the :meth:`.Select.subquery` method.
In the case of ``.c`` and ``.columns``, these attributes were never useful
in practice and have caused a great deal of confusion, hence were
deprecated back in version 1.4, and have emitted warnings since that
version. Accessing the columns that are specific to a :class:`.Select`
construct is done via the :attr:`.Select.selected_columns` attribute, which
was added in version 1.4 to suit the use case that users often expected
``.c`` to accomplish. In the larger sense, implicit production of
subqueries works against SQLAlchemy's modern practice of making SQL
structure as explicit as possible.
Note that this is **not related** to the usual :attr:`.FromClause.c` and
:attr:`.FromClause.columns` attributes, common to objects such as
:class:`.Table` and :class:`.Subquery`, which are unaffected by this
change.
Frazer McLean [Fri, 15 Nov 2024 18:07:00 +0000 (13:07 -0500)]
Add Range.__contains__
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Fixes #12093
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Wed, 13 Nov 2024 15:46:17 +0000 (10:46 -0500)]
apply quote to url.database portion
Adjusted URL parsing and stringification to apply url quoting to the
"database" portion of the URL. This allows a URL where the "database"
portion includes special characters such as question marks to be
accommodated.
Mike Bayer [Tue, 12 Nov 2024 19:50:50 +0000 (14:50 -0500)]
dont leak mutating bindparams list into AnalyzedFunction
Fixed issue in "lambda SQL" feature where the tracking of bound parameters
could be corrupted if the same lambda were evaluated across multiple
compile phases, including when using the same lambda across multiple engine
instances or with statement caching disabled.
Federico Caselli [Sat, 26 Oct 2024 19:50:36 +0000 (21:50 +0200)]
Improve Oracle identifier length detection
Use the connection attribute ``max_identifier_length`` available
in oracledb since version 2.5 when determining the identifier length
in the Oracle dialect.
Miguel Grillo [Thu, 24 Oct 2024 18:32:33 +0000 (14:32 -0400)]
Fixed: 12012: Add Support for `TABLESPACE` Specification in Table Definitions for Oracle
Fixes: #12016
**Description**
This PR adds support for specifying the `TABLESPACE` in table definitions in SQLAlchemy, specifically for Oracle. This feature is particularly useful for Oracle users who need to specify the tablespace where the table data will be stored.
**Changes Made**
1. Updated `construct_arguments` in `OracleDialect`:
- The `construct_arguments` list in the `OracleDialect` class has been updated to include the `tablespace` argument for the `Table` class.
2. Modified OracleDDLCompiler to Include TABLESPACE in post_create_table:
- The OracleDDLCompiler class has been modified to include the TABLESPACE clause at the end of the CREATE TABLE statement if the tablespace option is provided.
3. Added tablespace Argument to the Table Class:
- A new tablespace argument has been added to the Table class to allow specifying the tablespace in the table definition.
4. Documentation Update:
- The documentation has been updated to reflect the new feature and provide usage examples.
**Usage Example**
```Python
from sqlalchemy import create_engine, Table, Column, Integer, String, MetaData
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
The :class:`.Float` and :class:`.Numeric` types are no longer automatically
considered as auto-incrementing columns when the
:paramref:`_schema.Column.autoincrement` parameter is left at its default
of ``"auto"`` on a :class:`_schema.Column` that is part of the primary key.
When the parameter is set to ``True``, a :class:`.Numeric` type will be
accepted as an auto-incrementing datatype for primary key columns, but only
if its scale is explicitly given as zero; otherwise, an error is raised.
This is a change from 2.0 where all numeric types including floats were
automatically considered as "autoincrement" for primary key columns.
Mike Bayer [Tue, 22 Oct 2024 18:33:23 +0000 (14:33 -0400)]
lookup "secondary" directly, dont use eval()
The :paramref:`_orm.relationship.secondary` parameter no longer uses Python
``eval()`` to evaluate the given string. This parameter when passed a
string should resolve to a table name that's present in the local
:class:`.MetaData` collection only, and never needs to be any kind of
Python expression otherwise. To use a real deferred callable based on a
name that may not be locally present yet, use a lambda instead.
Mike Bayer [Tue, 22 Oct 2024 18:03:08 +0000 (14:03 -0400)]
remove first_init
The ``first_init`` ORM event has been removed. This event was
non-functional throughout the 1.4 and 2.0 series and could not be invoked
without raising an internal error, so it is not expected that there is any
real-world use of this event hook.
Mike Bayer [Mon, 21 Oct 2024 14:03:01 +0000 (10:03 -0400)]
refine in_() check to use proper duck-typing for __clause_element__
Fixed regression caused by an internal code change in response to recent
Mypy releases that caused the very unusual case of a list of ORM-mapped
attribute expressions passed to :meth:`.ColumnOperators.in_` to no longer
be accepted.
in this commit we had to revisit d8dd28c42e where mypy typing
didn't accept ColumnOperartors. the type here is the _HasClauseElement[_T]
protocol which means we need to use a duck type for a runtime check.
Mike Bayer [Tue, 15 Oct 2024 12:20:25 +0000 (08:20 -0400)]
add tests for pickling types inside an expression, some reduce methods
Fixed regression from 1.4 where some datatypes such as those derived from
:class:`.TypeDecorator` could not be pickled when they were part of a
larger SQL expression composition due to internal supporting structures
themselves not being pickleable.
Mike Bayer [Mon, 14 Oct 2024 15:15:21 +0000 (11:15 -0400)]
match ORM mapped cols to PK in interpret_returning_rows
Fixed bug in ORM "update with WHERE clause" feature where an explicit
``.returning()`` would interfere with the "fetch" synchronize strategy due
to an assumption that the ORM mapped class featured the primary key columns
in a specific position within the RETURNING. This has been fixed to use
appropriate ORM column targeting.
the _interpret_returning_rows method looked to be mostly not used as far
as its joined inheritance features, which appear to have never been
used as joined inheritance mappers are skipped.
Federico Caselli [Sun, 13 Oct 2024 16:32:46 +0000 (18:32 +0200)]
Render bind cast in json and jsonb in PG
Render bind cast for ``JSON`` and ``JSONB`` datatype on every dialect.
Previously this was only enabled in a subset of dialects. Fixes: #11994
Change-Id: Ib085deb3e84034dac9e4f4057d32f055d5533e52
Mike Bayer [Sun, 13 Oct 2024 14:04:23 +0000 (10:04 -0400)]
consult allow_partial_pks for NULL check in lazyload
Refined the check which the ORM lazy loader uses to detect "this would be
loading by primary key and the primary key is NULL, skip loading" to take
into account the current setting for the
:paramref:`.orm.Mapper.allow_partial_pks` parameter. If this parameter is
False, then a composite PK value that has partial NULL elements should also
be skipped. This can apply to some composite overlapping foreign key
configurations.
Federico Caselli [Sat, 12 Oct 2024 12:58:26 +0000 (14:58 +0200)]
Optimize MySQL foreign key reflection
Improved foreign keys reflection logic in MySQL 8+ to use a better
optimized query. The previous query could be quite slow in databases
with a large number of columns.
Added a better error when trying to map as dataclass a class while also
manually providing the ``__table__`` attribute.
This usage is currently not supported.
Mike Bayer [Thu, 10 Oct 2024 02:05:05 +0000 (22:05 -0400)]
_Binary as generic to LargeBinary
Datatypes that are binary based such as :class:`.VARBINARY` will resolve to
:class:`.LargeBinary` when the :meth:`.TypeEngine.as_generic()` method is
called.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
