Mike Bayer [Tue, 27 Jun 2023 14:17:36 +0000 (10:17 -0400)]
accomodate schema translate keys present or not present
Adjusted the :paramref:`_sa.create_engine.schema_translate_map` feature
such that **all** schema names in the statement are now tokenized,
regardless of whether or not a specific name is in the immediate schema
translate map given, and to fallback to substituting the original name when
the key is not in the actual schema translate map at execution time. These
two changes allow for repeated use of a compiled object with schema
schema_translate_maps that include or dont include various keys on each
run, allowing cached SQL constructs to continue to function at runtime when
schema translate maps with different sets of keys are used each time. In
addition, added detection of schema_translate_map dictionaries which gain
or lose a ``None`` key across calls for the same statement, which affects
compilation of the statement and is not compatible with caching; an
exception is raised for these scenarios.
Mike Bayer [Sat, 24 Jun 2023 01:57:41 +0000 (21:57 -0400)]
add option to create scalar object on none attribute set
Added new option to :func:`.association_proxy`
:paramref:`.association_proxy.create_on_none_assignment`; when an
association proxy which refers to a scalar relationship is assigned the
value ``None``, and the referenced object is not present, a new object is
created via the creator. This was apparently an undefined behavior in the
1.2 series that was silently removed.
Mike Bayer [Thu, 22 Jun 2023 22:11:07 +0000 (18:11 -0400)]
expand "is_unnatural" to include all mapper inheritance paths
Fixed issue in ORM loader strategy logic which further allows for long
chains of :func:`_orm.contains_eager` loader options across complex
inheriting polymorphic / aliased / of_type() relationship chains to take
proper effect in queries.
Mike Bayer [Wed, 21 Jun 2023 18:59:21 +0000 (14:59 -0400)]
qualify mypy1.4 update for python 3.9, 3.10 +
in I68084199858e9da901641d6036780437bcf5f2d6 we added a mypy1.4
check to check for new-style type messages. mypy only
does lowercase types on python 3.9 and above, OR syntax on 3.10
and above. qualify these both
Mike Bayer [Mon, 19 Jun 2023 15:54:35 +0000 (11:54 -0400)]
track state change within _connection_for_bind()
Additional hardening and documentation for the ORM :class:`_orm.Session`
"state change" system, which detects concurrent use of
:class:`_orm.Session` and :class:`_asyncio.AsyncSession` objects; an
additional check is added within the process to acquire connections from
the underlying engine, which is a critical section with regards to internal
connection management.
Tony Locke [Mon, 19 Jun 2023 20:32:18 +0000 (16:32 -0400)]
pg8000: Support range and multirange types
The pg8000 dialect now supports RANGE and MULTIRANGE datatypes, using the
existing RANGE API described at :ref:`postgresql_ranges`. Range and
multirange types are supported in the pg8000 driver from version 1.29.8.
Pull request courtesy Tony Locke.
Mike Bayer [Thu, 15 Jun 2023 22:34:57 +0000 (18:34 -0400)]
default Enum name to None and don't remove given name
Fixed issue in support for the :class:`.Enum` datatype in the
:paramref:`_orm.registry.type_annotation_map` first added as part of
:ticket:`8859` where using a custom :class:`.Enum` with fixed configuration
in the map would fail to transfer the :paramref:`.Enum.name` parameter,
which among other issues would prevent PostgreSQL enums from working if the
enum values were passed as individual values. Logic has been updated so
that "name" is transferred over, but also that the default :class:`.Enum`
which is against the plain Python `enum.Enum` class or other "empty" enum
won't set a hardcoded name of ``"enum"`` either.
Mike Bayer [Thu, 15 Jun 2023 13:18:38 +0000 (09:18 -0400)]
improve support for declared_attr returning ORMDescriptor
Fixed issue in ORM Annotated Declarative which prevented a
:class:`_orm.declared_attr` with or without
:attr:`_orm.declared_attr.directive` from being used on a mixin which did
not return a :class:`.Mapped` datatype, and instead returned a supplemental
ORM datatype such as :class:`.AssociationProxy`. The Declarative runtime
would erroneously try to interpret this annotation as needing to be
:class:`.Mapped` and raise an error.
Fixed typing issue where using the :class:`.AssociationProxy` return type
from a :class:`_orm.declared_attr` function was disallowed.
Mike Bayer [Wed, 14 Jun 2023 14:51:43 +0000 (10:51 -0400)]
warn for other mapper property objects assigned twice
was already in place for columns via other means
A warning is emitted when an ORM :func:`_orm.relationship` and other
:class:`.MapperProperty` objects are assigned to two different class
attributes at once; only one of the attributes will be mapped. A warning
for this condition was already in place for :class:`_schema.Column` and
:class:`_orm.mapped_column` objects.
Federico Caselli [Mon, 12 Jun 2023 20:55:15 +0000 (22:55 +0200)]
Fixed regression with callables as daclasses defaults
Fixed regression introduced in 2.0.16 by :ticket:`9879` where passing a
callable to the :paramref:`_orm.mapped_column.default` parameter of
:class:`_orm.mapped_column` while also setting ``init=False`` would
interpret this value as a Dataclass default value which would be assigned
directly to new instances of the object directly, bypassing the default
generator taking place as the as the :paramref:`_schema.Column.default`
value generator on the underlying :class:`_schema.Column`. This condition
is now detected so that the previous behavior is maintained, however a
deprecation warning for this ambiguous use is emitted; to populate the
default generator for a :class:`_schema.Column`, the
:paramref:`_orm.mapped_column.insert_default` parameter should be used,
which disambiguates from the :paramref:`_orm.mapped_column.default`
parameter whose name is fixed as per pep-681.
Matthew Martin [Sun, 11 Jun 2023 14:00:22 +0000 (10:00 -0400)]
Add `_WILDCARD_TOKEN` special case in `_chop_path`
### Description
<!-- Describe your changes in detail -->
Add special case to `_chop_path` for initial `_WILDCARD_TOKEN` matching `_DEFAULT_TOKEN` to allow for a top level `undefer_group` load option with other load options. Fixes #9870.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Janek Nouvertné [Sun, 11 Jun 2023 10:07:26 +0000 (06:07 -0400)]
Align `AsyncSession` method annotations with `Session` equivalents
Fixes a few differences in the parameter signatures of `asyncio.ext.AsyncSession` that were misaligned with `orm.Session`. Fixes #9925
### Description
- Change the annotation of the `params` parameter of `.scalar`, `.scalars` and `.stream_scalars` from `_CoreSingleExecuteParams` to `_CoreAnyExecuteParams`
- Add named keyword arguments `bind_arguments` and `execution_options` to `.connection`
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
feat: add `async_creator` argument to `create_async_engine`
Added new :paramref:`_asyncio.create_async_engine.async_creator` parameter
to :func:`.create_async_engine`, which accomplishes the same purpose as the
:paramref:`.create_engine.creator` parameter of :func:`.create_engine`.
This is a no-argument callable that provides a new asyncio connection,
using the asyncio database driver directly. The
:func:`.create_async_engine` function will wrap the driver-level connection
in the appropriate structures. Pull request curtesy of Jack Wotherspoon.
Mike Bayer [Thu, 8 Jun 2023 21:40:01 +0000 (17:40 -0400)]
ensure primary key values all present for ORM bulk UPDATE
Fixed bug in new feature where ORM bulk UPDATE can be combined with a WHERE
clause, added in version 2.0.11 as part of :ticket:`9583`, where sending
dictionaries that did not include the primary key values for each row would
run through the bulk process and include "pk=NULL" for the rows, silently
failing. An exception is now raised if primary key values for bulk UPDATE
are not supplied.
Additionally, document WHERE criteria feature as well as how to
invoke an UPDATE outside of "bulk orm by primary key"
Fixed issue where the :paramref:`.ColumnOperators.like.escape` and similar
parameters did not allow an empty string as an argument that would be
passed through as the "escape" character; this is a supported syntax by
PostgreSQL. Pull requset courtesy Martin Caslavsky.
Mike Bayer [Thu, 8 Jun 2023 15:41:41 +0000 (11:41 -0400)]
deprecate InstanceState.unloaded_expirable
The :attr:`_orm.InstanceState.unloaded_expirable` attribute is a synonym
for :attr:`_orm.InstanceState.unloaded`, and is now deprecated; this
attribute was always implementation-specific and should not have been
public.
Federico Caselli [Wed, 31 May 2023 20:36:00 +0000 (22:36 +0200)]
Guard against mapped classes in map_imperatively.
Added a check to prevent invocation of
meth:`_orm.registry.map_imperatively` using a mapped class as
paramref:`_orm.registry.map_imperatively.local_table`.
pavelserchenia [Fri, 26 May 2023 11:16:54 +0000 (07:16 -0400)]
PG nulls not distinct support
Added support for PostgreSQL 10 ``NULLS NOT DISTINCT`` feature of
unique indexes and unique constraint using the dialect option
postgresql_nulls_not_distinct``.
Updated the reflection logic to also correctly take this option
into account.
Mike Bayer [Mon, 5 Jun 2023 16:26:43 +0000 (12:26 -0400)]
once again note pydantic is never really going to work
In #9894 we see that pydantic's dataclass feature is doing an
end run around SQLAlchemy's collection mechanics that can't
be fixed without special steps, which we would surmise
is what SQLModel is doing here:
Mike Bayer [Tue, 30 May 2023 18:54:42 +0000 (14:54 -0400)]
use internal declarative creator for DeclarativeBaseNoMeta
Fixed issue where :class:`.DeclarativeBaseNoMeta` declarative base class
would not function with non-mapped mixins or abstract classes, raising an
``AttributeError`` instead.
Fixed an issue where generating dataclasses fields that specified a
default`` value and set ``init=False`` would not work.
The dataclasses behavior in this case is to set the default
value on the class, that's not compatible with the descriptors used
by SQLAlchemy. To support this case the default is transformed to
a ``default_factory`` when generating the dataclass.
Mike Bayer [Mon, 29 May 2023 16:25:42 +0000 (12:25 -0400)]
base _O around object, not Any
There were only a few trivial errors related to this
TypeVar being linked to object which were correctable at the
source. It's not clear if there were more errors with older
versions of mypy and/or the codebase when this was first developed.
Federico Caselli [Fri, 26 May 2023 22:36:04 +0000 (00:36 +0200)]
Improve PostgreSQL custom operators
Unified the custom PostgreSQL operator definitions, since they are
shared among multiple different data types.
Use proper precedence on PostgreSQL specific operators, such as ``@>``.
Previously the precedence was wrong, leasing to wrong parenthesis when
rending against and ``ANY`` or ``ALL`` construct.
Mike Bayer [Fri, 26 May 2023 14:51:02 +0000 (10:51 -0400)]
tighten up overwritten attribute detection and use a deprecation
A deprecation warning is emitted whenever a property is added to a
:class:`_orm.Mapper` where an ORM mapped property were already configured,
or an attribute is already present on the class. Previously, there was a
non-deprecation warning for this case that did not emit consistently. The
logic for this warning has been improved so that it detects end-user
replacement of attribute while not having false positives for internal
Declarative and other cases where replacement of descriptors with new ones
is expected.
Federico Caselli [Thu, 25 May 2023 22:17:40 +0000 (00:17 +0200)]
Improve PG reflection compat.
Cast ``NAME`` columns to ``TEXT`` when using ``ARRAY_AGG`` in PostgreSQL
reflection. This seems to improve compatibility with some PostgreSQL
derivatives that may not support aggregations on the ``NAME`` type.
Mike Bayer [Wed, 24 May 2023 02:21:30 +0000 (22:21 -0400)]
add reflection arguments, engine/conn bind to DeferredReflection.prepare
Improved :meth:`.DeferredReflection.prepare` to accept arbitrary ``**kw``
arguments that are passed to :meth:`_schema.MetaData.reflect`, allowing use
cases such as reflection of views as well as dialect-specific arguments to
be passed. Additionally, modernized the
:paramref:`.DeferredReflection.prepare.bind` argument so that either an
:class:`.Engine` or :class:`.Connection` are accepted as the "bind"
argument.
Mike Bayer [Mon, 22 May 2023 17:12:58 +0000 (13:12 -0400)]
restore and test correct defaults for validates() keyword args
Fixed regression in the 2.0 series where the default value of
:paramref:`_orm.validates.include_backrefs` got changed to ``False`` for
the :func:`_orm.validates` function. This default is now restored to
``True``.
Mike Bayer [Fri, 19 May 2023 19:56:31 +0000 (15:56 -0400)]
establish Uuid as emulated, pg.UUID as nativeforemulated
Repaired the base :class:`.Uuid` datatype for the PostgreSQL dialect to
make proper use of the PG-specific ``UUID`` internal datatype when
"native_uuid" is selected, so that PG driver behaviors are included. This
issue became apparent due to the insertmanyvalues improvement made as part
of :ticket:`9618`, where in a similar manner as that of :ticket:` 9739`
where the asyncpg driver is very sensitive to datatype casts being present
or not, the PostgreSQL driver-specific native ``UUID`` datatype must be
invoked when this generic type is used.
also consolidate imv tests for largebinary, uuid into
suite/test_insert.py
Mike Bayer [Fri, 19 May 2023 13:35:30 +0000 (09:35 -0400)]
use an ORM compile state for all statements with any ORM entities anywhere
As more projects are using new-style "2.0" ORM querying, it's becoming
apparent that the conditional nature of "autoflush", being based on whether
or not the given statement refers to ORM entities, is becoming more of a
key behavior. Up until now, the "ORM" flag for a statement has been loosely
based around whether or not the statement returns rows that correspond to
ORM entities or columns; the original purpose of the "ORM" flag was to
enable ORM-entity fetching rules which apply post-processing to Core result
sets as well as ORM loader strategies to the statement. For statements
that don't build on rows that contain ORM entities, the "ORM" flag was
considered to be mostly unnecessary.
It still may be the case that "autoflush" would be better taking effect for
*all* usage of :meth:`_orm.Session.execute` and related methods, even for
purely Core SQL constructs. However, this still could impact legacy cases
where this is not expected and may be more of a 2.1 thing. For now however,
the rules for the "ORM-flag" have been opened up so that a statement that
includes ORM entities or attributes anywhere within, including in the WHERE
/ ORDER BY / GROUP BY clause alone, within scalar subqueries, etc. will
enable this flag. This will cause "autoflush" to occur for such statements
and also be visible via the :attr:`_orm.ORMExecuteState.is_orm_statement`
event-level attribute.
Nick Crews [Thu, 18 May 2023 12:01:18 +0000 (08:01 -0400)]
Move try_cast() from mssql to base
Generalized the MSSQL :func:`_sql.try_cast` function into the
``sqlalchemy.`` import namespace so that it may be implemented by third
party dialects as well. Within SQLAlchemy, the :func:`_sql.try_cast`
function remains a SQL Server-only construct that will raise
:class:`.CompileError` if used with backends that don't support it.
:func:`_sql.try_cast` implements a CAST where un-castable conversions are
returned as NULL, instead of raising an error. Theoretically, the construct
could be implemented by third party dialects for Google BigQuery, DuckDB,
and Snowflake, and possibly others.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
