Mike Bayer [Mon, 5 Jun 2023 16:26:43 +0000 (12:26 -0400)]
once again note pydantic is never really going to work
In #9894 we see that pydantic's dataclass feature is doing an
end run around SQLAlchemy's collection mechanics that can't
be fixed without special steps, which we would surmise
is what SQLModel is doing here:
Fixed an issue where generating dataclasses fields that specified a
default`` value and set ``init=False`` would not work.
The dataclasses behavior in this case is to set the default
value on the class, that's not compatible with the descriptors used
by SQLAlchemy. To support this case the default is transformed to
a ``default_factory`` when generating the dataclass.
Mike Bayer [Mon, 29 May 2023 16:25:42 +0000 (12:25 -0400)]
base _O around object, not Any
There were only a few trivial errors related to this
TypeVar being linked to object which were correctable at the
source. It's not clear if there were more errors with older
versions of mypy and/or the codebase when this was first developed.
Federico Caselli [Fri, 26 May 2023 22:36:04 +0000 (00:36 +0200)]
Improve PostgreSQL custom operators
Unified the custom PostgreSQL operator definitions, since they are
shared among multiple different data types.
Use proper precedence on PostgreSQL specific operators, such as ``@>``.
Previously the precedence was wrong, leasing to wrong parenthesis when
rending against and ``ANY`` or ``ALL`` construct.
Mike Bayer [Fri, 26 May 2023 14:51:02 +0000 (10:51 -0400)]
tighten up overwritten attribute detection and use a deprecation
A deprecation warning is emitted whenever a property is added to a
:class:`_orm.Mapper` where an ORM mapped property were already configured,
or an attribute is already present on the class. Previously, there was a
non-deprecation warning for this case that did not emit consistently. The
logic for this warning has been improved so that it detects end-user
replacement of attribute while not having false positives for internal
Declarative and other cases where replacement of descriptors with new ones
is expected.
Federico Caselli [Thu, 25 May 2023 22:17:40 +0000 (00:17 +0200)]
Improve PG reflection compat.
Cast ``NAME`` columns to ``TEXT`` when using ``ARRAY_AGG`` in PostgreSQL
reflection. This seems to improve compatibility with some PostgreSQL
derivatives that may not support aggregations on the ``NAME`` type.
Mike Bayer [Wed, 24 May 2023 02:21:30 +0000 (22:21 -0400)]
add reflection arguments, engine/conn bind to DeferredReflection.prepare
Improved :meth:`.DeferredReflection.prepare` to accept arbitrary ``**kw``
arguments that are passed to :meth:`_schema.MetaData.reflect`, allowing use
cases such as reflection of views as well as dialect-specific arguments to
be passed. Additionally, modernized the
:paramref:`.DeferredReflection.prepare.bind` argument so that either an
:class:`.Engine` or :class:`.Connection` are accepted as the "bind"
argument.
Mike Bayer [Mon, 22 May 2023 17:12:58 +0000 (13:12 -0400)]
restore and test correct defaults for validates() keyword args
Fixed regression in the 2.0 series where the default value of
:paramref:`_orm.validates.include_backrefs` got changed to ``False`` for
the :func:`_orm.validates` function. This default is now restored to
``True``.
Mike Bayer [Fri, 19 May 2023 19:56:31 +0000 (15:56 -0400)]
establish Uuid as emulated, pg.UUID as nativeforemulated
Repaired the base :class:`.Uuid` datatype for the PostgreSQL dialect to
make proper use of the PG-specific ``UUID`` internal datatype when
"native_uuid" is selected, so that PG driver behaviors are included. This
issue became apparent due to the insertmanyvalues improvement made as part
of :ticket:`9618`, where in a similar manner as that of :ticket:` 9739`
where the asyncpg driver is very sensitive to datatype casts being present
or not, the PostgreSQL driver-specific native ``UUID`` datatype must be
invoked when this generic type is used.
also consolidate imv tests for largebinary, uuid into
suite/test_insert.py
Mike Bayer [Fri, 19 May 2023 13:35:30 +0000 (09:35 -0400)]
use an ORM compile state for all statements with any ORM entities anywhere
As more projects are using new-style "2.0" ORM querying, it's becoming
apparent that the conditional nature of "autoflush", being based on whether
or not the given statement refers to ORM entities, is becoming more of a
key behavior. Up until now, the "ORM" flag for a statement has been loosely
based around whether or not the statement returns rows that correspond to
ORM entities or columns; the original purpose of the "ORM" flag was to
enable ORM-entity fetching rules which apply post-processing to Core result
sets as well as ORM loader strategies to the statement. For statements
that don't build on rows that contain ORM entities, the "ORM" flag was
considered to be mostly unnecessary.
It still may be the case that "autoflush" would be better taking effect for
*all* usage of :meth:`_orm.Session.execute` and related methods, even for
purely Core SQL constructs. However, this still could impact legacy cases
where this is not expected and may be more of a 2.1 thing. For now however,
the rules for the "ORM-flag" have been opened up so that a statement that
includes ORM entities or attributes anywhere within, including in the WHERE
/ ORDER BY / GROUP BY clause alone, within scalar subqueries, etc. will
enable this flag. This will cause "autoflush" to occur for such statements
and also be visible via the :attr:`_orm.ORMExecuteState.is_orm_statement`
event-level attribute.
Nick Crews [Thu, 18 May 2023 12:01:18 +0000 (08:01 -0400)]
Move try_cast() from mssql to base
Generalized the MSSQL :func:`_sql.try_cast` function into the
``sqlalchemy.`` import namespace so that it may be implemented by third
party dialects as well. Within SQLAlchemy, the :func:`_sql.try_cast`
function remains a SQL Server-only construct that will raise
:class:`.CompileError` if used with backends that don't support it.
:func:`_sql.try_cast` implements a CAST where un-castable conversions are
returned as NULL, instead of raising an error. Theoretically, the construct
could be implemented by third party dialects for Google BigQuery, DuckDB,
and Snowflake, and possibly others.
Mike Bayer [Mon, 15 May 2023 15:37:48 +0000 (11:37 -0400)]
omit pypy from sizeof tests
Fixed test that relied on the ``sys.getsizeof()`` function to not run on
pypy, where this function appears to have different behavior than it does
on cpython.
Mike Bayer [Sat, 13 May 2023 16:32:31 +0000 (12:32 -0400)]
limit joinedload exclusion rules to immediate mapped columns
Fixed issue where using additional relationship criteria with the
:func:`_orm.joinedload` loader option, where the additional criteria itself
contained correlated subqueries that referred to the joined entities and
therefore also required "adaption" to aliased entities, would be excluded
from this adaption, producing the wrong ON clause for the joinedload.
Mike Bayer [Sat, 13 May 2023 03:42:09 +0000 (23:42 -0400)]
remove "aliased class pool" caching approach
Modified the ``JoinedLoader`` implementation to use a simpler approach in
one particular area where it previously used a cached structure that would
be shared among threads. The rationale is to avoid a potential race
condition which is suspected of being the cause of a particular crash
that's been reported multiple times. The cached structure in question is
still ultimately "cached" via the compiled SQL cache, so a performance
degradation is not anticipated.
The change also modifies the tests for None in context.secondary
to ensure no None values are in this list, as this is suspected
as being the immediate cause of the issue in #9777. The cached
AliasedClass thing is suspected as being the origination of the
cause, as under high concurrency many threads might all access
that AliasedClass immediately, which seems a reasonable place
that the "adapter returning None" symptom might be originating.
As of yet we don't have a self-contained reproducer for the
issue, some initial attempts with threads are not showing any
issue.
Mike Bayer [Fri, 12 May 2023 18:34:00 +0000 (14:34 -0400)]
include create_type in pg.ENUM.adapt; test all attrs
Fixed apparently very old issue where the
:paramref:`_postgresql.ENUM.create_type` parameter, when set to its
non-default of ``False``, would not be propagated when the
:class:`_schema.Column` which it's a part of were copied, as is common when
using ORM Declarative mixins.
Mike Bayer [Thu, 11 May 2023 14:06:10 +0000 (10:06 -0400)]
skip ORM loading setups for non-toplevel DML
Fixed regression where use of :func:`_dml.update` or :func:`_dml_delete`
within a :class:`_sql.CTE` construct, then used in a :func:`_sql.select`,
would raise a :class:`.CompileError` as a result of ORM related rules for
performing ORM-level update/delete statements.
Mike Bayer [Thu, 11 May 2023 02:03:18 +0000 (22:03 -0400)]
return empty MappedColumn() at early pep-593 step
Fixed issue in new ORM Annotated Declarative where using a
:class:`_schema.ForeignKey` (or other column-level constraint) inside of
:func:`_orm.mapped_column` which is then copied out to models via pep-593
``Annotated`` would apply duplicates of each constraint to the
:class:`_schema.Column` as produced in the target :class:`_schema.Table`,
leading to incorrect CREATE TABLE DDL as well as migration directives under
Alembic.
Mike Bayer [Wed, 10 May 2023 15:08:07 +0000 (11:08 -0400)]
add full parameter types for ORM with_for_update
Fixed typing for the :paramref:`_orm.Session.get.with_for_update` parameter
of :meth:`_orm.Session.get` and :meth:`_orm.Session.refresh` (as well as
corresponding methods on :class:`_asyncio.AsyncSession`) to accept boolean
``True`` and all other argument forms accepted by the parameter at runtime.
Mike Bayer [Sun, 30 Apr 2023 22:27:24 +0000 (18:27 -0400)]
fix test suite warnings
fix a handful of warnings that were emitting but not raising,
usually because they were inside an "expect_warnings" block.
modify "expect_warnings" to always use "raise_on_any_unexpected"
behavior; remove this parameter.
Fixed issue in semi-private ``await_only()`` and ``await_fallback()``
concurrency functions where the given awaitable would remain un-awaited if
the function threw a ``GreenletError``, which could cause "was not awaited"
warnings later on if the program continued. In this case, the given
awaitable is now cancelled before the exception is thrown.
Mike Bayer [Tue, 9 May 2023 15:19:43 +0000 (11:19 -0400)]
guard against duplicate mutable event listeners
Fixed issue in :class:`_mutable.Mutable` where event registration for ORM
mapped attributes would be called repeatedly for mapped inheritance
subclasses, leading to duplicate events being invoked in inheritance
hierarchies.
Mike Bayer [Tue, 9 May 2023 15:33:30 +0000 (11:33 -0400)]
allow column named twice warning to take effect
Fixed issue in :func:`_orm.mapped_column` construct where the correct
warning for "column X named directly multiple times" would not be emitted
when ORM mapped attributes referred to the same :class:`_schema.Column`, if
the :func:`_orm.mapped_column` construct were involved, raising an internal
assertion instead.
Fixed the base class for dialect-specific float/double types; Oracle
:class:`_oracle.BINARY_DOUBLE` now subclasses :class:`_sqltypes.Double`,
and internal types for :class:`_sqltypes.Float` for asyncpg and pg8000 now
correctly subclass :class:`_sqltypes.Float`.
Added suite tests to ensure that floating point types, such as
class:`_types.Float` and :class:`_types.Double` are not resolved as
class:`_types.Numeric` in the dialect, since it may not compatible in
all cases, such as when casting a value.
Mike Bayer [Fri, 28 Apr 2023 16:07:09 +0000 (12:07 -0400)]
implement FromLinter for UPDATE, DELETE statements
Implemented the "cartesian product warning" for UPDATE and DELETE
statements, those which include multiple tables that are not correlated
together in some way.
Fixed issue where :func:`_dml.update` construct that included multiple
tables and no VALUES clause would raise with an internal error. Current
behavior for :class:`_dml.Update` with no values is to generate a SQL
UPDATE statement with an empty "set" clause, so this has been made
consistent for this specific sub-case.
Mike Bayer [Fri, 5 May 2023 15:50:29 +0000 (11:50 -0400)]
add AsyncAttrs
Added a new helper mixin :class:`_asyncio.AsyncAttrs` that seeks to improve
the use of lazy-loader and other expired or deferred ORM attributes with
asyncio, providing a simple attribute accessor that provides an ``await``
interface to any ORM attribute, whether or not it needs to emit SQL.
Mike Bayer [Fri, 5 May 2023 13:16:10 +0000 (09:16 -0400)]
add explicit step to set populate_existing for bulk insert
Fixed issue in new :ref:`orm_queryguide_upsert_returning` feature where the
``populate_existing`` execution option was not being propagated to the
loading option, preventing existing attributes from being refreshed
in-place.
Mike Bayer [Thu, 4 May 2023 02:33:28 +0000 (22:33 -0400)]
add bind casts for BYTEA on asyncpg
Fixed another regression due to the "insertmanyvalues" change in 2.0.10 as
part of :ticket:`9618`, in a similar way as regression :ticket:`9701`, where
:class:`.LargeBinary` datatypes also need additional casts on when using the
asyncpg driver specifically in order to work with the new bulk INSERT
format.
Mike Bayer [Sun, 30 Apr 2023 17:56:33 +0000 (13:56 -0400)]
do not allow non-cache-key entity objects in annotations
Fixed critical caching issue where combination of :func:`_orm.aliased()`
:func:`_sql.case` and :func:`_hybrid.hybrid_property` expressions would
cause a cache key mismatch, leading to cache keys that held onto the actual
:func:`_orm.aliased` object while also not matching each other, filling up
the cache.
Fixed issues regarding reflection of comments for :class:`_schema.Table`
and :class:`_schema.Column` objects, where the comments contained control
characters such as newlines. Additional testing support for these
characters as well as extended Unicode characters in table and column
comments (the latter of which aren't supported by MySQL/MariaDB) added to
testing overall.
Mike Bayer [Thu, 27 Apr 2023 20:48:25 +0000 (16:48 -0400)]
improve natural_path usage in two places
Fixed loader strategy pathing issues where eager loaders such as
:func:`_orm.joinedload` / :func:`_orm.selectinload` would fail to traverse
fully for many-levels deep following a load that had a
:func:`_orm.with_polymorphic` or similar construct as an interim member.
Here we can take advantage of 2.0's refactoring of strategy_options
to identify the "chop_path" concept can be simplified to work
with "natural" paths alone.
In addition, identified existing
logic in PropRegistry that works fine, but needed the "is_unnatural"
attribute to be more accurate for a given path, so we set that
up front to True if the ancestor is_unnatural.
Mike Bayer [Fri, 28 Apr 2023 03:00:18 +0000 (23:00 -0400)]
adjust fwd_ref logic
Fixed issue where ORM Annotated Declarative would not resolve forward
references correctly in all cases; in particular, when using
``from __future__ import annotations`` in combination with Pydantic
dataclasses.
Add public alias of type _ColumnExpressionArgument
Added type ``ColumnExpressionArgument`` as a public alias of an internal
type. This type is useful since it's what' accepted by the sqlalchemy in
many api calls, such as :meth:`_sql.Select.where`, :meth:`_sql.and` and
many other.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
