Mike Bayer [Wed, 21 Apr 2021 14:39:09 +0000 (10:39 -0400)]
Limit dc field logic to only fields that are definitely dc
Fixed regression where recent changes to support Python dataclasses had the
inadvertent effect that an ORM mapped class could not successfully override
the ``__new__()`` method.
In this case the "__new__" method comes out as staticmethod in
cls.__dict__ vs. a function in the metaclass dict_, so comparing
using identity fails. I was hoping not to have too much
"dataclass" hardcoded, the logic here if it were generalized
to other attribute declaration systems there
would still have a flag that indicates an attribute is part of
the "special declaration system".
Mike Bayer [Tue, 20 Apr 2021 19:09:51 +0000 (15:09 -0400)]
Propagate compiler kw for visit_values to parameters
Fixed issue in SQL compiler where the bound parameters set up for a
:class:`.Values` construct wouldn't be positionally tracked correctly if
inside of a :class:`_sql.CTE`, affecting database drivers that support
VALUES + ctes and use positional parameters such as SQL Server in
particular as well as asyncpg. The fix also repairs support for
compiler flags such as ``literal_binds``.
Mike Bayer [Tue, 20 Apr 2021 18:09:51 +0000 (14:09 -0400)]
Allow immediateload to use_get for recursive call
Altered some of the behavior repaired in :ticket:`6232` where the
``immediateload`` loader strategy no longer goes into recursive loops; the
modification is that an eager load (joinedload, selectinload, or
subqueryload) from A->bs->B which then states ``immediateload`` for a
simple manytoone B->a->A that's in the identity map will populate the B->A,
so that this attribute is back-populated when the collection of A/A.bs are
loaded. This allows the objects to be functional when detached.
The :paramref:`_types.DateTime.timezone` parameter when set to ``True``
will now make use of the ``DATETIMEOFFSET`` column type with SQL Server
when used to emit DDL, rather than ``DATETIME`` where the flag was silently
ignored.
Mike Bayer [Mon, 19 Apr 2021 22:03:12 +0000 (18:03 -0400)]
Re-infer statements that got more specific on subsequent pass
Fixed issue where mypy plugin would not correctly interpret an explicit
:class:`_orm.Mapped` annotation in conjunction with a
:func:`_orm.relationship` that refers to a class by string name; the
correct annotation would be downgraded to a less specific one leading to
typing errors.
The thing figured out here is that after we've already scanned
a class in the semanal stage and created DeclClassApplied,
when we are called again with that same DeclClassApplied, for this
specific kind of case we actually now have *better* types than
we did before, where the left side that looked like
List?[Address?] now seems to say
builtins.list[official.module.Address] - so let's take the
right side expression again, this time embedded in our
Mapped._empty_constructor() expression, and run the infer
all over again just like mypy would. Just not setting the
"wrong" type here fixed the test cases but by re-applying the
whole infer we get the correct Mapped[] on the left side too.
Mike Bayer [Tue, 20 Apr 2021 15:33:02 +0000 (11:33 -0400)]
Narrow refresh populate_existing to just refresh_state
Fixed additional regression caused by the "eagerloaders on refresh" feature
added in :ticket:`1763` where the refresh operation historically would set
``populate_existing``, which given the new feature now overwrites pending
changes on eagerly loaded objects when autoflush is false. The
populate_existing flag has been turned off for this case and a more
specific method used to ensure the correct attributes refreshed.
Mike Bayer [Mon, 19 Apr 2021 00:30:14 +0000 (20:30 -0400)]
Bypass declared_attr w/ a custom wrapper for lambda criteria
Fixed bug in new :func:`_orm.with_loader_criteria` feature where using a
mixin class with :func:`_orm.declared_attr` on an attribute that were
accessed inside the custom lambda would emit a warning regarding using an
unmapped declared attr, when the lambda callable were first initialized.
This warning is now prevented using special instrumentation for this
lambda initialization step.
Fixed an issue that prevented using ``scalar_one`` or
``scalar_one_or_none()`` after a call to ``unique``.
Additionally includes some clarifications in result.py
and also removes pep-484 annotations for now as these
are duplicate on top of sqlalchemy2-stubs.
Mike Bayer [Fri, 16 Apr 2021 15:06:56 +0000 (11:06 -0400)]
Don't stringify unnamed column elements when proxying
Repaired and solidified issues regarding custom functions and other
arbitrary expression constructs which within SQLAlchemy's column labeling
mechanics would seek to use ``str(obj)`` to get a string representation to
use as an anonymous column name in the ``.c`` collection of a subquery.
This is a very legacy behavior that performs poorly and leads to lots of
issues, so has been revised to no longer perform any compilation by
establishing specific methods on :class:`.FunctionElement` to handle this
case, as SQL functions are the only use case that it came into play. An
effect of this behavior is that an unlabeled column expression with no
derivable name will be given an arbitrary label starting with the prefix
``"_no_label"`` in the ``.c`` collection of a subquery; these were
previously being represented either as the generic stringification of that
expression, or as an internal symbol.
This change seeks to make the concept of "anon name" more private
and renames anon_label and anon_key_label to _anon_name_label
and _anon_key_label. There's no end-user utility to these accessors
and we need to be able to reorganize these as well.
Mike Bayer [Sat, 17 Apr 2021 04:30:29 +0000 (00:30 -0400)]
pass asfrom correctly in compilers
Fixed an argument error in the default and PostgreSQL compilers that
would interfere with an UPDATE..FROM or DELETE..FROM..USING statement
that was then SELECTed from as a CTE.
The incorrect pattern was also fixed in the mysql and sybase dialects.
MySQL supports CTEs but not "returning".
Mike Bayer [Sat, 17 Apr 2021 03:58:48 +0000 (23:58 -0400)]
Uniquify FROMs when traversing through select
Fixed a critical performance issue where the traversal of a
:func:`_sql.select` construct would traverse a repetitive product of the
represented FROM clauses as they were each referred towards by columns in
the columns clause; for a series of nested subqueries with lots of columns
this could cause a large delay and significant memory growth. This
traversal is used by a wide variety of SQL and ORM functions, including by
the ORM :class:`_orm.Session` when it's configured to have
"table-per-bind", which while this is not a common use case, it seems to be
what Flask-SQLAlchemy is hardcoded as using, so the issue impacts
Flask-SQLAlchemy users. The traversal has been repaired to uniqify on FROM
clauses which was effectively what would happen implicitly with the pre-1.4
architecture.
Mike Bayer [Fri, 16 Apr 2021 12:56:17 +0000 (08:56 -0400)]
synonym fixes and enhancements
Fixed regression where an attribute that is mapped to a
:func:`_orm.synonym` could not be used in column loader options such as
:func:`_orm.load_only`.
Established support for :func:`_orm.synonym` in conjunction with hybrid
property, associationproxy, including that synonyms can be established
linking to these constructs which work fully. This is a behavior that was
semi-explicitly disallowed previously, however since it did not fail in
every scenario, explicit support for assoc proxy and hybrids has been
added.
Fixed a cache leak involving the :func:`_orm.with_expression` loader
option, where the given SQL expression would not be correctly considered as
part of the cache key.
Additionally, fixed regression involving the corresponding
:func:`_orm.query_expression` feature. While the bug technically exists in
1.3 as well, it was not exposed until 1.4. The "default expr" value of
``null()`` would be rendered when not needed, and additionally was also not
adapted correctly when the ORM rewrites statements such as when using
joined eager loading. The fix ensures "singleton" expressions like ``NULL``
and ``true`` aren't "adapted" to refer to columns in ORM statements, and
additionally ensures that a :func:`_orm.query_expression` with no default
expression doesn't render in the statement if a
:func:`_orm.with_expression` isn't used.
Mike Bayer [Wed, 14 Apr 2021 20:14:36 +0000 (16:14 -0400)]
don't base compilation off the int value of offset/limit part II
Fixed an additional regression in the same area as that of :ticket:`6184`,
where using a value of 0 for OFFSET in conjunction with LIMIT with SQL
Server would create a statement using "TOP", as was the behavior in 1.3,
however due to caching would then fail to respond accordingly to other
values of OFFSET. If the "0" wasn't first, then it would be fine. For the
fix, the "TOP" syntax is now only emitted if the OFFSET value is omitted
entirely, that is, :meth:`_sql.Select.offset` is not used. Note that this
change now requires that if the "with_ties" or "percent" modifiers are
used, the statement can't specify an OFFSET of zero, it now needs to be
omitted entirely.
Mike Bayer [Tue, 13 Apr 2021 14:52:00 +0000 (10:52 -0400)]
Support DEFAULT VALUES and VALUES(DEFAULT) individually
Fixed regression where the introduction of the INSERT syntax "INSERT...
VALUES (DEFAULT)" was not supported on some backends that do however
support "INSERT..DEFAULT VALUES", including SQLite. The two syntaxes are
now each individually supported or non-supported for each dialect, for
example MySQL supports "VALUES (DEFAULT)" but not "DEFAULT VALUES".
Support for Oracle is still not enabled as there are unresolved issues
in using RETURNING at the same time.
Mike Bayer [Wed, 14 Apr 2021 16:12:08 +0000 (12:12 -0400)]
Explicitly test for Connection in dialect.has_table()
The :meth:`_engine.Dialect.has_table` method now raises an informative
exception if a non-Connection is passed to it, as this incorrect behavior
seems to be common. This method is not intended for external use outside
of a dialect. Please use the :meth:`.Inspector.has_table` method
or for cross-compatibility with older SQLAlchemy versions, the
:meth:`_engine.Engine.has_table` method.
Mike Bayer [Tue, 13 Apr 2021 03:48:02 +0000 (23:48 -0400)]
Disable raiseload in immediateload strategy
Fixed issue in the new feature of :meth:`_orm.Session.refresh` introduced
by :ticket:`1763` where eagerly loaded relationships are also refreshed,
where the ``lazy="raise"`` and ``lazy="raise_on_sql"`` loader strategies
would interfere with the :func:`_orm.immediateload` loader strategy, thus
breaking the feature for relationships that were loaded with
:func:`_orm.selectinload`, :func:`_orm.subqueryload` as well.
Also update some docs re: refresh, populate existing, etc.
Mike Bayer [Mon, 12 Apr 2021 18:33:50 +0000 (14:33 -0400)]
Ensure bindparam key escaping applied in all cases
Fixed regression where the :class:`_sql.BindParameter` object would not
properly render for an IN expression (i.e. using the "post compile" feature
in 1.4) if the object were copied from either an internal cloning
operation, or from a pickle operation, and the parameter name contained
spaces or other special characters.
Mike Bayer [Sun, 11 Apr 2021 21:15:55 +0000 (17:15 -0400)]
Return Row for CursorResult.inserted_primary_key
The tuple returned by :attr:`.CursorResult.inserted_primary_key` is now a
:class:`_result.Row` object with a named tuple interface on top of the
existing tuple interface.
Mike Bayer [Fri, 9 Apr 2021 15:09:16 +0000 (11:09 -0400)]
Dont return outer transaction for _subtrans flag
Fixed critical regression where the :class:`_orm.Session` could fail to
"autobegin" a new transaction when a flush occurred without an existing
transaction in place, implicitly placing the :class:`_orm.Session` into
legacy autocommit mode which commit the transaction. The
:class:`_orm.Session` now has a check that will prevent this condition from
occurring, in addition to repairing the flush issue.
Additionally, scaled back part of the change made as part of :ticket:`5226`
which can run autoflush during an unexpire operation, to not actually
do this in the case of a :class:`_orm.Session` using legacy
:paramref:`_orm.Session.autocommit` mode, as this incurs a commit within
a refresh operation.
Mike Bayer [Fri, 9 Apr 2021 14:46:21 +0000 (10:46 -0400)]
Apply recursive check to immediateloader and generalize
Fixed critical regression caused by the new feature added as part of
:ticket:`1763`, eager loaders are invoked on unexpire operations. The new
feature makes use of the "immediateload" eager loader strategy as a
substitute for a collection loading strategy, which unlike the other
"post-load" strategies was not accommodating for recursive invocations
between mutually-dependent relationships, leading to recursion overflow
errors.
Add ``supports_schema = True`` to DefaultDialect and modify
requirements.py to use that attribute so third-party dialects
can explicitly indicate that they do *not* support schemas by
specifying ``supports_schema = False`` in their Dialect class.
Mike Bayer [Fri, 9 Apr 2021 13:25:04 +0000 (09:25 -0400)]
document scalar_subquery() with column_property()
Other doc cleanups and fixes as well, including more
explicitness that declarative attribute assignment only
works when a declarative base is involved; other forms
need to call add_property() directly.
Mike Bayer [Wed, 7 Apr 2021 17:57:39 +0000 (13:57 -0400)]
Fix LegacyRow/Row index access
Fixed up the behavior of the :class:`_result.Row` object when dictionary
access is used upon it, meaning converting to a dict via ``dict(row)`` or
accessing members using strings or other objects i.e. ``row["some_key"]``
works as it would with a dictionary, rather than raising ``TypeError`` as
would be the case with a tuple, whether or not the C extensions are in
place. This was originally supposed to emit a 2.0 deprecation warning for
the "non-future" case using :class:`_result.LegacyRow`, and was to raise
``TypeError`` for the "future" :class:`_result.Row` class. However, the C
version of :class:`_result.Row` was failing to raise this ``TypeError``,
and to complicate matters, the :meth:`_orm.Session.execute` method now
returns :class:`_result.Row` in all cases to maintain consistency with the
ORM result case, so users who didn't have C extensions installed would
see different behavior in this one case for existing pre-1.4 style
code.
Therefore, in order to soften the overall upgrade scheme as most users have
not been exposed to the more strict behavior of :class:`_result.Row` up
through 1.4.6, :class:`_result.LegacyRow` and :class:`_result.Row` both
provide for string-key access as well as support for ``dict(row)``, in all
cases emitting the 2.0 deprecation warning when ``SQLALCHEMY_WARN_20`` is
enabled. The :class:`_result.Row` object still uses tuple-like behavior for
``__contains__``, which is probably the only noticeable behavioral change
compared to :class:`_result.LegacyRow`, other than the removal of
dictionary-style methods ``values()`` and ``items()``.
Also remove filters for result set warnings.
callcounts updated for 2.7/ 3.9, am pushing jenkins to use python 3.9
now
Support `TypeDecorator` subclasses in `Column()` declarations
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Currently, the plugin resolves `TypeDecorator` subclasses in `Column()` declarations to `Mapping[_T]` instead of the correct type in the class declaration.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Thu, 8 Apr 2021 01:43:17 +0000 (21:43 -0400)]
Infer types in BindParameter when expanding=True
Enhanced the "expanding" feature used for :meth:`_sql.ColumnOperators.in_`
operations to infer the type of expression from the right hand list of
elements, if the left hand side does not have any explicit type set up.
This allows the expression to support stringification among other things.
In 1.3, "expanding" was not automatically used for
:meth:`_sql.ColumnOperators.in_` expressions, so in that sense this change
fixes a behavioral regression.
Mike Bayer [Wed, 7 Apr 2021 23:22:52 +0000 (19:22 -0400)]
convert subqueryload paths for multilevel
Fixed regression where the :func:`_orm.subqueryload` loader strategy would
fail to correctly accommodate sub-options, such as a :func:`_orm.defer`
option on a column, if the "path" of the subqueryload were more than one
level deep.
Mike Bayer [Wed, 7 Apr 2021 21:51:34 +0000 (17:51 -0400)]
support multivalues insert on strsqlcompiler
Fixed the "stringify" compiler to support a basic stringification
of a "multirow" INSERT statement, i.e. one with multiple tuples
following the VALUES keyword.
Mike Bayer [Wed, 7 Apr 2021 14:56:12 +0000 (10:56 -0400)]
Expand reg for schema translate map for most special characters
Fixed regression where usage of a token in the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
dictionary which contained special characters such as braces would fail to
be substituted properly. Use of square bracket characters ``[]`` is now
explicitly disallowed as these are used as a delimiter character in the
current implementation.
Mike Bayer [Wed, 7 Apr 2021 14:40:14 +0000 (10:40 -0400)]
Add test support for merge_frozen_result
Fixed regression where the :func:`_orm.merge_frozen_result` function relied
upon by the dogpile.caching example was not included in tests and began
failing due to incorrect internal arguments.
Mike Bayer [Wed, 7 Apr 2021 14:26:31 +0000 (10:26 -0400)]
Check for hybrid's attribute name and support no name
Fixed regression where the ORM compilation scheme would assume the function
name of a hybrid property would be the same as the attribute name in such a
way that an ``AttributeError`` would be raised, when it would attempt to
determine the correct name for each element in a result tuple. A similar
issue exists in 1.3 but only impacts the names of tuple rows. The fix here
adds a check that the hybrid's function name is actually present in the
``__dict__`` of the class or its superclasses before assigning this name;
otherwise, the hybrid is considered to be "unnamed" and ORM result tuples
will use the naming scheme of the underlying expression.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
