Mike Bayer [Sat, 17 Apr 2021 04:30:29 +0000 (00:30 -0400)]
pass asfrom correctly in compilers
Fixed an argument error in the default and PostgreSQL compilers that
would interfere with an UPDATE..FROM or DELETE..FROM..USING statement
that was then SELECTed from as a CTE.
The incorrect pattern was also fixed in the mysql and sybase dialects.
MySQL supports CTEs but not "returning".
Mike Bayer [Sat, 17 Apr 2021 03:58:48 +0000 (23:58 -0400)]
Uniquify FROMs when traversing through select
Fixed a critical performance issue where the traversal of a
:func:`_sql.select` construct would traverse a repetitive product of the
represented FROM clauses as they were each referred towards by columns in
the columns clause; for a series of nested subqueries with lots of columns
this could cause a large delay and significant memory growth. This
traversal is used by a wide variety of SQL and ORM functions, including by
the ORM :class:`_orm.Session` when it's configured to have
"table-per-bind", which while this is not a common use case, it seems to be
what Flask-SQLAlchemy is hardcoded as using, so the issue impacts
Flask-SQLAlchemy users. The traversal has been repaired to uniqify on FROM
clauses which was effectively what would happen implicitly with the pre-1.4
architecture.
Mike Bayer [Fri, 16 Apr 2021 12:56:17 +0000 (08:56 -0400)]
synonym fixes and enhancements
Fixed regression where an attribute that is mapped to a
:func:`_orm.synonym` could not be used in column loader options such as
:func:`_orm.load_only`.
Established support for :func:`_orm.synonym` in conjunction with hybrid
property, associationproxy, including that synonyms can be established
linking to these constructs which work fully. This is a behavior that was
semi-explicitly disallowed previously, however since it did not fail in
every scenario, explicit support for assoc proxy and hybrids has been
added.
Fixed a cache leak involving the :func:`_orm.with_expression` loader
option, where the given SQL expression would not be correctly considered as
part of the cache key.
Additionally, fixed regression involving the corresponding
:func:`_orm.query_expression` feature. While the bug technically exists in
1.3 as well, it was not exposed until 1.4. The "default expr" value of
``null()`` would be rendered when not needed, and additionally was also not
adapted correctly when the ORM rewrites statements such as when using
joined eager loading. The fix ensures "singleton" expressions like ``NULL``
and ``true`` aren't "adapted" to refer to columns in ORM statements, and
additionally ensures that a :func:`_orm.query_expression` with no default
expression doesn't render in the statement if a
:func:`_orm.with_expression` isn't used.
Mike Bayer [Wed, 14 Apr 2021 20:14:36 +0000 (16:14 -0400)]
don't base compilation off the int value of offset/limit part II
Fixed an additional regression in the same area as that of :ticket:`6184`,
where using a value of 0 for OFFSET in conjunction with LIMIT with SQL
Server would create a statement using "TOP", as was the behavior in 1.3,
however due to caching would then fail to respond accordingly to other
values of OFFSET. If the "0" wasn't first, then it would be fine. For the
fix, the "TOP" syntax is now only emitted if the OFFSET value is omitted
entirely, that is, :meth:`_sql.Select.offset` is not used. Note that this
change now requires that if the "with_ties" or "percent" modifiers are
used, the statement can't specify an OFFSET of zero, it now needs to be
omitted entirely.
Mike Bayer [Tue, 13 Apr 2021 14:52:00 +0000 (10:52 -0400)]
Support DEFAULT VALUES and VALUES(DEFAULT) individually
Fixed regression where the introduction of the INSERT syntax "INSERT...
VALUES (DEFAULT)" was not supported on some backends that do however
support "INSERT..DEFAULT VALUES", including SQLite. The two syntaxes are
now each individually supported or non-supported for each dialect, for
example MySQL supports "VALUES (DEFAULT)" but not "DEFAULT VALUES".
Support for Oracle is still not enabled as there are unresolved issues
in using RETURNING at the same time.
Mike Bayer [Wed, 14 Apr 2021 16:12:08 +0000 (12:12 -0400)]
Explicitly test for Connection in dialect.has_table()
The :meth:`_engine.Dialect.has_table` method now raises an informative
exception if a non-Connection is passed to it, as this incorrect behavior
seems to be common. This method is not intended for external use outside
of a dialect. Please use the :meth:`.Inspector.has_table` method
or for cross-compatibility with older SQLAlchemy versions, the
:meth:`_engine.Engine.has_table` method.
Mike Bayer [Tue, 13 Apr 2021 03:48:02 +0000 (23:48 -0400)]
Disable raiseload in immediateload strategy
Fixed issue in the new feature of :meth:`_orm.Session.refresh` introduced
by :ticket:`1763` where eagerly loaded relationships are also refreshed,
where the ``lazy="raise"`` and ``lazy="raise_on_sql"`` loader strategies
would interfere with the :func:`_orm.immediateload` loader strategy, thus
breaking the feature for relationships that were loaded with
:func:`_orm.selectinload`, :func:`_orm.subqueryload` as well.
Also update some docs re: refresh, populate existing, etc.
Mike Bayer [Mon, 12 Apr 2021 18:33:50 +0000 (14:33 -0400)]
Ensure bindparam key escaping applied in all cases
Fixed regression where the :class:`_sql.BindParameter` object would not
properly render for an IN expression (i.e. using the "post compile" feature
in 1.4) if the object were copied from either an internal cloning
operation, or from a pickle operation, and the parameter name contained
spaces or other special characters.
Mike Bayer [Sun, 11 Apr 2021 21:15:55 +0000 (17:15 -0400)]
Return Row for CursorResult.inserted_primary_key
The tuple returned by :attr:`.CursorResult.inserted_primary_key` is now a
:class:`_result.Row` object with a named tuple interface on top of the
existing tuple interface.
Mike Bayer [Fri, 9 Apr 2021 15:09:16 +0000 (11:09 -0400)]
Dont return outer transaction for _subtrans flag
Fixed critical regression where the :class:`_orm.Session` could fail to
"autobegin" a new transaction when a flush occurred without an existing
transaction in place, implicitly placing the :class:`_orm.Session` into
legacy autocommit mode which commit the transaction. The
:class:`_orm.Session` now has a check that will prevent this condition from
occurring, in addition to repairing the flush issue.
Additionally, scaled back part of the change made as part of :ticket:`5226`
which can run autoflush during an unexpire operation, to not actually
do this in the case of a :class:`_orm.Session` using legacy
:paramref:`_orm.Session.autocommit` mode, as this incurs a commit within
a refresh operation.
Mike Bayer [Fri, 9 Apr 2021 14:46:21 +0000 (10:46 -0400)]
Apply recursive check to immediateloader and generalize
Fixed critical regression caused by the new feature added as part of
:ticket:`1763`, eager loaders are invoked on unexpire operations. The new
feature makes use of the "immediateload" eager loader strategy as a
substitute for a collection loading strategy, which unlike the other
"post-load" strategies was not accommodating for recursive invocations
between mutually-dependent relationships, leading to recursion overflow
errors.
Add ``supports_schema = True`` to DefaultDialect and modify
requirements.py to use that attribute so third-party dialects
can explicitly indicate that they do *not* support schemas by
specifying ``supports_schema = False`` in their Dialect class.
Mike Bayer [Fri, 9 Apr 2021 13:25:04 +0000 (09:25 -0400)]
document scalar_subquery() with column_property()
Other doc cleanups and fixes as well, including more
explicitness that declarative attribute assignment only
works when a declarative base is involved; other forms
need to call add_property() directly.
Mike Bayer [Wed, 7 Apr 2021 17:57:39 +0000 (13:57 -0400)]
Fix LegacyRow/Row index access
Fixed up the behavior of the :class:`_result.Row` object when dictionary
access is used upon it, meaning converting to a dict via ``dict(row)`` or
accessing members using strings or other objects i.e. ``row["some_key"]``
works as it would with a dictionary, rather than raising ``TypeError`` as
would be the case with a tuple, whether or not the C extensions are in
place. This was originally supposed to emit a 2.0 deprecation warning for
the "non-future" case using :class:`_result.LegacyRow`, and was to raise
``TypeError`` for the "future" :class:`_result.Row` class. However, the C
version of :class:`_result.Row` was failing to raise this ``TypeError``,
and to complicate matters, the :meth:`_orm.Session.execute` method now
returns :class:`_result.Row` in all cases to maintain consistency with the
ORM result case, so users who didn't have C extensions installed would
see different behavior in this one case for existing pre-1.4 style
code.
Therefore, in order to soften the overall upgrade scheme as most users have
not been exposed to the more strict behavior of :class:`_result.Row` up
through 1.4.6, :class:`_result.LegacyRow` and :class:`_result.Row` both
provide for string-key access as well as support for ``dict(row)``, in all
cases emitting the 2.0 deprecation warning when ``SQLALCHEMY_WARN_20`` is
enabled. The :class:`_result.Row` object still uses tuple-like behavior for
``__contains__``, which is probably the only noticeable behavioral change
compared to :class:`_result.LegacyRow`, other than the removal of
dictionary-style methods ``values()`` and ``items()``.
Also remove filters for result set warnings.
callcounts updated for 2.7/ 3.9, am pushing jenkins to use python 3.9
now
Support `TypeDecorator` subclasses in `Column()` declarations
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Currently, the plugin resolves `TypeDecorator` subclasses in `Column()` declarations to `Mapping[_T]` instead of the correct type in the class declaration.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Thu, 8 Apr 2021 01:43:17 +0000 (21:43 -0400)]
Infer types in BindParameter when expanding=True
Enhanced the "expanding" feature used for :meth:`_sql.ColumnOperators.in_`
operations to infer the type of expression from the right hand list of
elements, if the left hand side does not have any explicit type set up.
This allows the expression to support stringification among other things.
In 1.3, "expanding" was not automatically used for
:meth:`_sql.ColumnOperators.in_` expressions, so in that sense this change
fixes a behavioral regression.
Mike Bayer [Wed, 7 Apr 2021 23:22:52 +0000 (19:22 -0400)]
convert subqueryload paths for multilevel
Fixed regression where the :func:`_orm.subqueryload` loader strategy would
fail to correctly accommodate sub-options, such as a :func:`_orm.defer`
option on a column, if the "path" of the subqueryload were more than one
level deep.
Mike Bayer [Wed, 7 Apr 2021 21:51:34 +0000 (17:51 -0400)]
support multivalues insert on strsqlcompiler
Fixed the "stringify" compiler to support a basic stringification
of a "multirow" INSERT statement, i.e. one with multiple tuples
following the VALUES keyword.
Mike Bayer [Wed, 7 Apr 2021 14:56:12 +0000 (10:56 -0400)]
Expand reg for schema translate map for most special characters
Fixed regression where usage of a token in the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
dictionary which contained special characters such as braces would fail to
be substituted properly. Use of square bracket characters ``[]`` is now
explicitly disallowed as these are used as a delimiter character in the
current implementation.
Mike Bayer [Wed, 7 Apr 2021 14:40:14 +0000 (10:40 -0400)]
Add test support for merge_frozen_result
Fixed regression where the :func:`_orm.merge_frozen_result` function relied
upon by the dogpile.caching example was not included in tests and began
failing due to incorrect internal arguments.
Mike Bayer [Wed, 7 Apr 2021 14:26:31 +0000 (10:26 -0400)]
Check for hybrid's attribute name and support no name
Fixed regression where the ORM compilation scheme would assume the function
name of a hybrid property would be the same as the attribute name in such a
way that an ``AttributeError`` would be raised, when it would attempt to
determine the correct name for each element in a result tuple. A similar
issue exists in 1.3 but only impacts the names of tuple rows. The fix here
adds a check that the hybrid's function name is actually present in the
``__dict__`` of the class or its superclasses before assigning this name;
otherwise, the hybrid is considered to be "unnamed" and ORM result tuples
will use the naming scheme of the underlying expression.
Table arguments name and metadata are positional only
The :class:`_sql.Table` object now raises an informative error message if
it is instantiated without passing at least the :paramref:`_sql.Table.name`
and :paramref:`_sql.Table.metadata` arguments positionally. Previously, if
these were passed as keyword arguments, the object would silently fail to
initialize correctly.
Mike Bayer [Tue, 6 Apr 2021 20:47:00 +0000 (16:47 -0400)]
Disable and disallow Result.unique() with yield_per
Fixed critical regression where the :meth:`_orm.Query.yield_per` method in
the ORM would set up the internal :class:`_engine.Result` to yield chunks
at a time, however made use of the new :meth:`_engine.Result.unique` method
which uniques across the entire result. This would lead to lost rows since
the ORM is using ``id(obj)`` as the uniquing function, which leads to
repeated identifiers for new objects as already-seen objects are garbage
collected. 1.3's behavior here was to "unique" across each chunk, which
does not actually produce "uniqued" results when results are yielded in
chunks. As the :meth:`_orm.Query.yield_per` method is already explicitly
disallowed when joined eager loading is in place, which is the primary
rationale for the "uniquing" feature, the "uniquing" feature is now turned
off entirely when :meth:`_orm.Query.yield_per` is used.
This regression only applies to the legacy :class:`_orm.Query` object; when
using :term:`2.0 style` execution, "uniquing" is not automatically applied.
To prevent the issue from arising from explicit use of
:meth:`_engine.Result.unique`, an error is now raised if rows are fetched
from a "uniqued" ORM-level :class:`_engine.Result` if any
:ref:`yield per <orm_queryguide_yield_per>` API is also in use, as the
purpose of ``yield_per`` is to allow for arbitrarily large numbers of rows,
which cannot be uniqued in memory without growing the number of entries to
fit the complete result size.
Mike Bayer [Tue, 6 Apr 2021 15:12:49 +0000 (11:12 -0400)]
Accommodate for callable fns for collection_class
Fixed issue where the Mypy plugin would fail to interpret the
"collection_class" of a relationship if it were a callable and not a class.
Also improved type matching and error reporting for collection-oriented
relationships.
Mike Bayer [Tue, 6 Apr 2021 02:14:18 +0000 (22:14 -0400)]
Disallow AliasedReturnsRows from execution
Executing a :class:`_sql.Subquery` using :meth:`_engine.Connection.execute`
is deprecated and will emit a deprecation warning; this use case was an
oversight that should have been removed from 1.4. The operation will now
execute the underlying :class:`_sql.Select` object directly for backwards
compatibility. Similarly, the :class:`_sql.CTE` class is also not
appropriate for execution. In 1.3, attempting to execute a CTE would result
in an invalid "blank" SQL statement being executed; since this use case was
not working it now raises :class:`_exc.ObjectNotExecutableError`.
Previously, 1.4 was attempting to execute the CTE as a statement however it
was working only erratically.
The change also breaks out StatementRole from ReturnsRowsRole, as these
roles should not be in the same lineage (some statements don't return
rows, the whole class of ReturnsRows that are from clauses are
not statements). Consolidate StatementRole and
CoerceTextStatementRole as there's no usage difference between
these. Simplify some old tests that were trying to make
sure that "execution options" didn't transmit from a cte/subquery
out to a select; as cte/subuqery() aren't executable in any case
the options are removed.
Mike Bayer [Mon, 5 Apr 2021 18:41:31 +0000 (14:41 -0400)]
Add pgcode / sqlstate for asyncpg error message
Added accessors ``.sqlstate`` and synonym ``.pgcode`` to the ``.orig``
attribute of the SQLAlchemy exception class raised by the asyncpg DBAPI
adapter, that is, the intermediary exception object that wraps on top of
that raised by the asyncpg library itself, but below the level of the
SQLAlchemy dialect.
Mike Bayer [Mon, 5 Apr 2021 20:32:14 +0000 (16:32 -0400)]
uniquify when popping literal_execute_params from param dict
Fixed further issues in the same area as that of :ticket:`6173` released in
1.4.5, where a "postcompile" parameter, again most typically those used for
LIMIT/OFFSET rendering in Oracle and SQL Server, would fail to be processed
correctly if the same parameter rendered in multiple places in the
statement.
Mike Bayer [Mon, 5 Apr 2021 21:37:56 +0000 (17:37 -0400)]
Detect (Entity, Entity) vs (Entity, onclause) in legacy join
Fixed regression where a deprecated form of :meth:`_orm.Query.join` were
used, passing a series of entities to join from without any ON clause in a
single :meth:`_orm.Query.join` call, would fail to function correctly.
Mike Bayer [Fri, 26 Mar 2021 23:45:29 +0000 (19:45 -0400)]
Adjust for mypy incremental behaviors
Applied a series of refactorings and fixes to accommodate for Mypy
"incremental" mode across multiple files, which previously was not taken
into account. In this mode the Mypy plugin has to accommodate Python
datatypes expressed in other files coming in with less information than
they have on a direct run.
Additionally, a new decorator :func:`_orm.declarative_mixin` is added,
which is necessary for the Mypy plugin to be able to definifitely identify
a Declarative mixin class that is otherwise not used inside a particular
Python file.
discussion:
With incremental / deserialized mypy runs, it appears
that when we look at a base class that comes from another file,
cls.info is set to a special undefined node
that matches CLASSDEF_NO_INFO, and we otherwise can't
touch it without crashing. Additionally, sometimes cls.defs.body
is present but empty.
However, it appears that both of these cases can be sidestepped,
first by doing a lookup() for the type name where we
get a SymbolTableNode that then has the TypeInfo we wanted
when we tried touching cls.info, and then however we got the
TypeInfo, if cls.defs.body is empty we can just look in the
names to get at the symbols for that class; we just can't
access AssignmentStmts, but that's fine because we just
need the information for classes we aren't actually type checking.
This work also revealed there's no easy way to detect a mixin
class so we just create a new decorator to mark that. will make
code look better in any case.
Mike Bayer [Sun, 4 Apr 2021 04:41:28 +0000 (00:41 -0400)]
Document inserted_primary_key_rows
This accessor was misleading in that it indicated a general
capability to return inserted primary key values for multiple
rows at once. Clarify this is not currently the case
as the feature is only suppported by the psycopg2 dialect
at the moment.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
