Mike Bayer [Wed, 7 Apr 2021 21:51:34 +0000 (17:51 -0400)]
support multivalues insert on strsqlcompiler
Fixed the "stringify" compiler to support a basic stringification
of a "multirow" INSERT statement, i.e. one with multiple tuples
following the VALUES keyword.
Mike Bayer [Wed, 7 Apr 2021 14:56:12 +0000 (10:56 -0400)]
Expand reg for schema translate map for most special characters
Fixed regression where usage of a token in the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
dictionary which contained special characters such as braces would fail to
be substituted properly. Use of square bracket characters ``[]`` is now
explicitly disallowed as these are used as a delimiter character in the
current implementation.
Mike Bayer [Wed, 7 Apr 2021 14:40:14 +0000 (10:40 -0400)]
Add test support for merge_frozen_result
Fixed regression where the :func:`_orm.merge_frozen_result` function relied
upon by the dogpile.caching example was not included in tests and began
failing due to incorrect internal arguments.
Mike Bayer [Wed, 7 Apr 2021 14:26:31 +0000 (10:26 -0400)]
Check for hybrid's attribute name and support no name
Fixed regression where the ORM compilation scheme would assume the function
name of a hybrid property would be the same as the attribute name in such a
way that an ``AttributeError`` would be raised, when it would attempt to
determine the correct name for each element in a result tuple. A similar
issue exists in 1.3 but only impacts the names of tuple rows. The fix here
adds a check that the hybrid's function name is actually present in the
``__dict__`` of the class or its superclasses before assigning this name;
otherwise, the hybrid is considered to be "unnamed" and ORM result tuples
will use the naming scheme of the underlying expression.
Table arguments name and metadata are positional only
The :class:`_sql.Table` object now raises an informative error message if
it is instantiated without passing at least the :paramref:`_sql.Table.name`
and :paramref:`_sql.Table.metadata` arguments positionally. Previously, if
these were passed as keyword arguments, the object would silently fail to
initialize correctly.
Mike Bayer [Tue, 6 Apr 2021 20:47:00 +0000 (16:47 -0400)]
Disable and disallow Result.unique() with yield_per
Fixed critical regression where the :meth:`_orm.Query.yield_per` method in
the ORM would set up the internal :class:`_engine.Result` to yield chunks
at a time, however made use of the new :meth:`_engine.Result.unique` method
which uniques across the entire result. This would lead to lost rows since
the ORM is using ``id(obj)`` as the uniquing function, which leads to
repeated identifiers for new objects as already-seen objects are garbage
collected. 1.3's behavior here was to "unique" across each chunk, which
does not actually produce "uniqued" results when results are yielded in
chunks. As the :meth:`_orm.Query.yield_per` method is already explicitly
disallowed when joined eager loading is in place, which is the primary
rationale for the "uniquing" feature, the "uniquing" feature is now turned
off entirely when :meth:`_orm.Query.yield_per` is used.
This regression only applies to the legacy :class:`_orm.Query` object; when
using :term:`2.0 style` execution, "uniquing" is not automatically applied.
To prevent the issue from arising from explicit use of
:meth:`_engine.Result.unique`, an error is now raised if rows are fetched
from a "uniqued" ORM-level :class:`_engine.Result` if any
:ref:`yield per <orm_queryguide_yield_per>` API is also in use, as the
purpose of ``yield_per`` is to allow for arbitrarily large numbers of rows,
which cannot be uniqued in memory without growing the number of entries to
fit the complete result size.
Mike Bayer [Tue, 6 Apr 2021 15:12:49 +0000 (11:12 -0400)]
Accommodate for callable fns for collection_class
Fixed issue where the Mypy plugin would fail to interpret the
"collection_class" of a relationship if it were a callable and not a class.
Also improved type matching and error reporting for collection-oriented
relationships.
Mike Bayer [Tue, 6 Apr 2021 02:14:18 +0000 (22:14 -0400)]
Disallow AliasedReturnsRows from execution
Executing a :class:`_sql.Subquery` using :meth:`_engine.Connection.execute`
is deprecated and will emit a deprecation warning; this use case was an
oversight that should have been removed from 1.4. The operation will now
execute the underlying :class:`_sql.Select` object directly for backwards
compatibility. Similarly, the :class:`_sql.CTE` class is also not
appropriate for execution. In 1.3, attempting to execute a CTE would result
in an invalid "blank" SQL statement being executed; since this use case was
not working it now raises :class:`_exc.ObjectNotExecutableError`.
Previously, 1.4 was attempting to execute the CTE as a statement however it
was working only erratically.
The change also breaks out StatementRole from ReturnsRowsRole, as these
roles should not be in the same lineage (some statements don't return
rows, the whole class of ReturnsRows that are from clauses are
not statements). Consolidate StatementRole and
CoerceTextStatementRole as there's no usage difference between
these. Simplify some old tests that were trying to make
sure that "execution options" didn't transmit from a cte/subquery
out to a select; as cte/subuqery() aren't executable in any case
the options are removed.
Mike Bayer [Mon, 5 Apr 2021 18:41:31 +0000 (14:41 -0400)]
Add pgcode / sqlstate for asyncpg error message
Added accessors ``.sqlstate`` and synonym ``.pgcode`` to the ``.orig``
attribute of the SQLAlchemy exception class raised by the asyncpg DBAPI
adapter, that is, the intermediary exception object that wraps on top of
that raised by the asyncpg library itself, but below the level of the
SQLAlchemy dialect.
Mike Bayer [Mon, 5 Apr 2021 20:32:14 +0000 (16:32 -0400)]
uniquify when popping literal_execute_params from param dict
Fixed further issues in the same area as that of :ticket:`6173` released in
1.4.5, where a "postcompile" parameter, again most typically those used for
LIMIT/OFFSET rendering in Oracle and SQL Server, would fail to be processed
correctly if the same parameter rendered in multiple places in the
statement.
Mike Bayer [Mon, 5 Apr 2021 21:37:56 +0000 (17:37 -0400)]
Detect (Entity, Entity) vs (Entity, onclause) in legacy join
Fixed regression where a deprecated form of :meth:`_orm.Query.join` were
used, passing a series of entities to join from without any ON clause in a
single :meth:`_orm.Query.join` call, would fail to function correctly.
Mike Bayer [Fri, 26 Mar 2021 23:45:29 +0000 (19:45 -0400)]
Adjust for mypy incremental behaviors
Applied a series of refactorings and fixes to accommodate for Mypy
"incremental" mode across multiple files, which previously was not taken
into account. In this mode the Mypy plugin has to accommodate Python
datatypes expressed in other files coming in with less information than
they have on a direct run.
Additionally, a new decorator :func:`_orm.declarative_mixin` is added,
which is necessary for the Mypy plugin to be able to definifitely identify
a Declarative mixin class that is otherwise not used inside a particular
Python file.
discussion:
With incremental / deserialized mypy runs, it appears
that when we look at a base class that comes from another file,
cls.info is set to a special undefined node
that matches CLASSDEF_NO_INFO, and we otherwise can't
touch it without crashing. Additionally, sometimes cls.defs.body
is present but empty.
However, it appears that both of these cases can be sidestepped,
first by doing a lookup() for the type name where we
get a SymbolTableNode that then has the TypeInfo we wanted
when we tried touching cls.info, and then however we got the
TypeInfo, if cls.defs.body is empty we can just look in the
names to get at the symbols for that class; we just can't
access AssignmentStmts, but that's fine because we just
need the information for classes we aren't actually type checking.
This work also revealed there's no easy way to detect a mixin
class so we just create a new decorator to mark that. will make
code look better in any case.
Mike Bayer [Sun, 4 Apr 2021 04:41:28 +0000 (00:41 -0400)]
Document inserted_primary_key_rows
This accessor was misleading in that it indicated a general
capability to return inserted primary key values for multiple
rows at once. Clarify this is not currently the case
as the feature is only suppported by the psycopg2 dialect
at the moment.
Mike Bayer [Fri, 2 Apr 2021 15:06:28 +0000 (11:06 -0400)]
Ensure Grouping._with_binary_element_type() maintains class
Fixed regression where use of the :meth:`.Operators.in_` method with a
:class:`_sql.Select` object against a non-table-bound column would produce
an ``AttributeError``, or more generally using a :class:`_sql.ScalarSelect`
that has no datatype in a binary expression would produce invalid state.
Mike Bayer [Thu, 1 Apr 2021 21:20:31 +0000 (17:20 -0400)]
Default caching to opt-out for 3rd party dialects
Added a new flag to the :class:`_engine.Dialect` class called
:attr:`_engine.Dialect.supports_statement_cache`. This flag now needs to be present
directly on a dialect class in order for SQLAlchemy's
:ref:`query cache <sql_caching>` to take effect for that dialect. The
rationale is based on discovered issues such as :ticket:`6173` revealing
that dialects which hardcode literal values from the compiled statement,
often the numerical parameters used for LIMIT / OFFSET, will not be
compatible with caching until these dialects are revised to use the
parameters present in the statement only. For third party dialects where
this flag is not applied, the SQL logging will show the message "dialect
does not support caching", indicating the dialect should seek to apply this
flag once they have verified that no per-statement literal values are being
rendered within the compilation phase.
Federico Caselli [Mon, 29 Mar 2021 20:30:49 +0000 (22:30 +0200)]
Add ``omit_aliases`` in ``Enum``.
Introduce a new parameter :paramref:`_types.Enum.omit_aliases` in
:class:`_types.Enum` type allow filtering aliases when using a pep435 Enum.
Previous versions of SQLAlchemy kept aliases in all cases, creating
database enum type with additional states, meaning that they were treated
as different values in the db. For backward compatibility this flag
defaults to ``False`` in the 1.4 series, but will be switched to ``True``
in a future version. A deprecation warning is raise if this flag is not
specified and the passed enum contains aliases.
Mike Bayer [Thu, 1 Apr 2021 16:46:41 +0000 (12:46 -0400)]
Apply quoting to render_derived() names
Fixed bug in new :meth:`_functions.FunctionElement.render_derived` feature
where column names rendered out explicitly in the alias SQL would not have
proper quoting applied for case sensitive names and other non-alphanumeric
names.
Mike Bayer [Thu, 1 Apr 2021 16:26:06 +0000 (12:26 -0400)]
Correct for Variant + ARRAY cases in psycopg2
Fixed regression caused by :ticket:`6023` where the PostgreSQL cast
operator applied to elements within an :class:`_types.ARRAY` when using
psycopg2 would fail to use the correct type in the case that the datatype
were also embedded within an instance of the :class:`_types.Variant`
adapter.
Additionally, repairs support for the correct CREATE TYPE to be emitted
when using a ``Variant(ARRAY(some_schema_type))``.
Federico Caselli [Tue, 30 Mar 2021 21:15:04 +0000 (23:15 +0200)]
Fix MSSQL / Oracle limit/offset regressions
Fixed a regression in MSSQL 2012+ that prevented the order clause
to be rendered when ``offset=0`` is used in a subquery.
Fixed critical regression where the Oracle compiler would not maintain the
correct parameter values in the LIMIT/OFFSET for a select due to a caching
issue.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #6163 Fixes: #6173
Change-Id: Ieb12354271d09ad935d684ee0db4fa0128837215
Mike Bayer [Wed, 31 Mar 2021 23:38:10 +0000 (19:38 -0400)]
Correct for CTE correspondence w/ aliased CTE
Fixed regression where the :func:`_orm.joinedload` loader strategy would
not successfully joinedload to a mapper that is mapper against a
:class:`.CTE` construct.
Mike Bayer [Sun, 28 Mar 2021 15:09:40 +0000 (11:09 -0400)]
Add DeclarativeMeta to globals
Fixed issue in mypy plugin where newly added support for
:func:`_orm.as_declarative` needed to more fully add the
``DeclarativeMeta`` class to the mypy interpreter's state so that it does
not result in a name not found error; additionally improves how global
names are setup for the plugin including the ``Mapped`` name.
Introduces directory oriented testing as well, where a full
set of files will be copied, mypy runs, then zero or more patches
are applied and mypy is run again, to fully test incremental
behaviors.
Mike Bayer [Wed, 31 Mar 2021 17:16:04 +0000 (13:16 -0400)]
Expand sibling tests for overlaps warning
Scaled back the warning message added in :ticket:`5171` to not warn for
overlapping columns in an inheritance scenario where a particular
relationship is local to a subclass and therefore does not represent an
overlap.
Add errors documentation for the warning and also expand
``util.warn()`` to include a code parameter.
Mike Bayer [Wed, 31 Mar 2021 15:48:22 +0000 (11:48 -0400)]
Repair PGInspector
Fixed issue where the PostgreSQL :class:`.PGInspector`, when generated
against an :class:`_engine.Engine`, would fail for ``.get_enums()``,
``.get_view_names()``, ``.get_foreign_table_names()`` and
``.get_table_oid()`` when used against a "future" style engine and not the
connection directly.
Mike Bayer [Wed, 31 Mar 2021 00:30:13 +0000 (20:30 -0400)]
asyncio API can be considered beta level
It's clear that there are at least many dozens of developers
working with the new asyncio extension and we have not had
any major bug reports. Other than ``Session.delete`` needing
to be an awaitable, there were no backwards-incompatible issues
and we've seen no issues with wrong behaviors.
hbusul [Mon, 29 Mar 2021 15:42:33 +0000 (11:42 -0400)]
accommodate new pg8000 disconnection exception
Modified the ``is_disconnect()`` handler for the pg8000 dialect, which now
accommodates for a new ``InterfaceError`` emitted by pg8000 1.19.0. Pull
request courtesy Hamdi Burak Usul.
Mike Bayer [Mon, 29 Mar 2021 21:57:20 +0000 (17:57 -0400)]
TypeDecorator passes "outer" flag to itself for set_parent accounting
Fixed bug first introduced in as some combination of :ticket:`2892`,
:ticket:`2919` nnd :ticket:`3832` where the attachment events for a
:class:`_types.TypeDecorator` would be doubled up against the "impl" class,
if the "impl" were also a :class:`_types.SchemaType`. The real-world case
is any :class:`_types.TypeDecorator` against :class:`_types.Enum` or
:class:`_types.Boolean` would get a doubled
:class:`_schema.CheckConstraint` when the ``create_constraint=True`` flag
is set.
Mike Bayer [Mon, 29 Mar 2021 20:40:16 +0000 (16:40 -0400)]
Run trans.close() at end of block if transaction already inactive
Modified the context manager used by :class:`_engine.Transaction` so that
an "already detached" warning is not emitted by the ending of the context
manager itself, if the transaction were already manually rolled back inside
the block. This applies to regular transactions, savepoint transactions,
and legacy "marker" transactions. A warning is still emitted if the
``.rollback()`` method is called explicitly more than once.
Mike Bayer [Tue, 30 Mar 2021 02:38:09 +0000 (22:38 -0400)]
Adapt for importlib_metadata changes
importlib_metadata deprecates the .get() method and provides
a different method .select() as of 3.7.1. Work around
the deprecation warning in this case.
Mike Bayer [Sat, 27 Mar 2021 12:36:32 +0000 (08:36 -0400)]
Commentary; run criteria.params() if statement isn't cached?
Considering adjustment to 56f9c7743e9083add69a10501a503f,
if statement is not cached, skip the relatively expensive
step of re-processing the criteria clause.
However, this causes the overall cache key of the statement
to come out differently which should also be avoided.
Likely we would not merge the actual change, just the
comment here.
Mike Bayer [Fri, 26 Mar 2021 19:40:34 +0000 (15:40 -0400)]
run new relationship criteria tests on empty session
The commit in 56f9c7743e9083add69a10501a503f4 forgot to make
sure the full reload took place, either via populate_existing()
or session.close(), with failure only showing up in pypy.
Mike Bayer [Fri, 26 Mar 2021 14:37:21 +0000 (10:37 -0400)]
Adapt loader_criteria params for current query
Fixed critical issue in the new :meth:`_orm.PropComparator.and_` feature
where loader strategies that emit secondary SELECT statements such as
:func:`_orm.selectinload` and :func:`_orm.lazyload` would fail to
accommodate for bound parameters in the user-defined criteria in terms of
the current statement being executed, as opposed to the cached statement,
causing stale bound values to be used.
This also adds a warning for the case where an object that uses
:func:`_orm.lazyload` in conjunction with :meth:`_orm.PropComparator.and_`
is attempted to be serialized; the loader criteria cannot reliably
be serialized and deserialized and eager loading should be used for this
case.
Mike Bayer [Tue, 23 Mar 2021 22:49:28 +0000 (18:49 -0400)]
support as_declarative, as_declarative_base
Added support for the Mypy extension to correctly interpret a declarative
base class that's generated using the :func:`_orm.as_declarative` function
as well as the :meth:`_orm.registry.as_declarative_base` method.
Mike Bayer [Thu, 25 Mar 2021 14:37:12 +0000 (10:37 -0400)]
don't rely on pk-only group_by
Fixed the test test_aliased_stmt_includes_unnamed_fn
recently added in c1c999c01d to include the additional
column in the GROUP BY statement, allowing this statement
to succeed on SQL Server which seems to not support "group by pk only"
syntax.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
