Stephen Rosen [Thu, 15 Apr 2021 18:35:03 +0000 (18:35 +0000)]
Move custom 2.0 deprecation warning message
Rather than appending to the message via the deprecation utilities,
the RemovedIn20Warning class has a custom __str__ which appends the
desired info to its message.
Stephen Rosen [Thu, 15 Apr 2021 16:49:17 +0000 (16:49 +0000)]
Update docs and HasDescriptionCode per review
Rename the mixin class and make a few changes to the docs. Notably,
put the '_error_s9r1' ReST ref in the errors doc. This requires a new
(small) section there to make sense.
Stephen Rosen [Mon, 12 Apr 2021 17:57:17 +0000 (17:57 +0000)]
Add support for 'code' in deprecation warnings
SADeprecationWarning and SQLAlchemyError now share 'code' and
'_code_str' through a small mixin/helper class.
In the case of the deprecation warnings, the code_str is injected into
`__str__`, which means the message will be built when a warning is
emitted. An alternative approach would be to build a distinct message
when the warning is instantiated.
`s9r1` is a new error code which points directly to the documentation
on backref cascades, and this is now in place as the code for
`cascade_backrefs` warnings.
Stephen Rosen [Fri, 9 Apr 2021 23:53:06 +0000 (23:53 +0000)]
Document backref() for setting cascade_backrefs
Add a section to the documentation for the cascade_backrefs
parameter to `relationship()` which explains that `cascade_backrefs`
can also be used as a parameter to the `backref()` function when a
relationship is being specified using `relationship.backref`.
Mike Bayer [Fri, 9 Apr 2021 15:09:16 +0000 (11:09 -0400)]
Dont return outer transaction for _subtrans flag
Fixed critical regression where the :class:`_orm.Session` could fail to
"autobegin" a new transaction when a flush occurred without an existing
transaction in place, implicitly placing the :class:`_orm.Session` into
legacy autocommit mode which commit the transaction. The
:class:`_orm.Session` now has a check that will prevent this condition from
occurring, in addition to repairing the flush issue.
Additionally, scaled back part of the change made as part of :ticket:`5226`
which can run autoflush during an unexpire operation, to not actually
do this in the case of a :class:`_orm.Session` using legacy
:paramref:`_orm.Session.autocommit` mode, as this incurs a commit within
a refresh operation.
Mike Bayer [Fri, 9 Apr 2021 14:46:21 +0000 (10:46 -0400)]
Apply recursive check to immediateloader and generalize
Fixed critical regression caused by the new feature added as part of
:ticket:`1763`, eager loaders are invoked on unexpire operations. The new
feature makes use of the "immediateload" eager loader strategy as a
substitute for a collection loading strategy, which unlike the other
"post-load" strategies was not accommodating for recursive invocations
between mutually-dependent relationships, leading to recursion overflow
errors.
Add ``supports_schema = True`` to DefaultDialect and modify
requirements.py to use that attribute so third-party dialects
can explicitly indicate that they do *not* support schemas by
specifying ``supports_schema = False`` in their Dialect class.
Mike Bayer [Fri, 9 Apr 2021 13:25:04 +0000 (09:25 -0400)]
document scalar_subquery() with column_property()
Other doc cleanups and fixes as well, including more
explicitness that declarative attribute assignment only
works when a declarative base is involved; other forms
need to call add_property() directly.
Mike Bayer [Wed, 7 Apr 2021 17:57:39 +0000 (13:57 -0400)]
Fix LegacyRow/Row index access
Fixed up the behavior of the :class:`_result.Row` object when dictionary
access is used upon it, meaning converting to a dict via ``dict(row)`` or
accessing members using strings or other objects i.e. ``row["some_key"]``
works as it would with a dictionary, rather than raising ``TypeError`` as
would be the case with a tuple, whether or not the C extensions are in
place. This was originally supposed to emit a 2.0 deprecation warning for
the "non-future" case using :class:`_result.LegacyRow`, and was to raise
``TypeError`` for the "future" :class:`_result.Row` class. However, the C
version of :class:`_result.Row` was failing to raise this ``TypeError``,
and to complicate matters, the :meth:`_orm.Session.execute` method now
returns :class:`_result.Row` in all cases to maintain consistency with the
ORM result case, so users who didn't have C extensions installed would
see different behavior in this one case for existing pre-1.4 style
code.
Therefore, in order to soften the overall upgrade scheme as most users have
not been exposed to the more strict behavior of :class:`_result.Row` up
through 1.4.6, :class:`_result.LegacyRow` and :class:`_result.Row` both
provide for string-key access as well as support for ``dict(row)``, in all
cases emitting the 2.0 deprecation warning when ``SQLALCHEMY_WARN_20`` is
enabled. The :class:`_result.Row` object still uses tuple-like behavior for
``__contains__``, which is probably the only noticeable behavioral change
compared to :class:`_result.LegacyRow`, other than the removal of
dictionary-style methods ``values()`` and ``items()``.
Also remove filters for result set warnings.
callcounts updated for 2.7/ 3.9, am pushing jenkins to use python 3.9
now
Support `TypeDecorator` subclasses in `Column()` declarations
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Currently, the plugin resolves `TypeDecorator` subclasses in `Column()` declarations to `Mapping[_T]` instead of the correct type in the class declaration.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Thu, 8 Apr 2021 01:43:17 +0000 (21:43 -0400)]
Infer types in BindParameter when expanding=True
Enhanced the "expanding" feature used for :meth:`_sql.ColumnOperators.in_`
operations to infer the type of expression from the right hand list of
elements, if the left hand side does not have any explicit type set up.
This allows the expression to support stringification among other things.
In 1.3, "expanding" was not automatically used for
:meth:`_sql.ColumnOperators.in_` expressions, so in that sense this change
fixes a behavioral regression.
Mike Bayer [Wed, 7 Apr 2021 23:22:52 +0000 (19:22 -0400)]
convert subqueryload paths for multilevel
Fixed regression where the :func:`_orm.subqueryload` loader strategy would
fail to correctly accommodate sub-options, such as a :func:`_orm.defer`
option on a column, if the "path" of the subqueryload were more than one
level deep.
Mike Bayer [Wed, 7 Apr 2021 21:51:34 +0000 (17:51 -0400)]
support multivalues insert on strsqlcompiler
Fixed the "stringify" compiler to support a basic stringification
of a "multirow" INSERT statement, i.e. one with multiple tuples
following the VALUES keyword.
Mike Bayer [Wed, 7 Apr 2021 14:56:12 +0000 (10:56 -0400)]
Expand reg for schema translate map for most special characters
Fixed regression where usage of a token in the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
dictionary which contained special characters such as braces would fail to
be substituted properly. Use of square bracket characters ``[]`` is now
explicitly disallowed as these are used as a delimiter character in the
current implementation.
Mike Bayer [Wed, 7 Apr 2021 14:40:14 +0000 (10:40 -0400)]
Add test support for merge_frozen_result
Fixed regression where the :func:`_orm.merge_frozen_result` function relied
upon by the dogpile.caching example was not included in tests and began
failing due to incorrect internal arguments.
Mike Bayer [Wed, 7 Apr 2021 14:26:31 +0000 (10:26 -0400)]
Check for hybrid's attribute name and support no name
Fixed regression where the ORM compilation scheme would assume the function
name of a hybrid property would be the same as the attribute name in such a
way that an ``AttributeError`` would be raised, when it would attempt to
determine the correct name for each element in a result tuple. A similar
issue exists in 1.3 but only impacts the names of tuple rows. The fix here
adds a check that the hybrid's function name is actually present in the
``__dict__`` of the class or its superclasses before assigning this name;
otherwise, the hybrid is considered to be "unnamed" and ORM result tuples
will use the naming scheme of the underlying expression.
Table arguments name and metadata are positional only
The :class:`_sql.Table` object now raises an informative error message if
it is instantiated without passing at least the :paramref:`_sql.Table.name`
and :paramref:`_sql.Table.metadata` arguments positionally. Previously, if
these were passed as keyword arguments, the object would silently fail to
initialize correctly.
Mike Bayer [Tue, 6 Apr 2021 20:47:00 +0000 (16:47 -0400)]
Disable and disallow Result.unique() with yield_per
Fixed critical regression where the :meth:`_orm.Query.yield_per` method in
the ORM would set up the internal :class:`_engine.Result` to yield chunks
at a time, however made use of the new :meth:`_engine.Result.unique` method
which uniques across the entire result. This would lead to lost rows since
the ORM is using ``id(obj)`` as the uniquing function, which leads to
repeated identifiers for new objects as already-seen objects are garbage
collected. 1.3's behavior here was to "unique" across each chunk, which
does not actually produce "uniqued" results when results are yielded in
chunks. As the :meth:`_orm.Query.yield_per` method is already explicitly
disallowed when joined eager loading is in place, which is the primary
rationale for the "uniquing" feature, the "uniquing" feature is now turned
off entirely when :meth:`_orm.Query.yield_per` is used.
This regression only applies to the legacy :class:`_orm.Query` object; when
using :term:`2.0 style` execution, "uniquing" is not automatically applied.
To prevent the issue from arising from explicit use of
:meth:`_engine.Result.unique`, an error is now raised if rows are fetched
from a "uniqued" ORM-level :class:`_engine.Result` if any
:ref:`yield per <orm_queryguide_yield_per>` API is also in use, as the
purpose of ``yield_per`` is to allow for arbitrarily large numbers of rows,
which cannot be uniqued in memory without growing the number of entries to
fit the complete result size.
Mike Bayer [Tue, 6 Apr 2021 15:12:49 +0000 (11:12 -0400)]
Accommodate for callable fns for collection_class
Fixed issue where the Mypy plugin would fail to interpret the
"collection_class" of a relationship if it were a callable and not a class.
Also improved type matching and error reporting for collection-oriented
relationships.
Mike Bayer [Tue, 6 Apr 2021 02:14:18 +0000 (22:14 -0400)]
Disallow AliasedReturnsRows from execution
Executing a :class:`_sql.Subquery` using :meth:`_engine.Connection.execute`
is deprecated and will emit a deprecation warning; this use case was an
oversight that should have been removed from 1.4. The operation will now
execute the underlying :class:`_sql.Select` object directly for backwards
compatibility. Similarly, the :class:`_sql.CTE` class is also not
appropriate for execution. In 1.3, attempting to execute a CTE would result
in an invalid "blank" SQL statement being executed; since this use case was
not working it now raises :class:`_exc.ObjectNotExecutableError`.
Previously, 1.4 was attempting to execute the CTE as a statement however it
was working only erratically.
The change also breaks out StatementRole from ReturnsRowsRole, as these
roles should not be in the same lineage (some statements don't return
rows, the whole class of ReturnsRows that are from clauses are
not statements). Consolidate StatementRole and
CoerceTextStatementRole as there's no usage difference between
these. Simplify some old tests that were trying to make
sure that "execution options" didn't transmit from a cte/subquery
out to a select; as cte/subuqery() aren't executable in any case
the options are removed.
Mike Bayer [Mon, 5 Apr 2021 18:41:31 +0000 (14:41 -0400)]
Add pgcode / sqlstate for asyncpg error message
Added accessors ``.sqlstate`` and synonym ``.pgcode`` to the ``.orig``
attribute of the SQLAlchemy exception class raised by the asyncpg DBAPI
adapter, that is, the intermediary exception object that wraps on top of
that raised by the asyncpg library itself, but below the level of the
SQLAlchemy dialect.
Mike Bayer [Mon, 5 Apr 2021 20:32:14 +0000 (16:32 -0400)]
uniquify when popping literal_execute_params from param dict
Fixed further issues in the same area as that of :ticket:`6173` released in
1.4.5, where a "postcompile" parameter, again most typically those used for
LIMIT/OFFSET rendering in Oracle and SQL Server, would fail to be processed
correctly if the same parameter rendered in multiple places in the
statement.
Mike Bayer [Mon, 5 Apr 2021 21:37:56 +0000 (17:37 -0400)]
Detect (Entity, Entity) vs (Entity, onclause) in legacy join
Fixed regression where a deprecated form of :meth:`_orm.Query.join` were
used, passing a series of entities to join from without any ON clause in a
single :meth:`_orm.Query.join` call, would fail to function correctly.
Mike Bayer [Fri, 26 Mar 2021 23:45:29 +0000 (19:45 -0400)]
Adjust for mypy incremental behaviors
Applied a series of refactorings and fixes to accommodate for Mypy
"incremental" mode across multiple files, which previously was not taken
into account. In this mode the Mypy plugin has to accommodate Python
datatypes expressed in other files coming in with less information than
they have on a direct run.
Additionally, a new decorator :func:`_orm.declarative_mixin` is added,
which is necessary for the Mypy plugin to be able to definifitely identify
a Declarative mixin class that is otherwise not used inside a particular
Python file.
discussion:
With incremental / deserialized mypy runs, it appears
that when we look at a base class that comes from another file,
cls.info is set to a special undefined node
that matches CLASSDEF_NO_INFO, and we otherwise can't
touch it without crashing. Additionally, sometimes cls.defs.body
is present but empty.
However, it appears that both of these cases can be sidestepped,
first by doing a lookup() for the type name where we
get a SymbolTableNode that then has the TypeInfo we wanted
when we tried touching cls.info, and then however we got the
TypeInfo, if cls.defs.body is empty we can just look in the
names to get at the symbols for that class; we just can't
access AssignmentStmts, but that's fine because we just
need the information for classes we aren't actually type checking.
This work also revealed there's no easy way to detect a mixin
class so we just create a new decorator to mark that. will make
code look better in any case.
Mike Bayer [Sun, 4 Apr 2021 04:41:28 +0000 (00:41 -0400)]
Document inserted_primary_key_rows
This accessor was misleading in that it indicated a general
capability to return inserted primary key values for multiple
rows at once. Clarify this is not currently the case
as the feature is only suppported by the psycopg2 dialect
at the moment.
Mike Bayer [Fri, 2 Apr 2021 15:06:28 +0000 (11:06 -0400)]
Ensure Grouping._with_binary_element_type() maintains class
Fixed regression where use of the :meth:`.Operators.in_` method with a
:class:`_sql.Select` object against a non-table-bound column would produce
an ``AttributeError``, or more generally using a :class:`_sql.ScalarSelect`
that has no datatype in a binary expression would produce invalid state.
Mike Bayer [Thu, 1 Apr 2021 21:20:31 +0000 (17:20 -0400)]
Default caching to opt-out for 3rd party dialects
Added a new flag to the :class:`_engine.Dialect` class called
:attr:`_engine.Dialect.supports_statement_cache`. This flag now needs to be present
directly on a dialect class in order for SQLAlchemy's
:ref:`query cache <sql_caching>` to take effect for that dialect. The
rationale is based on discovered issues such as :ticket:`6173` revealing
that dialects which hardcode literal values from the compiled statement,
often the numerical parameters used for LIMIT / OFFSET, will not be
compatible with caching until these dialects are revised to use the
parameters present in the statement only. For third party dialects where
this flag is not applied, the SQL logging will show the message "dialect
does not support caching", indicating the dialect should seek to apply this
flag once they have verified that no per-statement literal values are being
rendered within the compilation phase.
Federico Caselli [Mon, 29 Mar 2021 20:30:49 +0000 (22:30 +0200)]
Add ``omit_aliases`` in ``Enum``.
Introduce a new parameter :paramref:`_types.Enum.omit_aliases` in
:class:`_types.Enum` type allow filtering aliases when using a pep435 Enum.
Previous versions of SQLAlchemy kept aliases in all cases, creating
database enum type with additional states, meaning that they were treated
as different values in the db. For backward compatibility this flag
defaults to ``False`` in the 1.4 series, but will be switched to ``True``
in a future version. A deprecation warning is raise if this flag is not
specified and the passed enum contains aliases.
Mike Bayer [Thu, 1 Apr 2021 16:46:41 +0000 (12:46 -0400)]
Apply quoting to render_derived() names
Fixed bug in new :meth:`_functions.FunctionElement.render_derived` feature
where column names rendered out explicitly in the alias SQL would not have
proper quoting applied for case sensitive names and other non-alphanumeric
names.
Mike Bayer [Thu, 1 Apr 2021 16:26:06 +0000 (12:26 -0400)]
Correct for Variant + ARRAY cases in psycopg2
Fixed regression caused by :ticket:`6023` where the PostgreSQL cast
operator applied to elements within an :class:`_types.ARRAY` when using
psycopg2 would fail to use the correct type in the case that the datatype
were also embedded within an instance of the :class:`_types.Variant`
adapter.
Additionally, repairs support for the correct CREATE TYPE to be emitted
when using a ``Variant(ARRAY(some_schema_type))``.
Federico Caselli [Tue, 30 Mar 2021 21:15:04 +0000 (23:15 +0200)]
Fix MSSQL / Oracle limit/offset regressions
Fixed a regression in MSSQL 2012+ that prevented the order clause
to be rendered when ``offset=0`` is used in a subquery.
Fixed critical regression where the Oracle compiler would not maintain the
correct parameter values in the LIMIT/OFFSET for a select due to a caching
issue.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #6163 Fixes: #6173
Change-Id: Ieb12354271d09ad935d684ee0db4fa0128837215
Mike Bayer [Wed, 31 Mar 2021 23:38:10 +0000 (19:38 -0400)]
Correct for CTE correspondence w/ aliased CTE
Fixed regression where the :func:`_orm.joinedload` loader strategy would
not successfully joinedload to a mapper that is mapper against a
:class:`.CTE` construct.
Mike Bayer [Sun, 28 Mar 2021 15:09:40 +0000 (11:09 -0400)]
Add DeclarativeMeta to globals
Fixed issue in mypy plugin where newly added support for
:func:`_orm.as_declarative` needed to more fully add the
``DeclarativeMeta`` class to the mypy interpreter's state so that it does
not result in a name not found error; additionally improves how global
names are setup for the plugin including the ``Mapped`` name.
Introduces directory oriented testing as well, where a full
set of files will be copied, mypy runs, then zero or more patches
are applied and mypy is run again, to fully test incremental
behaviors.
Mike Bayer [Wed, 31 Mar 2021 17:16:04 +0000 (13:16 -0400)]
Expand sibling tests for overlaps warning
Scaled back the warning message added in :ticket:`5171` to not warn for
overlapping columns in an inheritance scenario where a particular
relationship is local to a subclass and therefore does not represent an
overlap.
Add errors documentation for the warning and also expand
``util.warn()`` to include a code parameter.
Mike Bayer [Wed, 31 Mar 2021 15:48:22 +0000 (11:48 -0400)]
Repair PGInspector
Fixed issue where the PostgreSQL :class:`.PGInspector`, when generated
against an :class:`_engine.Engine`, would fail for ``.get_enums()``,
``.get_view_names()``, ``.get_foreign_table_names()`` and
``.get_table_oid()`` when used against a "future" style engine and not the
connection directly.
Mike Bayer [Wed, 31 Mar 2021 00:30:13 +0000 (20:30 -0400)]
asyncio API can be considered beta level
It's clear that there are at least many dozens of developers
working with the new asyncio extension and we have not had
any major bug reports. Other than ``Session.delete`` needing
to be an awaitable, there were no backwards-incompatible issues
and we've seen no issues with wrong behaviors.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
