Mike Bayer [Wed, 9 Nov 2022 14:13:44 +0000 (09:13 -0500)]
try to support mypy 0.990
mypy introduces a crash we need to work around, also
some new rules. It also has either a behavioral change
regarding how output is rendered in relationship to
files being within sys.path or not, so work around
that for test_mypy_plugin_py3k.py
Mike Bayer [Fri, 4 Nov 2022 15:04:13 +0000 (11:04 -0400)]
support renamed symbols in annotation scans
Added support in ORM declarative annotations for class names specified for
:func:`_orm.relationship`, as well as the name of the :class:`_orm.Mapped`
symbol itself, to be different names than their direct class name, to
support scenarios such as where :class:`_orm.Mapped` is imported as
``from sqlalchemy.orm import Mapped as M``, or where related class names
are imported with an alternate name in a similar fashion. Additionally, a
target class name given as the lead argument for :func:`_orm.relationship`
will always supersede the name given in the left hand annotation, so that
otherwise un-importable names that also don't match the class name can
still be used in annotations.
Mike Bayer [Fri, 4 Nov 2022 16:48:43 +0000 (12:48 -0400)]
resolve synonyms in dictionary form of Session.get()
Improved "dictionary mode" for :meth:`_orm.Session.get` so that synonym
names which refer to primary key attribute names may be indicated in the
named dictionary.
Mike Bayer [Tue, 25 Oct 2022 13:10:09 +0000 (09:10 -0400)]
Support result.close() for all iterator patterns
This change contains new features for 2.0 only as well as some
behaviors that will be backported to 1.4.
For 1.4 and 2.0:
Fixed issue where the underlying DBAPI cursor would not be closed when
using :class:`_orm.Query` with :meth:`_orm.Query.yield_per` and direct
iteration, if a user-defined exception case were raised within the
iteration process, interrupting the iterator. This would lead to the usual
MySQL-related issues with server side cursors out of sync.
For 1.4 only:
A similar scenario can occur when using :term:`2.x` executions with direct
use of :class:`.Result`, in that case the end-user code has access to the
:class:`.Result` itself and should call :meth:`.Result.close` directly.
Version 2.0 will feature context-manager calling patterns to address this
use case. However within the 1.4 scope, ensured that ``.close()`` methods
are available on all :class:`.Result` implementations including
:class:`.ScalarResult`, :class:`.MappingResult`.
For 2.0 only:
To better support the use case of iterating :class:`.Result` and
:class:`.AsyncResult` objects where user-defined exceptions may interrupt
the iteration, both objects as well as variants such as
:class:`.ScalarResult`, :class:`.MappingResult`,
:class:`.AsyncScalarResult`, :class:`.AsyncMappingResult` now support
context manager usage, where the result will be closed at the end of
iteration.
Corrected various typing issues within the engine and async engine
packages.
Lele Gaifax [Wed, 2 Nov 2022 12:33:41 +0000 (08:33 -0400)]
Implement contains_value(), issubset() and issuperset() on PG Range
Added new methods :meth:`_postgresql.Range.contains` and
:meth:`_postgresql.Range.contained_by` to the new :class:`.Range` data
object, which mirror the behavior of the PostgreSQL ``@>`` and ``<@``
operators, as well as the
:meth:`_postgresql.AbstractRange.comparator_factory.contains` and
:meth:`_postgresql.AbstractRange.comparator_factory.contained_by` SQL
operator methods. Pull request courtesy Lele Gaifax.
Mike Bayer [Mon, 31 Oct 2022 19:09:34 +0000 (15:09 -0400)]
use simple decimal query to detect decimal char
Fixed issue where the ``nls_session_parameters`` view queried on first
connect in order to get the default decimal point character may not be
available depending on Oracle connection modes, and would therefore raise
an error. The approach to detecting decimal char has been simplified to
test a decimal value directly, instead of reading system views, which
works on any backend / driver.
Mike Bayer [Mon, 31 Oct 2022 13:51:51 +0000 (09:51 -0400)]
evaluate types in terms of the class in which they appear
Fixed issues within the declarative typing resolver (i.e. which resolves
``ForwardRef`` objects) where types that were declared for columns in one
particular source file would raise ``NameError`` when the ultimate mapped
class were in another source file. The types are now resolved in terms
of the module for each class in which the types are used.
Mike Bayer [Tue, 25 Oct 2022 20:00:50 +0000 (16:00 -0400)]
ensure pool.reset event always called for reset
Added new parameter :paramref:`.PoolEvents.reset.reset_state` parameter to
the :meth:`.PoolEvents.reset` event, with deprecation logic in place that
will continue to accept event hooks using the previous set of arguments.
This indicates various state information about how the reset is taking
place and is used to allow custom reset schemes to take place with full
context given.
Within this change a fix that's also backported to 1.4 is included which
re-enables the :meth:`.PoolEvents.reset` event to continue to take place
under all circumstances, including when :class:`.Connection` has already
"reset" the connection.
The two changes together allow custom reset schemes to be implemented using
the :meth:`.PoolEvents.reset` event, instead of the
:meth:`.PoolEvents.checkin` event (which continues to function as it always
has).
Mike Bayer [Sun, 30 Oct 2022 00:08:25 +0000 (20:08 -0400)]
fix test for same mapper to use "isa"
Fixed issue in joined eager loading where an assertion fail would occur
with a particular combination of outer/inner joined eager loads in
conjunction with an inherited subclass mapper as the middle target.
Mike Bayer [Fri, 28 Oct 2022 16:20:22 +0000 (12:20 -0400)]
update rel/fk FAQ entry
this entry still made the assumptions of behavior before
ticket #3061, that accessing a non-initialized scalar attribute
on a pending object would populate the attribute with None.
It also used the word "initialize" when referring to a persistent
object which is a misleading term, it's "loaded", even though
in this example it's "loading" the value of None.
Fix up the language to be more consistent with the #3061 change.
Mike Bayer [Thu, 27 Oct 2022 02:59:51 +0000 (22:59 -0400)]
ensure _ORMJoin transfers parententity from left side
Fixed bug involving :class:`.Select` constructs which used a combination of
:meth:`.Select.select_from` with an ORM entity followed by
:meth:`.Select.join` against the entity sent in
:meth:`.Select.select_from`, as well as using plain
:meth:`.Select.join_from`, which when combined with a columns clause that
didn't explicitly include that entity would then cause "automatic WHERE
criteria" features such as the IN expression required for a single-table
inheritance subclass, as well as the criteria set up by the
:func:`_orm.with_loader_criteria` option, to not be rendered for that
entity. The correct entity is now transferred to the :class:`.Join` object
that's generated internally, so that the criteria against the left
side entity is correctly added.
Mike Barry [Wed, 26 Oct 2022 17:40:32 +0000 (13:40 -0400)]
use only object_id() function for temp tables
Fixed issue with :meth:`.Inspector.has_table` when used against a temporary
table for the SQL Server dialect would fail an invalid object name error on
some Azure variants, due to an unnecessary information schema query that is
not supported on those server versions. Pull request courtesy Mike Barry.
the patch also fills out test support for has_table()
against temp tables, temp views, adding to the has_table() support just
added for views in #8700.
Mike Bayer [Thu, 27 Oct 2022 13:28:02 +0000 (09:28 -0400)]
apply basic escaping to anon_labels unconditionally
Fixed issue which prevented the :func:`_sql.literal_column` construct from
working properly within the context of a :class:`.Select` construct as well
as other potential places where "anonymized labels" might be generated, if
the literal expression contained characters which could interfere with
format strings, such as open parenthesis, due to an implementation detail
of the "anonymous label" structure.
Mike Bayer [Wed, 26 Oct 2022 17:27:21 +0000 (13:27 -0400)]
ensure inherited mapper attrs not interpreted as plain dataclass fields
Fixed issue in new dataclass mapping feature where a column declared on the
decalrative base / abstract base / mixin would leak into the constructor
for an inheriting subclass under some circumstances.
Mike Bayer [Mon, 24 Oct 2022 15:29:36 +0000 (11:29 -0400)]
reconcile Mapper properties ordering against mapped Table
Changed a fundamental configuration behavior of :class:`.Mapper`, where
:class:`_schema.Column` objects that are explicitly present in the
:paramref:`_orm.Mapper.properties` dictionary, either directly or enclosed
within a mapper property object, will now be mapped within the order of how
they appear within the mapped :class:`.Table` (or other selectable) itself
(assuming they are in fact part of that table's list of columns), thereby
maintaining the same order of columns in the mapped selectable as is
instrumented on the mapped class, as well as what renders in an ORM SELECT
statement for that mapper. Previously (where "previously" means since
version 0.0.1), :class:`.Column` objects in the
:paramref:`_orm.Mapper.properties` dictionary would always be mapped first,
ahead of when the other columns in the mapped :class:`.Table` would be
mapped, causing a discrepancy in the order in which the mapper would
assign attributes to the mapped class as well as the order in which they
would render in statements.
The change most prominently takes place in the way that Declarative
assigns declared columns to the :class:`.Mapper`, specifically how
:class:`.Column` (or :func:`_orm.mapped_column`) objects are handled
when they have a DDL name that is explicitly different from the mapped
attribute name, as well as when constructs such as :func:`_orm.deferred`
etc. are used. The new behavior will see the column ordering within
the mapped :class:`.Table` being the same order in which the attributes
are mapped onto the class, assigned within the :class:`.Mapper` itself,
and rendered in ORM statements such as SELECT statements, independent
of how the :class:`_schema.Column` was configured against the
:class:`.Mapper`.
Mike Bayer [Tue, 25 Oct 2022 14:22:14 +0000 (10:22 -0400)]
raise for non-Load opt passed to options()
Fixed the exception that's raised when the
:func:`_orm.with_loader_criteria` option is attempted to be used within a
specific loader path, like in loader.options().
:func:`_orm.with_loader_criteria` is only intended to be used at the top
level.
Mike Bayer [Mon, 24 Oct 2022 23:24:11 +0000 (19:24 -0400)]
add Oracle-specific parameter escapes for expanding params
Fixed issue where bound parameter names, including those automatically
derived from similarly-named database columns, which contained characters
that normally require quoting with Oracle would not be escaped when using
"expanding parameters" with the Oracle dialect, causing execution errors.
The usual "quoting" for bound parameters used by the Oracle dialect is not
used with the "expanding parameters" architecture, so escaping for a large
range of characters is used instead, now using a list of characters/escapes
that are specific to Oracle.
Mike Bayer [Sun, 23 Oct 2022 23:24:54 +0000 (19:24 -0400)]
skip ad-hoc properties within subclass_load_via_in
Fixed issue where "selectin_polymorphic" loading for inheritance mappers
would not function correctly if the :param:`_orm.Mapper.polymorphic_on`
parameter referred to a SQL expression that was not directly mapped on the
class.
Mike Bayer [Sun, 23 Oct 2022 14:34:33 +0000 (10:34 -0400)]
test support for has_table()->view; backport to 1.4
For 1.4 only; in 2.0 this just refines the test suite a bit.
Fixed regression which occurred throughout the 1.4 series where the
:meth:`.Inspector.has_table` method, which historically reported on views
as well, stopped working for SQL Server. The issue is not present in the
2.0 series which uses a different reflection architecture. Test support is
added to ensure ``has_table()`` remains working per spec re: views.
Mike Bayer [Fri, 21 Oct 2022 16:37:04 +0000 (12:37 -0400)]
Only convert Range for sqlalchemy Range object
Refined the new approach to range objects described at :ref:`change_7156`
to accommodate driver-specific range and multirange objects, to better
accommodate both legacy code as well as when passing results from raw SQL
result sets back into new range or multirange expressions.
Mike Bayer [Fri, 21 Oct 2022 04:11:10 +0000 (00:11 -0400)]
allow legacy forms with __allow_unmapped__
Improved support for legacy 1.4 mappings that use annotations which don't
include ``Mapped[]``, by ensuring the ``__allow_unmapped__`` attribute can
be used to allow such legacy annotations to pass through Annotated
Declarative without raising an error and without being interpreted in an
ORM runtime context. Additionally improved the error message generated when
this condition is detected, and added more documentation for how this
situation should be handled. Unfortunately the 1.4 WARN_SQLALCHEMY_20
migration warning cannot detect this particular configurational issue at
runtime with its current architecture.
Mike Bayer [Thu, 20 Oct 2022 16:28:29 +0000 (12:28 -0400)]
move API docs downwards
Sphinx 5.3 (compared to 5.1.1) is now putting all the autodoc
names into the TOC. So we have to start being more careful
to make sure API docs are well below narrative docs, because
this new style is a wall of text. i dont yet see any options
to turn it off, but it does seem like a good improvement, just makes
doc organization a more difficult endeavor.
Mike Bayer [Thu, 20 Oct 2022 16:05:33 +0000 (12:05 -0400)]
collect annotation earlier for mapped_column present
Fixed issue with new dataclass mapping feature where arguments passed to
the dataclasses API could sometimes be mis-ordered when dealing with mixins
that override :func:`_orm.mapped_column` declarations, leading to
initializer problems.
the change made here is specific to the test case given which regards
mapped_column() most specifically. cases that involve relationship()
etc. are not tested here, however mapped_column() is the only attribute
that's implicit without an instance given on the right side, and is also
most common for mixins. not clear if there are more issues in this
area, however it appears that we need only adjust the order in which we
accommodate grabbing the annotations in order to affect how dataclasses
sees the class; that is, we have control over ``__annotations__`` here
so dont have to worry about ``cls.__dict__``.
Mike Bayer [Wed, 19 Oct 2022 13:41:44 +0000 (09:41 -0400)]
more many-to-one typing
since we are typing centric, note this configuration
as we have just supported in #8668.
Note also I am just taking "backref" out of the basic
version of the docs here totally, this doc is already
a lot to read / take in without making it even more
confusing; backref still has an entirely dedicated
docs page which can have all the additional behaviors
of backref() described.
Additionally, get other "optional" forms to work including
``cls | None`` and ``Union[cls, None]``.
Mike Bayer [Wed, 19 Oct 2022 13:17:03 +0000 (09:17 -0400)]
rename tables to _table for basic relationships
the names "parent" / "child" are confusing for new users
in that they are used for table names as well as
"back_populates='parent'", use a disambiguated name.
In this change, there's now overlap between the
variable named "association_table" and the table name
"association_table". not sure of a better naming system.
Mike Bayer [Wed, 19 Oct 2022 01:01:05 +0000 (21:01 -0400)]
de-optionalize union types to support Optional for m2o
Fixed bug in new ORM typed declarative mappings where we did not include
the ability to use ``Optional[]`` in the type annotation for a many-to-one
relationship, even though this is common.
Mike Bayer [Tue, 18 Oct 2022 17:25:06 +0000 (13:25 -0400)]
call super().__init_subclass__(); support GenericAlias
Improved the :class:`.DeclarativeBase` class so that when combined with
other mixins like :class:`.MappedAsDataclass`, the order of the classes may
be in either order.
Added support for mapped classes that are also ``Generic`` subclasses,
to be specified as a ``GenericAlias`` object (e.g. ``MyClass[str]``)
within statements and calls to :func:`_sa.inspect`.
Mike Bayer [Tue, 18 Oct 2022 13:44:37 +0000 (09:44 -0400)]
further qualify pyodbc setinputsizes types for long stirngs
Fixed regression caused by SQL Server pyodbc change :ticket:`8177` where we
now use ``setinputsizes()`` by default; for VARCHAR, this fails if the
character size is greater than 4000 (or 2000, depending on data) characters
as the incoming datatype is NVARCHAR, which has a limit of 4000 characters,
despite the fact that VARCHAR can handle unlimited characters. Additional
pyodbc-specific typing information is now passed to ``setinputsizes()``
when the datatype's size is > 2000 characters. The change is also applied
to the :class:`.JSON` type which was also impacted by this issue for large
JSON serializations.
The :class:`.Sequence` construct restores itself to the DDL behavior it
had prior to the 1.4 series, where creating a :class:`.Sequence` with
no additional arguments will emit a simple ``CREATE SEQUENCE`` instruction
**without** any additional parameters for "start value". For most backends,
this is how things worked previously in any case; **however**, for
MS SQL Server, the default value on this database is
``-2**63``; to prevent this generally impractical default
from taking effect on SQL Server, the :paramref:`.Sequence.start` parameter
should be provided. As usage of :class:`.Sequence` is unusual
for SQL Server which for many years has standardized on ``IDENTITY``,
it is hoped that this change has minimal impact.
Mike Bayer [Mon, 17 Oct 2022 19:09:01 +0000 (15:09 -0400)]
update SEQUENCE docs ahead of default change
for backport to 1.4 as well, remove references to
Firebird, and also revert "associate Sequence with MetaData"
step as this is not needed usually, just note that schema
is not shared. encourage users to use IDENTITY instead.
Mike Bayer [Mon, 17 Oct 2022 15:53:07 +0000 (11:53 -0400)]
simplify unmapped col eval fallback
Removed the warning that emits when using ORM-enabled update/delete
regarding evaluation of columns by name, first added in :ticket:`4073`;
this warning actually covers up a scenario that otherwise could populate
the wrong Python value for an ORM mapped attribute depending on what the
actual column is, so this deprecated case is removed. In 2.0, ORM enabled
update/delete uses "auto" for "synchronize_session", which should do the
right thing automatically for any given UPDATE expression.
Mike Bayer [Sun, 16 Oct 2022 22:30:44 +0000 (18:30 -0400)]
repair type qualify in _ServerDefaultType; other pyright tweaks
as we haven't done full pylance / pyright strict typing internally,
some of the things pyright reports on specifically will leak
out into user code, such as this mapped_column() issue.
So we will have to look more closely at pyright strict
mode going forward for the release.
Fixed typing issue where pylance strict mode would report "partially
unknown" datatype for the :func:`_orm.mapped_column` construct.
Also repaired a trailing comma and pyright complaining about overloads
for orm.composite.
Mike Bayer [Sat, 15 Oct 2022 19:20:21 +0000 (15:20 -0400)]
accommodate arbitrary embedded params in insertmanyvalues
Fixed bug in new "insertmanyvalues" feature where INSERT that included a
subquery with :func:`_sql.bindparam` inside of it would fail to render
correctly in "insertmanyvalues" format. This affected psycopg2 most
directly as "insertmanyvalues" is used unconditionally with this driver.
Mike Bayer [Sat, 15 Oct 2022 15:12:25 +0000 (11:12 -0400)]
disable isort in pyproject.toml
disable isort, for IDEs that just default isort to be turned on, e.g. vscode.
we use flake8-import-order for import sorting, using zimports to actually
reformat code. isort is nicer in many ways but doesn't have our
"import *" fixer and also is not 100% compatible with flake8-import-order.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
