Mike Bayer [Sun, 26 Feb 2023 14:31:36 +0000 (09:31 -0500)]
include columns from superclasses that indicate "selectin"
Added support for the :paramref:`_orm.Mapper.polymorphic_load` parameter to
be applied to each mapper in an inheritance hierarchy more than one level
deep, allowing columns to load for all classes in the hierarchy that
indicate ``"selectin"`` using a single statement, rather than ignoring
elements on those intermediary classes that nonetheless indicate they also
would participate in ``"selectin"`` loading and were not part of the
base-most SELECT statement.
Matus Valo [Sun, 26 Feb 2023 20:10:23 +0000 (15:10 -0500)]
Declare KEY_OBJECTS_ONLY as cdef variable
A small optimization to the Cython implementation of :class:`.ResultProxy`
using a cdef for a particular int value to avoid Python overhead. Pull
request courtesy Matus Valo.
Mike Bayer [Tue, 21 Feb 2023 15:34:01 +0000 (10:34 -0500)]
apply a fixed locals w/ Mapped to all de-stringify
Continued the fix for :ticket:`8853`, allowing the :class:`_orm.Mapped`
name to be fully qualified regardless of whether or not
``from __annotations__ import future`` were present. This issue first fixed
in 2.0.0b3 confirmed that this case worked via the test suite, however the
test suite apparently was not testing the behavior for the name ``Mapped``
not being locally present at all; string resolution has been updated to
ensure the ``Mapped`` symbol is locatable as applies to how the ORM uses
these functions.
Mike Bayer [Sun, 26 Feb 2023 20:34:57 +0000 (15:34 -0500)]
use read-only Mapping for values dictionary type
Improved typing for the mapping passed to :meth:`.UpdateBase.values` to be
more open-ended about collection type, by indicating read-only ``Mapping``
instead of writeable ``Dict``, the latter of which would error out under
typing tools on too limited of a key type.
Mike Bayer [Fri, 24 Feb 2023 15:58:25 +0000 (10:58 -0500)]
locate automap base in hierarchy directly
Fixed issue in automap where calling ``.prepare()`` from one of the mapped
classes would not use the correct base class when automap detected new
tables, instead using the given class, leading to mappers trying to
configure inheritance. While one should normally call ``.prepare()`` from
the base in any case, it shouldn't misbehave that badly when called from a
subclass.
Mike Bayer [Sat, 18 Feb 2023 14:10:20 +0000 (09:10 -0500)]
consider column.name directly when evaluating use_existing_column
Fixed issue where new :paramref:`_orm.mapped_column.use_existing_column`
feature would not work if the two same-named columns were mapped under
attribute names that were differently-named from the explicit name given to
the column itself. The attribute names can now be differently named when
using this parameter.
Federico Caselli [Thu, 16 Feb 2023 20:52:18 +0000 (21:52 +0100)]
Allow custom sorting of column in the ORM.
To accommodate a change in column ordering used by ORM Declarative in
SQLAlchemy 2.0, a new parameter :paramref:`_orm.mapped_column.sort_order`
has been added that can be used to control the order of the columns defined
in the table by the ORM, for common use cases such as mixins with primary
key columns that should appear first in tables. The change notes at
:ref:`change_9297` illustrate the default change in ordering behavior
(which is part of all SQLAlchemy 2.0 releases) as well as use of the
:paramref:`_orm.mapped_column.sort_order` to control column ordering when
using mixins and multiple classes (new in 2.0.4).
Mike Bayer [Thu, 16 Feb 2023 14:39:07 +0000 (09:39 -0500)]
modernize hybrids and apply typing
Improved the typing support for the :ref:`hybrids_toplevel`
extension, updated all documentation to use ORM Annotated Declarative
mappings, and added a new modifier called :attr:`.hybrid_property.inplace`.
This modifier provides a way to alter the state of a :class:`.hybrid_property`
**in place**, which is essentially what very early versions of hybrids
did, before SQLAlchemy version 1.2.0 :ticket:`3912` changed this to
remove in-place mutation. This in-place mutation is now restored on an
**opt-in** basis to allow a single hybrid to have multiple methods
set up, without the need to name all the methods the same and without the
need to carefully "chain" differently-named methods in order to maintain
the composition. Typing tools such as Mypy and Pyright do not allow
same-named methods on a class, so with this change a succinct method
of setting up hybrids with typing support is restored.
mike bayer [Wed, 15 Feb 2023 22:20:06 +0000 (23:20 +0100)]
Fix coercion issue for tuple bindparams
Fixed issue where element types of a tuple value would be hardcoded to take
on the types from a compared-to tuple, when the comparison were using the
:meth:`.ColumnOperators.in_` operator. This was inconsistent with the usual
way that types are determined for a binary expression, which is that the
actual element type on the right side is considered first before applying
the left-hand-side type.
Mike Bayer [Mon, 13 Feb 2023 16:17:09 +0000 (11:17 -0500)]
immediateload lazy relationships named in refresh.attribute_names
The :meth:`_orm.Session.refresh` method will now immediately load a
relationship-bound attribute that is explicitly named within the
:paramref:`_orm.Session.refresh.attribute_names` collection even if it is
currently linked to the "select" loader, which normally is a "lazy" loader
that does not fire off during a refresh. The "lazy loader" strategy will
now detect that the operation is specifically a user-initiated
:meth:`_orm.Session.refresh` operation which named this attribute
explicitly, and will then call upon the "immediateload" strategy to
actually emit SQL to load the attribute. This should be helpful in
particular for some asyncio situations where the loading of an unloaded
lazy-loaded attribute must be forced, without using the actual lazy-loading
attribute pattern not supported in asyncio.
Mike Bayer [Fri, 10 Feb 2023 21:06:23 +0000 (16:06 -0500)]
add dataclasses callable and apply annotations more strictly
Added new parameter ``dataclasses_callable`` to both the
:class:`_orm.MappedAsDataclass` class as well as the
:meth:`_orm.registry.mapped_as_dataclass` method which allows an
alternative callable to Python ``dataclasses.dataclass`` to be used in
order to produce dataclasses. The use case here is to drop in Pydantic's
dataclass function instead. Adjustments have been made to the mixin support
added for :ticket:`9179` in version 2.0.1 so that the ``__annotations__``
collection of the mixin is rewritten to not include the
:class:`_orm.Mapped` container, in the same way as occurs with mapped
classes, so that the Pydantic dataclasses constructor is not exposed to
unknown types.
Mike Bayer [Mon, 13 Feb 2023 17:23:28 +0000 (12:23 -0500)]
completely lift-and-copy and_ / or_ documentation
Sphinx refuses to write the correct docstring for these without
placing a completely literal string with no interpolation of
any kind. Current site has blank for these.
Fix docs for `case` expression to match new syntax (#9279)
* Fix docs for `case` expression to match new syntax
Previously (before v1.4), the `whens` arg (when `value` is *not* used) used to be a list of
conditions (a 2 item-tuple of condition + value). From v1.4, these are passed
as positional args and the old syntax is not supported anymore.
Mike Bayer [Fri, 10 Feb 2023 13:39:21 +0000 (08:39 -0500)]
generalize adapt_on_names to expect non-named elements
The fix in #9217 opened up adapt_on_names to more kinds of
expressions than it was prepared for; adjust that logic
and also refine in the ORM where we are using it, as we
dont need it (yet) for the DML RETURNING use case.
Fixed regression introduced in version 2.0.2 due to :ticket:`9217` where
using DML RETURNING statements, as well as
:meth:`_sql.Select.from_statement` constructs as was "fixed" in
:ticket:`9217`, in conjunction with ORM mapped classes that used
expressions such as with :func:`_orm.column_property`, would lead to an
internal error within Core where it would attempt to match the expression
by name. The fix repairs the Core issue, and also adjusts the fix in
:ticket:`9217` to not take effect for the DML RETURNING use case, where it
adds unnecessary overhead.
Mike Bayer [Thu, 9 Feb 2023 20:36:38 +0000 (15:36 -0500)]
apply self_group to all elements of multi-expression
Fixed critical regression in SQL expression formulation in the 2.0 series
due to :ticket:`7744` which improved support for SQL expressions that
contained many elements against the same operator repeatedly; parenthesis
grouping would be lost with expression elements beyond the first two
elements.
Mike Bayer [Thu, 9 Feb 2023 17:05:47 +0000 (12:05 -0500)]
pin sphinx-copybutton and change config
sphinx-copybutton introduced a new feature
in 0.5.1 which includes a default configuration
that breaks the regexp prompt matching scheme.
set copybutton_exclude to not include ".gp" as that's the class
where we exactly look for the prompts we are matching.
While we're there, use this new feature to exclude our sql
styles, even though this is not strictly necessary in our case.
pin sphinx-copybutton at 0.5.1 to avoid future problems.
Mike Bayer [Sun, 5 Feb 2023 19:12:50 +0000 (14:12 -0500)]
do not return asyncio connections to the pool under gc
Repaired a regression caused by the fix for :ticket:`8419` which caused
asyncpg connections to be reset (i.e. transaction ``rollback()`` called)
and returned to the pool normally in the case that the connection were not
explicitly returned to the connection pool and was instead being
intercepted by Python garbage collection, which would fail if the garbage
collection operation were being called outside of the asyncio event loop,
leading to a large amount of stack trace activity dumped into logging
and standard output.
The correct behavior is restored, which is that all asyncio connections
that are garbage collected due to not being explicitly returned to the
connection pool are detached from the pool and discarded, along with a
warning, rather than being returned the pool, as they cannot be reliably
reset. In the case of asyncpg connections, the asyncpg-specific
``terminate()`` method will be used to end the connection more gracefully
within this process as opposed to just dropping it.
This change includes a small behavioral change that is hoped to be useful
for debugging asyncio applications, where the warning that's emitted in the
case of asyncio connections being unexpectedly garbage collected has been
made slightly more aggressive by moving it outside of a ``try/except``
block and into a ``finally:`` block, where it will emit unconditionally
regardless of whether the detach/termination operation succeeded or not. It
will also have the effect that applications or test suites which promote
Python warnings to exceptions will see this as a full exception raise,
whereas previously it was not possible for this warning to actually
propagate as an exception. Applications and test suites which need to
tolerate this warning in the interim should adjust the Python warnings
filter to allow these warnings to not raise.
The behavior for traditional sync connections remains unchanged, that
garbage collected connections continue to be returned to the pool normally
without emitting a warning. This will likely be changed in a future major
release to at least emit a similar warning as is emitted for asyncio
drivers, as it is a usage error for pooled connections to be intercepted by
garbage collection without being properly returned to the pool.
Mike Bayer [Mon, 6 Feb 2023 18:54:56 +0000 (13:54 -0500)]
add error code 1049 for mysql has_table
Fixed regression caused by issue :ticket:`9058` which adjusted the MySQL
dialect's ``has_table()`` to again use "DESCRIBE", where the specific error
code raised by MySQL version 8 when using a non-existent schema name was
unexpected and failed to be interpreted as a boolean result.
Fixed the SQLite dialect's ``has_table()`` function to correctly report
False for queries that include a non-None schema name for a schema that
doesn't exist; previously, a database error was raised.
jazzthief [Thu, 2 Feb 2023 18:48:13 +0000 (13:48 -0500)]
Dedicated bitwise operators
Added a full suite of new SQL bitwise operators, for performing
database-side bitwise expressions on appropriate data values such as
integers, bit-strings, and similar. Pull request courtesy Yegor Statkevich.
Mike Bayer [Thu, 2 Feb 2023 19:38:37 +0000 (14:38 -0500)]
port history meta to 2.0
first change:
Reworked the :ref:`examples_versioned_history` to work with
version 2.0, while at the same time improving the overall working of
this example to use newer APIs, including a newly added hook
:meth:`_orm.MapperEvents.after_mapper_constructed`.
second change:
Added new event hook :meth:`_orm.MapperEvents.after_mapper_constructed`,
which supplies an event hook to take place right as the
:class:`_orm.Mapper` object has been fully constructed, but before the
:meth:`_orm.registry.configure` call has been called. This allows code that
can create additional mappings and table structures based on the initial
configuration of a :class:`_orm.Mapper`, which also integrates within
Declarative configuration. Previously, when using Declarative, where the
:class:`_orm.Mapper` object is created within the class creation process,
there was no documented means of running code at this point. The change
is to immediately benefit custom mapping schemes such as that
of the :ref:`examples_versioned_history` example, which generate additional
mappers and tables in response to the creation of mapped classes.
third change:
The infrequently used :attr:`_orm.Mapper.iterate_properties` attribute and
:meth:`_orm.Mapper.get_property` method, which are primarily used
internally, no longer implicitly invoke the :meth:`_orm.registry.configure`
process. Public access to these methods is extremely rare and the only
benefit to having :meth:`_orm.registry.configure` would have been allowing
"backref" properties be present in these collections. In order to support
the new :meth:`_orm.MapperEvents.after_mapper_constructed` event, iteration
and access to the internal :class:`_orm.MapperProperty` objects is now
possible without triggering an implicit configure of the mapper itself.
The more-public facing route to iteration of all mapper attributes, the
:attr:`_orm.Mapper.attrs` collection and similar, will still implicitly
invoke the :meth:`_orm.registry.configure` step thus making backref
attributes available.
In all cases, the :meth:`_orm.registry.configure` is always available to
be called directly.
fourth change:
Fixed obscure ORM inheritance issue caused by :ticket:`8705` where some
scenarios of inheriting mappers that indicated groups of columns from the
local table and the inheriting table together under a
:func:`_orm.column_property` would nonetheless warn that properties of the
same name were being combined implicitly.
Caspar Wylie [Fri, 3 Feb 2023 14:23:26 +0000 (09:23 -0500)]
use mysql 8 syntax for ON DUPLICATE KEY UPDATE
Added support for MySQL 8's new ``AS <name> ON DUPLICATE KEY`` syntax when
using :meth:`_mysql.Insert.on_duplicate_key_update`, which is required for
newer versions of MySQL 8 as the previous syntax using ``VALUES()`` now
emits a deprecation warning with those versions. Server version detection
is employed to determine if traditional MariaDB / MySQL < 8 ``VALUES()``
syntax should be used, vs. the newer MySQL 8 required syntax. Pull request
courtesy Caspar Wylie.
Mike Bayer [Mon, 6 Feb 2023 15:16:26 +0000 (10:16 -0500)]
check for superclasses of user defined init
Fixed regression caused by the fix for :ticket:`9171`, which itself was
fixing a regression, involving the mechanics of ``__init__()`` on classes
that extend from :class:`_orm.DeclarativeBase`. The change made it such
that ``__init__()`` was applied to the user-defined base if there were no
``__init__()`` method directly on the class. This has been adjusted so that
``__init__()`` is applied only if no other class in the hierarchy of the
user-defined base has an ``__init__()`` method. This again allows
user-defined base classes based on :class:`_orm.DeclarativeBase` to include
mixins that themselves include a custom ``__init__()`` method.
Mike Bayer [Sat, 4 Feb 2023 21:35:21 +0000 (16:35 -0500)]
coerce elements in mapper.primary_key, process in __mapper_args__
Repaired ORM Declarative mappings to allow for the
:paramref:`_orm.Mapper.primary_key` parameter to be specified within
``__mapper_args__`` when using :func:`_orm.mapped_column`. Despite this
usage being directly in the 2.0 documentation, the :class:`_orm.Mapper` was
not accepting the :func:`_orm.mapped_column` construct in this context. Ths
feature was already working for the :paramref:`_orm.Mapper.version_id_col`
and :paramref:`_orm.Mapper.polymorphic_on` parameters.
As part of this change, the ``__mapper_args__`` attribute may be specified
without using :func:`_orm.declared_attr` on a non-mapped mixin class,
including a ``"primary_key"`` entry that refers to :class:`_schema.Column`
or :func:`_orm.mapped_column` objects locally present on the mixin;
Declarative will also translate these columns into the correct ones for a
particular mapped class. This again was working already for the
:paramref:`_orm.Mapper.version_id_col` and
:paramref:`_orm.Mapper.polymorphic_on` parameters. Additionally,
elements within ``"primary_key"`` may be indicated as string names of
existing mapped properties.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
