Mike Bayer [Mon, 6 Mar 2023 03:25:46 +0000 (22:25 -0500)]
comment out pickle/ process test and attempt 2.0.5.post1
if .post1 fails to work out, we'll just do 2.0.6
The test_pickle_rows_other_process test is failing during wheel
builds as it seems that the "subprocess" run is not using the
cython extensions, leading to a pickle mismatch between the
cythonized and the pure python version of a row. comment
out this test and attempt to release as 2.0.5.post1 so that
wheels can build.
Mike Bayer [Sat, 4 Mar 2023 20:31:41 +0000 (15:31 -0500)]
KeyFuncDict regression fixes and dataclass fixes
adapt None-key warning for non-mapped attributes
Fixed multiple regressions due to :ticket:`8372`, involving
:func:`_orm.attribute_mapped_collection` (now called
:func:`_orm.attribute_keyed_dict`).
First, the collection was no longer usable with "key" attributes that were
not themselves ordinary mapped attributes; attributes linked to descriptors
and/or association proxy attributes have been fixed.
Second, if an event or other operation needed access to the "key" in order
to populate the dictionary from an mapped attribute that was not
loaded, this also would raise an error inappropriately, rather than
trying to load the attribute as was the behavior in 1.4. This is also
fixed.
For both cases, the behavior of :ticket:`8372` has been expanded.
:ticket:`8372` introduced an error that raises when the derived key that
would be used as a mapped dictionary key is effectively unassigned. In this
change, a warning only is emitted if the effective value of the ".key"
attribute is ``None``, where it cannot be unambiguously determined if this
``None`` was intentional or not. ``None`` will be not supported as mapped
collection dictionary keys going forward (as it typically refers to NULL
which means "unknown"). Setting
:paramref:`_orm.attribute_keyed_dict.ignore_unpopulated_attribute` will now
cause such ``None`` keys to be ignored as well.
Add value constructors to dictionary collections
Added constructor arguments to the built-in mapping collection types
including :class:`.KeyFuncDict`, :func:`_orm.attribute_keyed_dict`,
:func:`_orm.column_keyed_dict` so that these dictionary types may be
constructed in place given the data up front; this provides further
compatibility with tools such as Python dataclasses ``.asdict()`` which
relies upon invoking these classes directly as ordinary dictionary classes.
Mike Bayer [Fri, 3 Mar 2023 14:30:58 +0000 (09:30 -0500)]
audition pymssql once more; retire sane_rowcount_returning
pymssql seems to be maintained again and seems to be working
completely, so let's try re-enabling it.
Fixed issue in the new :class:`.Uuid` datatype which prevented it from
working with the pymssql driver. As pymssql seems to be maintained again,
restored testing support for pymssql.
Tweaked the pymssql dialect to take better advantage of
RETURNING for INSERT statements in order to retrieve last inserted primary
key values, in the same way as occurs for the mssql+pyodbc dialect right
now.
Identified that the ``sqlite`` and ``mssql+pyodbc`` dialects are now
compatible with the SQLAlchemy ORM's "versioned rows" feature, since
SQLAlchemy now computes rowcount for a RETURNING statement in this specific
case by counting the rows returned, rather than relying upon
``cursor.rowcount``. In particular, the ORM versioned rows use case
(documented at :ref:`mapper_version_counter`) should now be fully
supported with the SQL Server pyodbc dialect.
Nils Philippsen [Tue, 28 Feb 2023 21:04:54 +0000 (16:04 -0500)]
restore old *args approach for MutableDict.pop()
The typing change in ba0e508141206efc55cdab91df21c1
changed the semantics of pop() and possibly setdefault() in order to
try working at runtime with a two-argument signature. however
the implementation for this in cpython likely uses a strict `*args`
approach where the lack of the second parameter is explicit, rather
than matching to a constant. Restore the old implementation inside
of a "not TYPE_CHECKING" block while keeping the type annotated forms
intact for typing only.
Fixed regression caused by typing added to ``sqlalchemy.ext.mutable`` for
:ticket:`8667`, where the semantics of the ``.pop()`` method changed such
that the method was non-working. Pull request courtesy Nils Philippsen.
Mike Bayer [Fri, 24 Feb 2023 21:15:21 +0000 (16:15 -0500)]
ensure event handlers called for all do_ping
The support for pool ping listeners to receive exception events via the
:meth:`.ConnectionEvents.handle_error` event added in 2.0.0b1 for
:ticket:`5648` failed to take into account dialect-specific ping routines
such as that of MySQL and PostgreSQL. The dialect feature has been reworked
so that all dialects participate within event handling. Additionally,
a new boolean element :attr:`.ExceptionContext.is_pre_ping` is added
which identifies if this operation is occurring within the pre-ping
operation.
For this release, third party dialects which implement a custom
:meth:`_engine.Dialect.do_ping` method can opt in to the newly improved
behavior by having their method no longer catch exceptions or check
exceptions for "is_disconnect", instead just propagating all exceptions
outwards. Checking the exception for "is_disconnect" is now done by an
enclosing method on the default dialect, which ensures that the event hook
is invoked for all exception scenarios before testing the exception as a
"disconnect" exception. If an existing ``do_ping()`` method continues to
catch exceptions and check "is_disconnect", it will continue to work as it
did previously, but ``handle_error`` hooks will not have access to the
exception if it isn't propagated outwards.
Mike Bayer [Wed, 1 Mar 2023 16:07:25 +0000 (11:07 -0500)]
TextualSelect is ReturnsRowsRole
Fixed typing bug where :meth:`_sql.Select.from_statement` would not accept
:func:`_sql.text` or :class:`.TextualSelect` objects as a valid type.
Additionally repaired the :class:`.TextClause.columns` method to have a
return type, which was missing.
Mike Bayer [Thu, 2 Mar 2023 01:44:49 +0000 (20:44 -0500)]
allow multiparams with scalars
Fixed bug where the :meth:`_engine.Connection.scalars` method was not typed
as allowing a multiple-parameters list, which is now supported using
insertmanyvalues operations.
easy_markie_tee [Wed, 1 Mar 2023 21:51:43 +0000 (16:51 -0500)]
Fix Typo In Tutorial (#9399)
Missing a word on the page `Working with Database Metadata`.
First paragraph under section 'Setting up MetaData with Table objects'.
"...the database which we query from is know [as] a table."
Co-authored-by: markie tee <cassette.head@gmail.com>
Mike Bayer [Tue, 28 Feb 2023 16:05:48 +0000 (11:05 -0500)]
ensure single import per line
This adds the very small plugin flake8-import-single which
will prevent us from having an import with more than one symbol
on a line.
Flake8 by itself prevents this pattern with E401:
import collections, os, sys
However does not do anything with this:
from sqlalchemy import Column, text
Both statements have the same issues generating merge artifacts
as well as presenting a manual decision to be made. While
zimports generally cleans up such imports at the top level, we
don't enforce zimports / pre-commit use.
the plugin finds the same issue for imports that are inside of
test methods. We shouldn't usually have imports in test methods
so most of them here are moved to be top level.
The version is pinned at 0.1.5; the project seems to have no
activity since 2019, however there are three 0.1.6dev releases
on pypi which stopped in September 2019, they seem to be
experiments with packaging. The source for 0.1.5
is extremely simple and only reveals one method to flake8
(the run() method).
Mike Bayer [Sun, 26 Feb 2023 14:31:36 +0000 (09:31 -0500)]
include columns from superclasses that indicate "selectin"
Added support for the :paramref:`_orm.Mapper.polymorphic_load` parameter to
be applied to each mapper in an inheritance hierarchy more than one level
deep, allowing columns to load for all classes in the hierarchy that
indicate ``"selectin"`` using a single statement, rather than ignoring
elements on those intermediary classes that nonetheless indicate they also
would participate in ``"selectin"`` loading and were not part of the
base-most SELECT statement.
Grey Li [Sun, 26 Feb 2023 10:56:37 +0000 (05:56 -0500)]
Add separate version notes for scalars
Add separate 1.4.24 and 1.4.26 version notes for the .scalars method;
this covers Session, scoped_session, AsyncSession, async_scoped_session
as the "scoped" versions did not have the method added until 1.4.26
as part of :ticket:`7103`.
Also indicate scoped_session as ``sqlalchemy.orm.scoped_session`` in
docs rather than ``sqlalchemy.orm.scoping.scoped_session``. This is
also happening in I77da54891860095edcb1f0625ead99fee89bd76f separately,
as both changesets refer to scoped_session without using ".scoping".
Matus Valo [Sun, 26 Feb 2023 20:10:23 +0000 (15:10 -0500)]
Declare KEY_OBJECTS_ONLY as cdef variable
A small optimization to the Cython implementation of :class:`.ResultProxy`
using a cdef for a particular int value to avoid Python overhead. Pull
request courtesy Matus Valo.
Mike Bayer [Tue, 21 Feb 2023 15:34:01 +0000 (10:34 -0500)]
apply a fixed locals w/ Mapped to all de-stringify
Continued the fix for :ticket:`8853`, allowing the :class:`_orm.Mapped`
name to be fully qualified regardless of whether or not
``from __annotations__ import future`` were present. This issue first fixed
in 2.0.0b3 confirmed that this case worked via the test suite, however the
test suite apparently was not testing the behavior for the name ``Mapped``
not being locally present at all; string resolution has been updated to
ensure the ``Mapped`` symbol is locatable as applies to how the ORM uses
these functions.
Mike Bayer [Sun, 26 Feb 2023 20:34:57 +0000 (15:34 -0500)]
use read-only Mapping for values dictionary type
Improved typing for the mapping passed to :meth:`.UpdateBase.values` to be
more open-ended about collection type, by indicating read-only ``Mapping``
instead of writeable ``Dict``, the latter of which would error out under
typing tools on too limited of a key type.
Mike Bayer [Fri, 24 Feb 2023 15:58:25 +0000 (10:58 -0500)]
locate automap base in hierarchy directly
Fixed issue in automap where calling ``.prepare()`` from one of the mapped
classes would not use the correct base class when automap detected new
tables, instead using the given class, leading to mappers trying to
configure inheritance. While one should normally call ``.prepare()`` from
the base in any case, it shouldn't misbehave that badly when called from a
subclass.
Mike Bayer [Sat, 18 Feb 2023 14:10:20 +0000 (09:10 -0500)]
consider column.name directly when evaluating use_existing_column
Fixed issue where new :paramref:`_orm.mapped_column.use_existing_column`
feature would not work if the two same-named columns were mapped under
attribute names that were differently-named from the explicit name given to
the column itself. The attribute names can now be differently named when
using this parameter.
Federico Caselli [Thu, 16 Feb 2023 20:52:18 +0000 (21:52 +0100)]
Allow custom sorting of column in the ORM.
To accommodate a change in column ordering used by ORM Declarative in
SQLAlchemy 2.0, a new parameter :paramref:`_orm.mapped_column.sort_order`
has been added that can be used to control the order of the columns defined
in the table by the ORM, for common use cases such as mixins with primary
key columns that should appear first in tables. The change notes at
:ref:`change_9297` illustrate the default change in ordering behavior
(which is part of all SQLAlchemy 2.0 releases) as well as use of the
:paramref:`_orm.mapped_column.sort_order` to control column ordering when
using mixins and multiple classes (new in 2.0.4).
Mike Bayer [Thu, 16 Feb 2023 14:39:07 +0000 (09:39 -0500)]
modernize hybrids and apply typing
Improved the typing support for the :ref:`hybrids_toplevel`
extension, updated all documentation to use ORM Annotated Declarative
mappings, and added a new modifier called :attr:`.hybrid_property.inplace`.
This modifier provides a way to alter the state of a :class:`.hybrid_property`
**in place**, which is essentially what very early versions of hybrids
did, before SQLAlchemy version 1.2.0 :ticket:`3912` changed this to
remove in-place mutation. This in-place mutation is now restored on an
**opt-in** basis to allow a single hybrid to have multiple methods
set up, without the need to name all the methods the same and without the
need to carefully "chain" differently-named methods in order to maintain
the composition. Typing tools such as Mypy and Pyright do not allow
same-named methods on a class, so with this change a succinct method
of setting up hybrids with typing support is restored.
mike bayer [Wed, 15 Feb 2023 22:20:06 +0000 (23:20 +0100)]
Fix coercion issue for tuple bindparams
Fixed issue where element types of a tuple value would be hardcoded to take
on the types from a compared-to tuple, when the comparison were using the
:meth:`.ColumnOperators.in_` operator. This was inconsistent with the usual
way that types are determined for a binary expression, which is that the
actual element type on the right side is considered first before applying
the left-hand-side type.
Mike Bayer [Mon, 13 Feb 2023 16:17:09 +0000 (11:17 -0500)]
immediateload lazy relationships named in refresh.attribute_names
The :meth:`_orm.Session.refresh` method will now immediately load a
relationship-bound attribute that is explicitly named within the
:paramref:`_orm.Session.refresh.attribute_names` collection even if it is
currently linked to the "select" loader, which normally is a "lazy" loader
that does not fire off during a refresh. The "lazy loader" strategy will
now detect that the operation is specifically a user-initiated
:meth:`_orm.Session.refresh` operation which named this attribute
explicitly, and will then call upon the "immediateload" strategy to
actually emit SQL to load the attribute. This should be helpful in
particular for some asyncio situations where the loading of an unloaded
lazy-loaded attribute must be forced, without using the actual lazy-loading
attribute pattern not supported in asyncio.
Mike Bayer [Fri, 10 Feb 2023 21:06:23 +0000 (16:06 -0500)]
add dataclasses callable and apply annotations more strictly
Added new parameter ``dataclasses_callable`` to both the
:class:`_orm.MappedAsDataclass` class as well as the
:meth:`_orm.registry.mapped_as_dataclass` method which allows an
alternative callable to Python ``dataclasses.dataclass`` to be used in
order to produce dataclasses. The use case here is to drop in Pydantic's
dataclass function instead. Adjustments have been made to the mixin support
added for :ticket:`9179` in version 2.0.1 so that the ``__annotations__``
collection of the mixin is rewritten to not include the
:class:`_orm.Mapped` container, in the same way as occurs with mapped
classes, so that the Pydantic dataclasses constructor is not exposed to
unknown types.
Mike Bayer [Mon, 13 Feb 2023 17:23:28 +0000 (12:23 -0500)]
completely lift-and-copy and_ / or_ documentation
Sphinx refuses to write the correct docstring for these without
placing a completely literal string with no interpolation of
any kind. Current site has blank for these.
Fix docs for `case` expression to match new syntax (#9279)
* Fix docs for `case` expression to match new syntax
Previously (before v1.4), the `whens` arg (when `value` is *not* used) used to be a list of
conditions (a 2 item-tuple of condition + value). From v1.4, these are passed
as positional args and the old syntax is not supported anymore.
Mike Bayer [Fri, 10 Feb 2023 13:39:21 +0000 (08:39 -0500)]
generalize adapt_on_names to expect non-named elements
The fix in #9217 opened up adapt_on_names to more kinds of
expressions than it was prepared for; adjust that logic
and also refine in the ORM where we are using it, as we
dont need it (yet) for the DML RETURNING use case.
Fixed regression introduced in version 2.0.2 due to :ticket:`9217` where
using DML RETURNING statements, as well as
:meth:`_sql.Select.from_statement` constructs as was "fixed" in
:ticket:`9217`, in conjunction with ORM mapped classes that used
expressions such as with :func:`_orm.column_property`, would lead to an
internal error within Core where it would attempt to match the expression
by name. The fix repairs the Core issue, and also adjusts the fix in
:ticket:`9217` to not take effect for the DML RETURNING use case, where it
adds unnecessary overhead.
Mike Bayer [Thu, 9 Feb 2023 20:36:38 +0000 (15:36 -0500)]
apply self_group to all elements of multi-expression
Fixed critical regression in SQL expression formulation in the 2.0 series
due to :ticket:`7744` which improved support for SQL expressions that
contained many elements against the same operator repeatedly; parenthesis
grouping would be lost with expression elements beyond the first two
elements.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
