Grey Li [Sun, 26 Feb 2023 10:56:37 +0000 (05:56 -0500)]
Add separate version notes for scalars
Add separate 1.4.24 and 1.4.26 version notes for the .scalars method;
this covers Session, scoped_session, AsyncSession, async_scoped_session
as the "scoped" versions did not have the method added until 1.4.26
as part of :ticket:`7103`.
Also indicate scoped_session as ``sqlalchemy.orm.scoped_session`` in
docs rather than ``sqlalchemy.orm.scoping.scoped_session``. This is
also happening in I77da54891860095edcb1f0625ead99fee89bd76f separately,
as both changesets refer to scoped_session without using ".scoping".
Mike Bayer [Sat, 18 Feb 2023 14:10:20 +0000 (09:10 -0500)]
consider column.name directly when evaluating use_existing_column
Fixed issue where new :paramref:`_orm.mapped_column.use_existing_column`
feature would not work if the two same-named columns were mapped under
attribute names that were differently-named from the explicit name given to
the column itself. The attribute names can now be differently named when
using this parameter.
Federico Caselli [Thu, 16 Feb 2023 20:52:18 +0000 (21:52 +0100)]
Allow custom sorting of column in the ORM.
To accommodate a change in column ordering used by ORM Declarative in
SQLAlchemy 2.0, a new parameter :paramref:`_orm.mapped_column.sort_order`
has been added that can be used to control the order of the columns defined
in the table by the ORM, for common use cases such as mixins with primary
key columns that should appear first in tables. The change notes at
:ref:`change_9297` illustrate the default change in ordering behavior
(which is part of all SQLAlchemy 2.0 releases) as well as use of the
:paramref:`_orm.mapped_column.sort_order` to control column ordering when
using mixins and multiple classes (new in 2.0.4).
Mike Bayer [Thu, 16 Feb 2023 14:39:07 +0000 (09:39 -0500)]
modernize hybrids and apply typing
Improved the typing support for the :ref:`hybrids_toplevel`
extension, updated all documentation to use ORM Annotated Declarative
mappings, and added a new modifier called :attr:`.hybrid_property.inplace`.
This modifier provides a way to alter the state of a :class:`.hybrid_property`
**in place**, which is essentially what very early versions of hybrids
did, before SQLAlchemy version 1.2.0 :ticket:`3912` changed this to
remove in-place mutation. This in-place mutation is now restored on an
**opt-in** basis to allow a single hybrid to have multiple methods
set up, without the need to name all the methods the same and without the
need to carefully "chain" differently-named methods in order to maintain
the composition. Typing tools such as Mypy and Pyright do not allow
same-named methods on a class, so with this change a succinct method
of setting up hybrids with typing support is restored.
mike bayer [Wed, 15 Feb 2023 22:20:06 +0000 (23:20 +0100)]
Fix coercion issue for tuple bindparams
Fixed issue where element types of a tuple value would be hardcoded to take
on the types from a compared-to tuple, when the comparison were using the
:meth:`.ColumnOperators.in_` operator. This was inconsistent with the usual
way that types are determined for a binary expression, which is that the
actual element type on the right side is considered first before applying
the left-hand-side type.
Mike Bayer [Mon, 13 Feb 2023 16:17:09 +0000 (11:17 -0500)]
immediateload lazy relationships named in refresh.attribute_names
The :meth:`_orm.Session.refresh` method will now immediately load a
relationship-bound attribute that is explicitly named within the
:paramref:`_orm.Session.refresh.attribute_names` collection even if it is
currently linked to the "select" loader, which normally is a "lazy" loader
that does not fire off during a refresh. The "lazy loader" strategy will
now detect that the operation is specifically a user-initiated
:meth:`_orm.Session.refresh` operation which named this attribute
explicitly, and will then call upon the "immediateload" strategy to
actually emit SQL to load the attribute. This should be helpful in
particular for some asyncio situations where the loading of an unloaded
lazy-loaded attribute must be forced, without using the actual lazy-loading
attribute pattern not supported in asyncio.
Mike Bayer [Fri, 10 Feb 2023 21:06:23 +0000 (16:06 -0500)]
add dataclasses callable and apply annotations more strictly
Added new parameter ``dataclasses_callable`` to both the
:class:`_orm.MappedAsDataclass` class as well as the
:meth:`_orm.registry.mapped_as_dataclass` method which allows an
alternative callable to Python ``dataclasses.dataclass`` to be used in
order to produce dataclasses. The use case here is to drop in Pydantic's
dataclass function instead. Adjustments have been made to the mixin support
added for :ticket:`9179` in version 2.0.1 so that the ``__annotations__``
collection of the mixin is rewritten to not include the
:class:`_orm.Mapped` container, in the same way as occurs with mapped
classes, so that the Pydantic dataclasses constructor is not exposed to
unknown types.
Mike Bayer [Mon, 13 Feb 2023 17:23:28 +0000 (12:23 -0500)]
completely lift-and-copy and_ / or_ documentation
Sphinx refuses to write the correct docstring for these without
placing a completely literal string with no interpolation of
any kind. Current site has blank for these.
Fix docs for `case` expression to match new syntax (#9279)
* Fix docs for `case` expression to match new syntax
Previously (before v1.4), the `whens` arg (when `value` is *not* used) used to be a list of
conditions (a 2 item-tuple of condition + value). From v1.4, these are passed
as positional args and the old syntax is not supported anymore.
Mike Bayer [Fri, 10 Feb 2023 13:39:21 +0000 (08:39 -0500)]
generalize adapt_on_names to expect non-named elements
The fix in #9217 opened up adapt_on_names to more kinds of
expressions than it was prepared for; adjust that logic
and also refine in the ORM where we are using it, as we
dont need it (yet) for the DML RETURNING use case.
Fixed regression introduced in version 2.0.2 due to :ticket:`9217` where
using DML RETURNING statements, as well as
:meth:`_sql.Select.from_statement` constructs as was "fixed" in
:ticket:`9217`, in conjunction with ORM mapped classes that used
expressions such as with :func:`_orm.column_property`, would lead to an
internal error within Core where it would attempt to match the expression
by name. The fix repairs the Core issue, and also adjusts the fix in
:ticket:`9217` to not take effect for the DML RETURNING use case, where it
adds unnecessary overhead.
Mike Bayer [Thu, 9 Feb 2023 20:36:38 +0000 (15:36 -0500)]
apply self_group to all elements of multi-expression
Fixed critical regression in SQL expression formulation in the 2.0 series
due to :ticket:`7744` which improved support for SQL expressions that
contained many elements against the same operator repeatedly; parenthesis
grouping would be lost with expression elements beyond the first two
elements.
Mike Bayer [Thu, 9 Feb 2023 17:05:47 +0000 (12:05 -0500)]
pin sphinx-copybutton and change config
sphinx-copybutton introduced a new feature
in 0.5.1 which includes a default configuration
that breaks the regexp prompt matching scheme.
set copybutton_exclude to not include ".gp" as that's the class
where we exactly look for the prompts we are matching.
While we're there, use this new feature to exclude our sql
styles, even though this is not strictly necessary in our case.
pin sphinx-copybutton at 0.5.1 to avoid future problems.
Mike Bayer [Sun, 5 Feb 2023 19:12:50 +0000 (14:12 -0500)]
do not return asyncio connections to the pool under gc
Repaired a regression caused by the fix for :ticket:`8419` which caused
asyncpg connections to be reset (i.e. transaction ``rollback()`` called)
and returned to the pool normally in the case that the connection were not
explicitly returned to the connection pool and was instead being
intercepted by Python garbage collection, which would fail if the garbage
collection operation were being called outside of the asyncio event loop,
leading to a large amount of stack trace activity dumped into logging
and standard output.
The correct behavior is restored, which is that all asyncio connections
that are garbage collected due to not being explicitly returned to the
connection pool are detached from the pool and discarded, along with a
warning, rather than being returned the pool, as they cannot be reliably
reset. In the case of asyncpg connections, the asyncpg-specific
``terminate()`` method will be used to end the connection more gracefully
within this process as opposed to just dropping it.
This change includes a small behavioral change that is hoped to be useful
for debugging asyncio applications, where the warning that's emitted in the
case of asyncio connections being unexpectedly garbage collected has been
made slightly more aggressive by moving it outside of a ``try/except``
block and into a ``finally:`` block, where it will emit unconditionally
regardless of whether the detach/termination operation succeeded or not. It
will also have the effect that applications or test suites which promote
Python warnings to exceptions will see this as a full exception raise,
whereas previously it was not possible for this warning to actually
propagate as an exception. Applications and test suites which need to
tolerate this warning in the interim should adjust the Python warnings
filter to allow these warnings to not raise.
The behavior for traditional sync connections remains unchanged, that
garbage collected connections continue to be returned to the pool normally
without emitting a warning. This will likely be changed in a future major
release to at least emit a similar warning as is emitted for asyncio
drivers, as it is a usage error for pooled connections to be intercepted by
garbage collection without being properly returned to the pool.
Mike Bayer [Mon, 6 Feb 2023 18:54:56 +0000 (13:54 -0500)]
add error code 1049 for mysql has_table
Fixed regression caused by issue :ticket:`9058` which adjusted the MySQL
dialect's ``has_table()`` to again use "DESCRIBE", where the specific error
code raised by MySQL version 8 when using a non-existent schema name was
unexpected and failed to be interpreted as a boolean result.
Fixed the SQLite dialect's ``has_table()`` function to correctly report
False for queries that include a non-None schema name for a schema that
doesn't exist; previously, a database error was raised.
jazzthief [Thu, 2 Feb 2023 18:48:13 +0000 (13:48 -0500)]
Dedicated bitwise operators
Added a full suite of new SQL bitwise operators, for performing
database-side bitwise expressions on appropriate data values such as
integers, bit-strings, and similar. Pull request courtesy Yegor Statkevich.
Mike Bayer [Thu, 2 Feb 2023 19:38:37 +0000 (14:38 -0500)]
port history meta to 2.0
first change:
Reworked the :ref:`examples_versioned_history` to work with
version 2.0, while at the same time improving the overall working of
this example to use newer APIs, including a newly added hook
:meth:`_orm.MapperEvents.after_mapper_constructed`.
second change:
Added new event hook :meth:`_orm.MapperEvents.after_mapper_constructed`,
which supplies an event hook to take place right as the
:class:`_orm.Mapper` object has been fully constructed, but before the
:meth:`_orm.registry.configure` call has been called. This allows code that
can create additional mappings and table structures based on the initial
configuration of a :class:`_orm.Mapper`, which also integrates within
Declarative configuration. Previously, when using Declarative, where the
:class:`_orm.Mapper` object is created within the class creation process,
there was no documented means of running code at this point. The change
is to immediately benefit custom mapping schemes such as that
of the :ref:`examples_versioned_history` example, which generate additional
mappers and tables in response to the creation of mapped classes.
third change:
The infrequently used :attr:`_orm.Mapper.iterate_properties` attribute and
:meth:`_orm.Mapper.get_property` method, which are primarily used
internally, no longer implicitly invoke the :meth:`_orm.registry.configure`
process. Public access to these methods is extremely rare and the only
benefit to having :meth:`_orm.registry.configure` would have been allowing
"backref" properties be present in these collections. In order to support
the new :meth:`_orm.MapperEvents.after_mapper_constructed` event, iteration
and access to the internal :class:`_orm.MapperProperty` objects is now
possible without triggering an implicit configure of the mapper itself.
The more-public facing route to iteration of all mapper attributes, the
:attr:`_orm.Mapper.attrs` collection and similar, will still implicitly
invoke the :meth:`_orm.registry.configure` step thus making backref
attributes available.
In all cases, the :meth:`_orm.registry.configure` is always available to
be called directly.
fourth change:
Fixed obscure ORM inheritance issue caused by :ticket:`8705` where some
scenarios of inheriting mappers that indicated groups of columns from the
local table and the inheriting table together under a
:func:`_orm.column_property` would nonetheless warn that properties of the
same name were being combined implicitly.
Caspar Wylie [Fri, 3 Feb 2023 14:23:26 +0000 (09:23 -0500)]
use mysql 8 syntax for ON DUPLICATE KEY UPDATE
Added support for MySQL 8's new ``AS <name> ON DUPLICATE KEY`` syntax when
using :meth:`_mysql.Insert.on_duplicate_key_update`, which is required for
newer versions of MySQL 8 as the previous syntax using ``VALUES()`` now
emits a deprecation warning with those versions. Server version detection
is employed to determine if traditional MariaDB / MySQL < 8 ``VALUES()``
syntax should be used, vs. the newer MySQL 8 required syntax. Pull request
courtesy Caspar Wylie.
Mike Bayer [Mon, 6 Feb 2023 15:16:26 +0000 (10:16 -0500)]
check for superclasses of user defined init
Fixed regression caused by the fix for :ticket:`9171`, which itself was
fixing a regression, involving the mechanics of ``__init__()`` on classes
that extend from :class:`_orm.DeclarativeBase`. The change made it such
that ``__init__()`` was applied to the user-defined base if there were no
``__init__()`` method directly on the class. This has been adjusted so that
``__init__()`` is applied only if no other class in the hierarchy of the
user-defined base has an ``__init__()`` method. This again allows
user-defined base classes based on :class:`_orm.DeclarativeBase` to include
mixins that themselves include a custom ``__init__()`` method.
Mike Bayer [Sat, 4 Feb 2023 21:35:21 +0000 (16:35 -0500)]
coerce elements in mapper.primary_key, process in __mapper_args__
Repaired ORM Declarative mappings to allow for the
:paramref:`_orm.Mapper.primary_key` parameter to be specified within
``__mapper_args__`` when using :func:`_orm.mapped_column`. Despite this
usage being directly in the 2.0 documentation, the :class:`_orm.Mapper` was
not accepting the :func:`_orm.mapped_column` construct in this context. Ths
feature was already working for the :paramref:`_orm.Mapper.version_id_col`
and :paramref:`_orm.Mapper.polymorphic_on` parameters.
As part of this change, the ``__mapper_args__`` attribute may be specified
without using :func:`_orm.declared_attr` on a non-mapped mixin class,
including a ``"primary_key"`` entry that refers to :class:`_schema.Column`
or :func:`_orm.mapped_column` objects locally present on the mixin;
Declarative will also translate these columns into the correct ones for a
particular mapped class. This again was working already for the
:paramref:`_orm.Mapper.version_id_col` and
:paramref:`_orm.Mapper.polymorphic_on` parameters. Additionally,
elements within ``"primary_key"`` may be indicated as string names of
existing mapped properties.
Mike Bayer [Fri, 3 Feb 2023 15:50:14 +0000 (10:50 -0500)]
disallow ORM instrumented attributes from reaching dataclasses
More adjustments to ORM Declarative Dataclasses mappings, building on the
improved support for mixins with dataclasses added in 2.0.1 via
:ticket:`9179`, where a combination of using mixins plus ORM inheritance
would mis-classify fields in some cases leading to their dataclass
arguments such as ``init=False`` being lost.
Mike Bayer [Thu, 2 Feb 2023 22:22:22 +0000 (17:22 -0500)]
dont add non-server-side cols to returning for versioning
Fixed regression where using the :paramref:`_orm.Mapper.version_id_col`
feature with a regular Python-side incrementing column would fail to work
for SQLite and other databases that don't support "rowcount" with
"RETURNING", as "RETURNING" would be assumed for such columns even though
that's not what actually takes place.
Mike Bayer [Wed, 1 Feb 2023 23:16:39 +0000 (18:16 -0500)]
Fixed regression when using from_statement in orm context.
Fixed regression when using :meth:`_sql.Select.from_statement` in an ORM
context, where matching of columns to SQL labels based on name alone was
disabled for ORM-statements that weren't fully textual. This would prevent
arbitrary SQL expressions with column-name labels from matching up to the
entity to be loaded, which previously would work within the 1.4
and previous series, so the previous behavior has been restored.
Frederik Aalund [Mon, 30 Jan 2023 16:50:40 +0000 (11:50 -0500)]
Add support for typing.Literal in Mapped
Added support for :pep:`586` ``Literal`` to be used in the
:paramref:`_orm.registry.type_annotation_map` as well as within
:class:`.Mapped` constructs. To use custom types such as these, they must
appear explicitly within the :paramref:`_orm.registry.type_annotation_map`
to be mapped. Pull request courtesy Frederik Aalund.
As part of this change, the support for :class:`.sqltypes.Enum` in the
:paramref:`_orm.registry.type_annotation_map` has been expanded to include
support for ``Literal[]`` types consisting of string values to be used,
in addition to ``enum.Enum`` datatypes. If a ``Literal[]`` datatype
is used within ``Mapped[]`` that is not linked in
:paramref:`_orm.registry.type_annotation_map` to a specific datatype,
a :class:`.sqltypes.Enum` will be used by default.
Fixed issue involving the use of :class:`.sqltypes.Enum` within the
:paramref:`_orm.registry.type_annotation_map` where the
:paramref:`_sqltypes.Enum.native_enum` parameter would not be correctly
copied to the mapped column datatype, if it were overridden
as stated in the documentation to set this parameter to False.
Harry Lees [Tue, 31 Jan 2023 13:38:34 +0000 (08:38 -0500)]
Unify doc typing
### Description
<!-- Describe your changes in detail -->
Fixes #9168
This PR replaces common occurrences of [PEP 585](https://peps.python.org/pep-0585/) style type annotations with annotations compatible with older versions of Python.
I searched for instances of the following supported types from the PEP and replaced with their legacy typing couterparts.
* tuple # typing.Tuple
* list # typing.List
* dict # typing.Dict
* set # typing.Set
* frozenset # typing.FrozenSet
* type # typing.Type
I excluded changelog files from being altered, I think some of these could be changed if necessary but others are likely to require manual checking as the change may target the new typing style specifically.
For any examples that included imports, I tried to ensure that the correct typing imports were included and properly ordered.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
