Fix docs for `case` expression to match new syntax (#9279)
* Fix docs for `case` expression to match new syntax
Previously (before v1.4), the `whens` arg (when `value` is *not* used) used to be a list of
conditions (a 2 item-tuple of condition + value). From v1.4, these are passed
as positional args and the old syntax is not supported anymore.
Mike Bayer [Fri, 10 Feb 2023 13:39:21 +0000 (08:39 -0500)]
generalize adapt_on_names to expect non-named elements
The fix in #9217 opened up adapt_on_names to more kinds of
expressions than it was prepared for; adjust that logic
and also refine in the ORM where we are using it, as we
dont need it (yet) for the DML RETURNING use case.
Fixed regression introduced in version 2.0.2 due to :ticket:`9217` where
using DML RETURNING statements, as well as
:meth:`_sql.Select.from_statement` constructs as was "fixed" in
:ticket:`9217`, in conjunction with ORM mapped classes that used
expressions such as with :func:`_orm.column_property`, would lead to an
internal error within Core where it would attempt to match the expression
by name. The fix repairs the Core issue, and also adjusts the fix in
:ticket:`9217` to not take effect for the DML RETURNING use case, where it
adds unnecessary overhead.
Mike Bayer [Thu, 9 Feb 2023 20:36:38 +0000 (15:36 -0500)]
apply self_group to all elements of multi-expression
Fixed critical regression in SQL expression formulation in the 2.0 series
due to :ticket:`7744` which improved support for SQL expressions that
contained many elements against the same operator repeatedly; parenthesis
grouping would be lost with expression elements beyond the first two
elements.
Mike Bayer [Thu, 9 Feb 2023 17:05:47 +0000 (12:05 -0500)]
pin sphinx-copybutton and change config
sphinx-copybutton introduced a new feature
in 0.5.1 which includes a default configuration
that breaks the regexp prompt matching scheme.
set copybutton_exclude to not include ".gp" as that's the class
where we exactly look for the prompts we are matching.
While we're there, use this new feature to exclude our sql
styles, even though this is not strictly necessary in our case.
pin sphinx-copybutton at 0.5.1 to avoid future problems.
Mike Bayer [Sun, 5 Feb 2023 19:12:50 +0000 (14:12 -0500)]
do not return asyncio connections to the pool under gc
Repaired a regression caused by the fix for :ticket:`8419` which caused
asyncpg connections to be reset (i.e. transaction ``rollback()`` called)
and returned to the pool normally in the case that the connection were not
explicitly returned to the connection pool and was instead being
intercepted by Python garbage collection, which would fail if the garbage
collection operation were being called outside of the asyncio event loop,
leading to a large amount of stack trace activity dumped into logging
and standard output.
The correct behavior is restored, which is that all asyncio connections
that are garbage collected due to not being explicitly returned to the
connection pool are detached from the pool and discarded, along with a
warning, rather than being returned the pool, as they cannot be reliably
reset. In the case of asyncpg connections, the asyncpg-specific
``terminate()`` method will be used to end the connection more gracefully
within this process as opposed to just dropping it.
This change includes a small behavioral change that is hoped to be useful
for debugging asyncio applications, where the warning that's emitted in the
case of asyncio connections being unexpectedly garbage collected has been
made slightly more aggressive by moving it outside of a ``try/except``
block and into a ``finally:`` block, where it will emit unconditionally
regardless of whether the detach/termination operation succeeded or not. It
will also have the effect that applications or test suites which promote
Python warnings to exceptions will see this as a full exception raise,
whereas previously it was not possible for this warning to actually
propagate as an exception. Applications and test suites which need to
tolerate this warning in the interim should adjust the Python warnings
filter to allow these warnings to not raise.
The behavior for traditional sync connections remains unchanged, that
garbage collected connections continue to be returned to the pool normally
without emitting a warning. This will likely be changed in a future major
release to at least emit a similar warning as is emitted for asyncio
drivers, as it is a usage error for pooled connections to be intercepted by
garbage collection without being properly returned to the pool.
Mike Bayer [Mon, 6 Feb 2023 18:54:56 +0000 (13:54 -0500)]
add error code 1049 for mysql has_table
Fixed regression caused by issue :ticket:`9058` which adjusted the MySQL
dialect's ``has_table()`` to again use "DESCRIBE", where the specific error
code raised by MySQL version 8 when using a non-existent schema name was
unexpected and failed to be interpreted as a boolean result.
Fixed the SQLite dialect's ``has_table()`` function to correctly report
False for queries that include a non-None schema name for a schema that
doesn't exist; previously, a database error was raised.
jazzthief [Thu, 2 Feb 2023 18:48:13 +0000 (13:48 -0500)]
Dedicated bitwise operators
Added a full suite of new SQL bitwise operators, for performing
database-side bitwise expressions on appropriate data values such as
integers, bit-strings, and similar. Pull request courtesy Yegor Statkevich.
Mike Bayer [Thu, 2 Feb 2023 19:38:37 +0000 (14:38 -0500)]
port history meta to 2.0
first change:
Reworked the :ref:`examples_versioned_history` to work with
version 2.0, while at the same time improving the overall working of
this example to use newer APIs, including a newly added hook
:meth:`_orm.MapperEvents.after_mapper_constructed`.
second change:
Added new event hook :meth:`_orm.MapperEvents.after_mapper_constructed`,
which supplies an event hook to take place right as the
:class:`_orm.Mapper` object has been fully constructed, but before the
:meth:`_orm.registry.configure` call has been called. This allows code that
can create additional mappings and table structures based on the initial
configuration of a :class:`_orm.Mapper`, which also integrates within
Declarative configuration. Previously, when using Declarative, where the
:class:`_orm.Mapper` object is created within the class creation process,
there was no documented means of running code at this point. The change
is to immediately benefit custom mapping schemes such as that
of the :ref:`examples_versioned_history` example, which generate additional
mappers and tables in response to the creation of mapped classes.
third change:
The infrequently used :attr:`_orm.Mapper.iterate_properties` attribute and
:meth:`_orm.Mapper.get_property` method, which are primarily used
internally, no longer implicitly invoke the :meth:`_orm.registry.configure`
process. Public access to these methods is extremely rare and the only
benefit to having :meth:`_orm.registry.configure` would have been allowing
"backref" properties be present in these collections. In order to support
the new :meth:`_orm.MapperEvents.after_mapper_constructed` event, iteration
and access to the internal :class:`_orm.MapperProperty` objects is now
possible without triggering an implicit configure of the mapper itself.
The more-public facing route to iteration of all mapper attributes, the
:attr:`_orm.Mapper.attrs` collection and similar, will still implicitly
invoke the :meth:`_orm.registry.configure` step thus making backref
attributes available.
In all cases, the :meth:`_orm.registry.configure` is always available to
be called directly.
fourth change:
Fixed obscure ORM inheritance issue caused by :ticket:`8705` where some
scenarios of inheriting mappers that indicated groups of columns from the
local table and the inheriting table together under a
:func:`_orm.column_property` would nonetheless warn that properties of the
same name were being combined implicitly.
Caspar Wylie [Fri, 3 Feb 2023 14:23:26 +0000 (09:23 -0500)]
use mysql 8 syntax for ON DUPLICATE KEY UPDATE
Added support for MySQL 8's new ``AS <name> ON DUPLICATE KEY`` syntax when
using :meth:`_mysql.Insert.on_duplicate_key_update`, which is required for
newer versions of MySQL 8 as the previous syntax using ``VALUES()`` now
emits a deprecation warning with those versions. Server version detection
is employed to determine if traditional MariaDB / MySQL < 8 ``VALUES()``
syntax should be used, vs. the newer MySQL 8 required syntax. Pull request
courtesy Caspar Wylie.
Mike Bayer [Mon, 6 Feb 2023 15:16:26 +0000 (10:16 -0500)]
check for superclasses of user defined init
Fixed regression caused by the fix for :ticket:`9171`, which itself was
fixing a regression, involving the mechanics of ``__init__()`` on classes
that extend from :class:`_orm.DeclarativeBase`. The change made it such
that ``__init__()`` was applied to the user-defined base if there were no
``__init__()`` method directly on the class. This has been adjusted so that
``__init__()`` is applied only if no other class in the hierarchy of the
user-defined base has an ``__init__()`` method. This again allows
user-defined base classes based on :class:`_orm.DeclarativeBase` to include
mixins that themselves include a custom ``__init__()`` method.
Mike Bayer [Sat, 4 Feb 2023 21:35:21 +0000 (16:35 -0500)]
coerce elements in mapper.primary_key, process in __mapper_args__
Repaired ORM Declarative mappings to allow for the
:paramref:`_orm.Mapper.primary_key` parameter to be specified within
``__mapper_args__`` when using :func:`_orm.mapped_column`. Despite this
usage being directly in the 2.0 documentation, the :class:`_orm.Mapper` was
not accepting the :func:`_orm.mapped_column` construct in this context. Ths
feature was already working for the :paramref:`_orm.Mapper.version_id_col`
and :paramref:`_orm.Mapper.polymorphic_on` parameters.
As part of this change, the ``__mapper_args__`` attribute may be specified
without using :func:`_orm.declared_attr` on a non-mapped mixin class,
including a ``"primary_key"`` entry that refers to :class:`_schema.Column`
or :func:`_orm.mapped_column` objects locally present on the mixin;
Declarative will also translate these columns into the correct ones for a
particular mapped class. This again was working already for the
:paramref:`_orm.Mapper.version_id_col` and
:paramref:`_orm.Mapper.polymorphic_on` parameters. Additionally,
elements within ``"primary_key"`` may be indicated as string names of
existing mapped properties.
Mike Bayer [Fri, 3 Feb 2023 15:50:14 +0000 (10:50 -0500)]
disallow ORM instrumented attributes from reaching dataclasses
More adjustments to ORM Declarative Dataclasses mappings, building on the
improved support for mixins with dataclasses added in 2.0.1 via
:ticket:`9179`, where a combination of using mixins plus ORM inheritance
would mis-classify fields in some cases leading to their dataclass
arguments such as ``init=False`` being lost.
Mike Bayer [Thu, 2 Feb 2023 22:22:22 +0000 (17:22 -0500)]
dont add non-server-side cols to returning for versioning
Fixed regression where using the :paramref:`_orm.Mapper.version_id_col`
feature with a regular Python-side incrementing column would fail to work
for SQLite and other databases that don't support "rowcount" with
"RETURNING", as "RETURNING" would be assumed for such columns even though
that's not what actually takes place.
Mike Bayer [Wed, 1 Feb 2023 23:16:39 +0000 (18:16 -0500)]
Fixed regression when using from_statement in orm context.
Fixed regression when using :meth:`_sql.Select.from_statement` in an ORM
context, where matching of columns to SQL labels based on name alone was
disabled for ORM-statements that weren't fully textual. This would prevent
arbitrary SQL expressions with column-name labels from matching up to the
entity to be loaded, which previously would work within the 1.4
and previous series, so the previous behavior has been restored.
Frederik Aalund [Mon, 30 Jan 2023 16:50:40 +0000 (11:50 -0500)]
Add support for typing.Literal in Mapped
Added support for :pep:`586` ``Literal`` to be used in the
:paramref:`_orm.registry.type_annotation_map` as well as within
:class:`.Mapped` constructs. To use custom types such as these, they must
appear explicitly within the :paramref:`_orm.registry.type_annotation_map`
to be mapped. Pull request courtesy Frederik Aalund.
As part of this change, the support for :class:`.sqltypes.Enum` in the
:paramref:`_orm.registry.type_annotation_map` has been expanded to include
support for ``Literal[]`` types consisting of string values to be used,
in addition to ``enum.Enum`` datatypes. If a ``Literal[]`` datatype
is used within ``Mapped[]`` that is not linked in
:paramref:`_orm.registry.type_annotation_map` to a specific datatype,
a :class:`.sqltypes.Enum` will be used by default.
Fixed issue involving the use of :class:`.sqltypes.Enum` within the
:paramref:`_orm.registry.type_annotation_map` where the
:paramref:`_sqltypes.Enum.native_enum` parameter would not be correctly
copied to the mapped column datatype, if it were overridden
as stated in the documentation to set this parameter to False.
Harry Lees [Tue, 31 Jan 2023 13:38:34 +0000 (08:38 -0500)]
Unify doc typing
### Description
<!-- Describe your changes in detail -->
Fixes #9168
This PR replaces common occurrences of [PEP 585](https://peps.python.org/pep-0585/) style type annotations with annotations compatible with older versions of Python.
I searched for instances of the following supported types from the PEP and replaced with their legacy typing couterparts.
* tuple # typing.Tuple
* list # typing.List
* dict # typing.Dict
* set # typing.Set
* frozenset # typing.FrozenSet
* type # typing.Type
I excluded changelog files from being altered, I think some of these could be changed if necessary but others are likely to require manual checking as the change may target the new typing style specifically.
For any examples that included imports, I tried to ensure that the correct typing imports were included and properly ordered.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Mon, 30 Jan 2023 20:12:52 +0000 (15:12 -0500)]
support NewType in type_annotation_map
Added support for :pep:`484` ``NewType`` to be used in the
:paramref:`_orm.registry.type_annotation_map` as well as within
:class:`.Mapped` constructs. These types will behave in the same way as
custom subclasses of types right now; they must appear explicitly within
the :paramref:`_orm.registry.type_annotation_map` to be mapped.
Within this change, the lookup between decl_api._resolve_type
and TypeEngine._resolve_for_python_type is streamlined to not
inspect the given type multiple times, instead passing
in from decl_api to TypeEngine the already "flattened" version
of a Generic or NewType type.
Mike Bayer [Mon, 30 Jan 2023 00:51:39 +0000 (19:51 -0500)]
apply of_type error message to load.options() as well
Improved the error reporting when linking strategy options from a base
class to another attribute that's off a subclass, where ``of_type()``
should be used. Previously, when :meth:`.Load.options` is used, the message
would lack informative detail that ``of_type()`` should be used, which was
not the case when linking the options directly. The informative detail now
emits even if :meth:`.Load.options` is used.
Mike Bayer [Sun, 29 Jan 2023 00:50:25 +0000 (19:50 -0500)]
don't count / gather INSERT bind names inside of a CTE
Fixed regression related to the implementation for the new
"insertmanyvalues" feature where an internal ``TypeError`` would occur in
arrangements where a :func:`_sql.insert` would be referred towards inside
of another :func:`_sql.insert` via a CTE; made additional repairs for this
use case for positional dialects such as asyncpg when using
"insertmanyvalues".
at the core here is a change to positional insertmanyvalues
where we now get exactly the positions for the "manyvalues" within
the larger list, allowing non-"manyvalues" on the left and right
sides at the same time, not assuming anything about how RETURNING
renders etc., since CTEs are in the mix also.
When using the :class:`.MappedAsDataclass` superclass, all classes within
the hierarchy that are subclasses of this class will now be run through the
``@dataclasses.dataclass`` function whether or not they are actually
mapped, so that non-ORM fields declared on non-mapped classes within the
hierarchy will be used when mapped subclasses are turned into dataclasses.
This behavior applies both to intermediary classes mapped with
``__abstract__ = True`` as well as to the user-defined declarative base
itself, assuming :class:`.MappedAsDataclass` is present as a superclass for
these classes.
This allows non-mapped attributes such as ``InitVar`` declarations on
superclasses to be used, without the need to run the
``@dataclasses.dataclass`` decorator explicitly on each non-mapped class.
The new behavior is considered as correct as this is what the :pep:`681`
implementation expects when using a superclass to indicate dataclass
behavior.
Mike Bayer [Sun, 29 Jan 2023 15:10:30 +0000 (10:10 -0500)]
derive optional for nullable from interior of pep-593 types
Improved the ruleset used to interpret :pep:`593` ``Annotated`` types when
used with Annotated Declarative mapping, the inner type will be checked for
"Optional" in all cases which will be added to the criteria by which the
column is set as "nullable" or not; if the type within the ``Annotated``
container is optional (or unioned with ``None``), the column will be
considered nullable if there are no explicit
:paramref:`_orm.mapped_column.nullable` parameters overriding it.
Mike Bayer [Sun, 29 Jan 2023 04:41:42 +0000 (23:41 -0500)]
allow single tables and entities for "of"
Opened up typing on :meth:`.Select.with_for_update.of` to also accept table
and mapped class arguments, as seems to be available for the MySQL dialect.
Mike Bayer [Sat, 28 Jan 2023 19:56:17 +0000 (14:56 -0500)]
Place DDLConstraintColumn Role in Mapped
Fixed typing issue where :func:`_orm.mapped_column` objects typed as
:class:`_orm.Mapped` wouldn't be accepted in schema constraints such as
:class:`_schema.ForeignKey`, :class:`_schema.UniqueConstraint` or
:class:`_schema.Index`.
Mike Bayer [Sat, 28 Jan 2023 19:37:33 +0000 (14:37 -0500)]
add __init__ to DeclarativeBase directly
Fixed regression in :class:`.DeclarativeBase` class where the registry's
default constructor would not be applied to the base itself, which is
different from how the previous :func:`_orm.declarative_base` construct
works. This would prevent a mapped class with its own ``__init__()`` method
from calling ``super().__init__()`` in order to access the registry's
default constructor and automatically populate attributes, instead hitting
``object.__init__()`` which would raise a ``TypeError`` on any arguments.
This is a very simple change in code, however explaining it is
very complicated.
Mike Bayer [Sat, 28 Jan 2023 14:37:50 +0000 (09:37 -0500)]
Correct #7664 to include DropSchema
Corrected the fix for :ticket:`7664`, released in version 2.0.0, to also
include :class:`.DropSchema` which was inadvertently missed in this fix,
allowing stringification without a dialect. The fixes for both constructs
is backported to the 1.4 series as of 1.4.47.
Yurii Karabas [Sat, 28 Jan 2023 14:27:35 +0000 (09:27 -0500)]
Set correct type annotations for ColumnElement.cast
<!-- Provide a general summary of your proposed changes in the Title field above -->
Fixes: #9156
### Description
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Fri, 27 Jan 2023 23:15:04 +0000 (18:15 -0500)]
fix regression based on mis-match of set/frozenset
Fixed regression where ORM models that used joined table inheritance with a
composite foreign key would encounter an internal error in the mapper
internals.
Michał Górny [Fri, 27 Jan 2023 08:31:01 +0000 (03:31 -0500)]
remove redundant wheel dep from pyproject.toml
### Description
Remove the redundant `wheel` dependency, as it is added by the backend automatically. Listing it explicitly in the documentation was a historical mistake and has been fixed since, see: https://github.com/pypa/setuptools/commit/f7d30a9529378cf69054b5176249e5457aaf640a
### Checklist
This pull request is:
- [x] A misc build system change (it doesn't really fit the other categories)
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 24 Jan 2023 21:31:01 +0000 (16:31 -0500)]
add hooks/docs for automap w/ multiple schemas
Added new feature to :class:`.Automap` for autoload of classes across
multiple schemas which may have overlapping names, by providing both a
:paramref:`.Automap.prepare.modulename_for_class` parameter as well as a
new collection :attr:`.AutomapBase.by_module`, which stores a dot-separated
namespace of module names linked to classes.
Mike Bayer [Thu, 26 Jan 2023 13:52:01 +0000 (08:52 -0500)]
add typing to legacy operators
Added typing to legacy operators such as ``isnot()``, ``notin_()``, etc.
which previously were referencing the newer operators but were not
themselves typed.
Mike Bayer [Thu, 26 Jan 2023 14:23:07 +0000 (09:23 -0500)]
add Mapped to _ORMColCollectionElement
Fixed issue where using the :paramref:`_orm.relationship.remote_side`
and similar parameters, passing an annotated declarative object typed as
:class:`_orm.Mapped`, would not be accepted by the type checker.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
