Mike Bayer [Mon, 16 Jan 2023 15:31:39 +0000 (10:31 -0500)]
dont assume copy_with() on builtins list, dict, etc; improve error msg.
Fixed issue where using an ``Annotated`` type in the
``type_annotation_map`` which itself contained a plain container type (e.g.
``list``, ``dict``) generic type as the target type would produce an
internal error where the ORM were trying to interpret the ``Annotated``
instance.
Added an error message when a :func:`_orm.relationship` is mapped against
an abstract container type, such as ``Mapped[Sequence[B]]``, without
providing the :paramref:`_orm.relationship.container_class` parameter which
is necessary when the type is abstract. Previously the the abstract
container would attempt to be instantiated and fail.
Mike Bayer [Sun, 15 Jan 2023 18:11:38 +0000 (13:11 -0500)]
super-fine pass through the metadata tutorial
try to keep the wordiness down here, using sidebars
and topics for non-essential information. Sphinx seems
to read out attrs from under TYPE_CHECKING sections now
so link out the attrs in DeclarativeBase w/ docstrings,
not sure why we didn't think of this earlier. looks great
Mike Bayer [Sun, 15 Jan 2023 03:24:36 +0000 (22:24 -0500)]
apply pep-612 to hybrid_method; accept SQLCoreOperations
Fixes to the annotations within the ``sqlalchemy.ext.hybrid`` extension for
more effective typing of user-defined methods. The typing now uses
:pep:`612` features, now supported by recent versions of Mypy, to maintain
argument signatures for :class:`.hybrid_method`. Return values for hybrid
methods are accepted as SQL expressions in contexts such as
:meth:`_sql.Select.where` while still supporting SQL methods.
Mike Bayer [Fri, 13 Jan 2023 22:24:14 +0000 (17:24 -0500)]
implement polymorphic_abstract=True feature
Added a new parameter to :class:`_orm.Mapper` called
:paramref:`_orm.Mapper.polymorphic_abstract`. The purpose of this directive
is so that the ORM will not consider the class to be instantiated or loaded
directly, only subclasses. The actual effect is that the
:class:`_orm.Mapper` will prevent direct instantiation of instances
of the class and will expect that the class does not have a distinct
polymorphic identity configured.
In practice, the class that is mapped with
:paramref:`_orm.Mapper.polymorphic_abstract` can be used as the target of a
:func:`_orm.relationship` as well as be used in queries; subclasses must of
course include polymorphic identities in their mappings.
The new parameter is automatically applied to classes that subclass
the :class:`.AbstractConcreteBase` class, as this class is not intended
to be instantiated.
Additionally, updated some areas of the single table inheritance
documentation to include mapped_column(nullable=False) for all
subclass-only columns; the mappings as given didn't work as the
columns were no longer nullable using Annotated Declarative Table
style.
Mike Bayer [Thu, 12 Jan 2023 16:25:39 +0000 (11:25 -0500)]
add with_loader_criteria() test for #8064 / #9091
test related to #8064, added after discussion #9091 which
requested this behavior for with_loader_criteria() where it was
found to be working as of this issue, just not tested
Mike Bayer [Tue, 10 Jan 2023 14:51:23 +0000 (09:51 -0500)]
fix ORM support for column-named bindparam() in crud .values()
Fixed bug / regression where using :func:`.bindparam()` with the same name
as a column in the :meth:`.Update.values` method of :class:`.Update`, as
well as the :meth:`.Insert.values` method of :class:`.Insert` in 2.0 only,
would in some cases silently fail to honor the SQL expression in which the
parameter were presented, replacing the expression with a new parameter of
the same name and discarding any other elements of the SQL expression, such
as SQL functions, etc. The specific case would be statements that were
constructed against ORM entities rather than plain :class:`.Table`
instances, but would occur if the statement were invoked with a
:class:`.Session` or a :class:`.Connection`.
:class:`.Update` part of the issue was present in both 2.0 and 1.4 and is
backported to 1.4.
Mike Bayer [Thu, 5 Jan 2023 15:34:37 +0000 (10:34 -0500)]
warn and skip for FKs that refer to invisible cols for Oracle
Supported use case for foreign key constraints where the local column is
marked as "invisible". The errors normally generated when a
:class:`.ForeignKeyConstraint` is created that check for the target column
are disabled when reflecting, and the constraint is skipped with a warning
in the same way which already occurs for an :class:`.Index` with a similar
issue.
tests are added for indexes, unique constraints, and primary key
constraints, which were already working; indexes and uniques warn,
primary keys don't which we would assume is because we never see those
PK columns in the first place.
Constraints now raise an informative ConstraintColumnNotFoundError
in the general case for strings in the "pending colargs" collection
not being resolvable.
Mike Bayer [Mon, 9 Jan 2023 13:53:57 +0000 (08:53 -0500)]
accept TableClause through mapped selectable chain
type annotation somehow decided that TableClause doesn't have
primary key fields which is not the case at all. In particular
the "views" recipe relies on TableClause so adding a restriction
like this does not make any sense.
It seems the issue was to open this up for typing, by allowing
TableClause out as far as ddl.sort_tables() typing is passing
for now. Support it out in get_bind() etc.
Mike Bayer [Thu, 5 Jan 2023 04:32:23 +0000 (23:32 -0500)]
revert MySQL to use DESCRIBE for has_table()
Restored the behavior of :meth:`.Inspector.has_table` to report on
temporary tables for MySQL / MariaDB. This is currently the behavior for
all other included dialects, but was removed for MySQL in 1.4 due to no
longer using the DESCRIBE command; there was no documented support for temp
tables being reported by the :meth:`.Inspector.has_table` method in this
version or on any previous version, so the previous behavior was undefined.
As SQLAlchemy 2.0 has added formal support for temp table status via
:meth:`.Inspector.has_table`, the MySQL /MariaDB dialect has been reverted
to use the "DESCRIBE" statement as it did in the SQLAlchemy 1.3 series and
previously, and test support is added to include MySQL / MariaDB for
this behavior. The previous issues with ROLLBACK being emitted which
1.4 sought to improve upon don't apply in SQLAlchemy 2.0 due to
simplifications in how :class:`.Connection` handles transactions.
DESCRIBE is necessary as MariaDB in particular has no consistently
available public information schema of any kind in order to report on temp
tables other than DESCRIBE/SHOW COLUMNS, which rely on throwing an error
in order to report no results.
Michael Gorven [Wed, 4 Jan 2023 17:30:42 +0000 (12:30 -0500)]
[asyncpg] Extract rowcount for SELECT statements
Added support to the asyncpg dialect to return the ``cursor.rowcount``
value for SELECT statements when available. While this is not a typical use
for ``cursor.rowcount``, the other PostgreSQL dialects generally provide
this value. Pull request courtesy Michael Gorven.
Mike Bayer [Tue, 3 Jan 2023 23:21:25 +0000 (18:21 -0500)]
tutorial updates re: Core /ORM commonality
updates for Insert / bulk insert, executemanyvalues,
as well as beginning to describe Table / declared class more
closely together, mentioning typing support.
Fixed a long-standing issue where sphinx would complain about
the Insert symbol being ambiguous.
Gleb Kisenkov [Wed, 28 Dec 2022 19:23:23 +0000 (14:23 -0500)]
Type annotations for sqlalchemy.orm.events
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
An attempt to annotate `lib/sqlalchemy/orm/events.py` with type hints (issue #6810).
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Wed, 28 Dec 2022 17:04:07 +0000 (12:04 -0500)]
ensure whereclause, returning copied as tuples
Fixed issue in the internal SQL traversal for DML statements like
:class:`_dml.Update` and :class:`_dml.Delete` which would cause among other
potential issues, a specific issue using lambda statements with the ORM
update/delete feature.
Mike Bayer [Tue, 20 Dec 2022 17:48:08 +0000 (12:48 -0500)]
establish explicit join transaction modes
The behavior of "joining an external transaction into a Session" has been
revised and improved, allowing explicit control over how the
:class:`_orm.Session` will accommodate an incoming
:class:`_engine.Connection` that already has a transaction and possibly a
savepoint already established. The new parameter
:paramref:`_orm.Session.join_transaction_mode` includes a series of option
values which can accommodate the existing transaction in several ways, most
importantly allowing a :class:`_orm.Session` to operate in a fully
transactional style using savepoints exclusively, while leaving the
externally initiated transaction non-committed and active under all
circumstances, allowing test suites to rollback all changes that take place
within tests.
Additionally, revised the :meth:`_orm.Session.close` method to fully close
out savepoints that may still be present, which also allows the
"external transaction" recipe to proceed without warnings if the
:class:`_orm.Session` did not explicitly end its own SAVEPOINT
transactions.
Mike Bayer [Tue, 27 Dec 2022 17:29:38 +0000 (12:29 -0500)]
pass more contextual information to PyWrapper param create
Fixed issue in lambda SQL feature where the calculated type of a literal
value would not take into account the type coercion rules of the "compared
to type", leading to a lack of typing information for SQL expressions, such
as comparisons to :class:`.JSON` elements and similar.
Mike Bayer [Sun, 18 Dec 2022 21:33:22 +0000 (16:33 -0500)]
reorganize pre_session_exec around do_orm_execute
Allow do_orm_execute() events to both receive the complete
state of bind_argments, load_options, update_delete_options
as they do already, but also allow them to *change* all those
things via new execution options. Options like autoflush,
populate_existing etc. can now be updated within a
do_orm_execute() hook and those changes will take effect
all the way through.
Took a few tries to get something that covers every case here,
in particular horizontal sharding which is consuming those
options as well as using context.invoke(), without excess
complexity. The good news seems to be that a simple
reorg and replacing the "reentrant" boolean with
"is this before do_orm_execute is invoked" was all that was
needed.
As part of this we add a new "identity_token" option allowing
this option to be controlled from do_orm_execute() as well
as from the outside.
Mike Bayer [Fri, 23 Dec 2022 14:09:44 +0000 (09:09 -0500)]
improve async_sessionmaker docs
* illustrate patterns where the async_sessionmaker is being reused.
if you are not reusing it, there is no point to making it
* fix reference to async_sessionmaker
* add typing
Mike Bayer [Thu, 22 Dec 2022 23:14:31 +0000 (18:14 -0500)]
expand out Index if passed to "constraint"
Fixed bug where the PostgreSQL
:paramref:`_postgresql.OnConflictClause.constraint` parameter would accept
an :class:`.Index` object, however would not expand this index out into its
individual index expressions, instead rendering its name in an ON CONFLICT
ON CONSTRAINT clause, which is not accepted by PostgreSQL; the "constraint
name" form only accepts unique or exclude constraint names. The parameter
continues to accept the index but now expands it out into its component
expressions for the render.
Mike Bayer [Wed, 21 Dec 2022 21:10:34 +0000 (16:10 -0500)]
check for adapt to same class in AbstractRange
Fixed regression where newly revised PostgreSQL range types such as
:class:`_postgresql.INT4RANGE` could not be set up as the impl of a
:class:`.TypeDecorator` custom type, instead raising a ``TypeError``.
Mike Bayer [Mon, 19 Dec 2022 20:15:35 +0000 (15:15 -0500)]
add joins_implicitly to column_valued()
Added parameter
:paramref:`.FunctionElement.column_valued.joins_implicitly`, which is
useful in preventing the "cartesian product" warning when making use of
table-valued or column-valued functions. This parameter was already
introduced for :meth:`.FunctionElement.table_valued` in :ticket:`7845`,
however it failed to be added for :meth:`.FunctionElement.column_valued`
as well.
Mike Bayer [Sun, 18 Dec 2022 20:35:39 +0000 (15:35 -0500)]
remove __allow_unmapped__ requirement from dataclasses
Removed the requirement that the ``__allow_unmapped__`` attribute be used
on Declarative Dataclass Mapped class when non-``Mapped[]`` annotations are
detected; previously, an error message that was intended to support legacy
ORM typed mappings would be raised, which additionally did not mention
correct patterns to use with Dataclasses specifically. This error message
is now no longer raised if :meth:`_orm.registry.mapped_as_dataclass` or
:class:`_orm.MappedAsDataclass` is used.
Mike Bayer [Mon, 19 Dec 2022 13:34:51 +0000 (08:34 -0500)]
add exclusion for unusual chars in column names
Added new exclusion rule for third party dialects called
``unusual_column_name_characters``, which can be "closed" for third party
dialects that don't support column names with unusual characters such as
dots, slashes, or percent signs in them, even if the name is properly
quoted.
Mike Bayer [Fri, 16 Dec 2022 20:44:32 +0000 (15:44 -0500)]
include pk cols in refresh() if relationships are requested
A series of changes and improvements regarding
:meth:`_orm.Session.refresh`. The overall change is that primary key
attributes for an object are now included in a refresh operation
unconditionally when relationship-bound attributes are to be refreshed,
even if not expired and even if not specified in the refresh.
* Improved :meth:`_orm.Session.refresh` so that if autoflush is enabled
(as is the default for :class:`_orm.Session`), the autoflush takes place
at an earlier part of the refresh process so that pending primary key
changes are applied without errors being raised. Previously, this
autoflush took place too late in the process and the SELECT statement
would not use the correct key to locate the row and an
:class:`.InvalidRequestError` would be raised.
* When the above condition is present, that is, unflushed primary key
changes are present on the object, but autoflush is not enabled,
the refresh() method now explicitly disallows the operation to proceed,
and an informative :class:`.InvalidRequestError` is raised asking that
the pending primary key changes be flushed first. Previously,
this use case was simply broken and :class:`.InvalidRequestError`
would be raised anyway. This restriction is so that it's safe for the
primary key attributes to be refreshed, as is necessary for the case of
being able to refresh the object with relationship-bound secondary
eagerloaders also being emitted. This rule applies in all cases to keep
API behavior consistent regardless of whether or not the PK cols are
actually needed in the refresh, as it is unusual to be refreshing
some attributes on an object while keeping other attributes "pending"
in any case.
* The :meth:`_orm.Session.refresh` method has been enhanced such that
attributes which are :func:`_orm.relationship`-bound and linked to an
eager loader, either at mapping time or via last-used loader options,
will be refreshed in all cases even when a list of attributes is passed
that does not include any columns on the parent row. This builds upon the
feature first implemented for non-column attributes as part of
:ticket:`1763` fixed in 1.4 allowing eagerly-loaded relationship-bound
attributes to participate in the :meth:`_orm.Session.refresh` operation.
If the refresh operation does not indicate any columns on the parent row
to be refreshed, the primary key columns will nonetheless be included
in the refresh operation, which allows the load to proceed into the
secondary relationship loaders indicated as it does normally.
Previously an :class:`.InvalidRequestError` error would be raised
for this condition (:ticket:`8703`)
* Fixed issue where an unnecessary additional SELECT would be emitted in
the case where :meth:`_orm.Session.refresh` were called with a
combination of expired attributes, as well as an eager loader such as
:func:`_orm.selectinload` that emits a "secondary" query, if the primary
key attributes were also in an expired state. As the primary key
attributes are now included in the refresh automatically, there is no
additional load for these attributes when a relationship loader
goes to select for them (:ticket:`8997`)
* Fixed regression caused by :ticket:`8126` released in 2.0.0b1 where the
:meth:`_orm.Session.refresh` method would fail with an
``AttributeError``, if passed both an expired column name as well as the
name of a relationship-bound attribute that was linked to a "secondary"
eagerloader such as the :func:`_orm.selectinload` eager loader
(:ticket:`8996`)
Mike Bayer [Fri, 16 Dec 2022 19:05:48 +0000 (14:05 -0500)]
dont call platform.architecture()
Fixed regression where the base compat module was calling upon
``platform.architecture()`` in order to detect some system properties,
which results in an over-broad system call against the system-level
``file`` call that is unavailable under some circumstances, including
within some secure environment configurations.
Mike Bayer [Fri, 16 Dec 2022 17:16:21 +0000 (12:16 -0500)]
ensure all visit methods accept **kw
Added test support to ensure that all compiler ``visit_xyz()`` methods
across all :class:`.Compiler` implementations in SQLAlchemy accept a
``**kw`` parameter, so that all compilers accept additional keyword
arguments under all circumstances.
Mike Bayer [Fri, 16 Dec 2022 17:56:21 +0000 (12:56 -0500)]
make bind escape lookup extensible
To accommodate for third party dialects with different character escaping
needs regarding bound parameters, the system by which SQLAlchemy "escapes"
(i.e., replaces with another character in its place) special characters in
bound parameter names has been made extensible for third party dialects,
using the :attr:`.SQLCompiler.bindname_escape_chars` dictionary which can
be overridden at the class declaration level on any :class:`.SQLCompiler`
subclass. As part of this change, also added the dot ``"."`` as a default
"escaped" character.
Mike Bayer [Fri, 9 Dec 2022 00:31:37 +0000 (19:31 -0500)]
add eager_defaults="auto" for inserts
Added a new default value for the :paramref:`.Mapper.eager_defaults`
parameter "auto", which will automatically fetch table default values
during a unit of work flush, if the dialect supports RETURNING for the
INSERT being run, as well as
:ref:`insertmanyvalues <engine_insertmanyvalues>` available. Eager fetches
for server-side UPDATE defaults, which are very uncommon, continue to only
take place if :paramref:`.Mapper.eager_defaults` is set to ``True``, as
there is no batch-RETURNING form for UPDATE statements.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
