Mike Bayer [Thu, 30 Mar 2023 16:55:45 +0000 (12:55 -0400)]
copy deferred attributes for mapped_column
Fixed issue where the :func:`_orm.mapped_column` construct would raise an
internal error if used on a Declarative mixin and included the
:paramref:`_orm.mapped_column.deferred` parameter.
Mike Bayer [Sun, 26 Mar 2023 16:30:35 +0000 (12:30 -0400)]
disable raise sql for the delete cascade
Towards maintaining consistency with unit-of-work changes made for
:ticket:`5984` and :ticket:`8862`, both of which disable "lazy='raise'"
handling within :class:`_orm.Session` processes that aren't triggered by
attribute access, the :meth:`_orm.Session.delete` method will now also
disable "lazy='raise'" handling when it traverses relationship paths in
order to process the "delete" and "delete-orphan" cascade rules.
Previously, there was no easy way to generically call
:meth:`_orm.Session.delete` on an object that had "lazy='raise'" set up
such that only the necessary relationships would be loaded. As
"lazy='raise'" is primarily intended to catch SQL loading that emits on
attribute access, :meth:`_orm.Session.delete` is now made to behave like
other :class:`_orm.Session` methods including :meth:`_orm.Session.merge` as
well as :meth:`_orm.Session.flush` along with autoflush.
Federico Caselli [Mon, 27 Mar 2023 20:03:44 +0000 (22:03 +0200)]
Improve from_select typing
Properly type :paramref:`_dml.Insert.from_select.names` to accept
a list of string or columns or mapped attributes. Fixes: #9514
Change-Id: I2a3396454e79bd2113fc3d57d1252a2e6d141310
Mike Bayer [Mon, 27 Mar 2023 14:19:32 +0000 (10:19 -0400)]
check for recursion with container types
Fixed issue in ORM Annotated Declarative where using a recursive type (e.g.
using a nested Dict type) would result in a recursion overflow in the ORM's
annotation resolution logic, even if this datatype were not necessary to
map the column.
J. Nick Koston [Sun, 26 Mar 2023 02:00:25 +0000 (22:00 -0400)]
Fix creating zero length char with MySQL dialect
Fixed issue where string datatypes such as :class:`.CHAR`,
:class:`.VARCHAR`, :class:`.TEXT`, as well as binary :class:`.BLOB`, could
not be produced with an explicit length of zero, which has special meaning
for MySQL. Pull request courtesy J. Nick Koston.
Mike Bayer [Mon, 27 Mar 2023 13:48:58 +0000 (09:48 -0400)]
support DeclarativeBase for versioned history example
Fixed issue in "versioned history" example where using a declarative base
that is derived from :class:`_orm.DeclarativeBase` would fail to be mapped.
Additionally, repaired the given test suite so that the documented
instructions for running the example using Python unittest now work again.
Mike Bayer [Fri, 24 Mar 2023 15:11:54 +0000 (11:11 -0400)]
warn for all unmapped expressions
Expanded the warning emitted when a plain :func:`_sql.column` object is
present in a Declarative mapping to include any arbitrary SQL expression
that is not declared within an appropriate property type such as
:func:`_orm.column_property`, :func:`_orm.deferred`, etc. These attributes
are otherwise not mapped at all and remain unchanged within the class
dictionary. As it seems likely that such an expression is usually not
what's intended, this case now warns for all such otherwise ignored
expressions, rather than just the :func:`_sql.column` case.
Mike Bayer [Wed, 22 Mar 2023 15:56:04 +0000 (11:56 -0400)]
use clone, not constructor, in BindParameter.render_literal_execute()
Fixed issue where the :meth:`_sql.BindParameter.render_literal_execute`
method would fail when called on a parameter that also had ORM annotations
associated with it. In practice, this would be observed as a failure of SQL
compilation when using some combinations of a dialect that uses "FETCH
FIRST" such as Oracle along with a :class:`_sql.Select` construct that uses
:meth:`_sql.Select.limit`, within some ORM contexts, including if the
statement were embedded within a relationship primaryjoin expression.
Mike Bayer [Tue, 21 Mar 2023 17:32:41 +0000 (13:32 -0400)]
return None for no parententity on Proxy
Fixed regression where accessing the expression value of a hybrid property
on a class that was either unmapped or not-yet-mapped (such as calling upon
it within a :func:`_orm.declared_attr` method) would raise an internal
error, as an internal fetch for the parent class' mapper would fail and an
instruction for this failure to be ignored were inadvertently removed in
2.0.
Mike Bayer [Sun, 19 Mar 2023 14:37:15 +0000 (10:37 -0400)]
dont render VARCHAR length for PG casts
Fixed critical regression in PostgreSQL dialects such as asyncpg which rely
upon explicit casts in SQL in order for datatypes to be passed to the
driver correctly, where a :class:`.String` datatype would be cast along
with the exact column length being compared, leading to implicit truncation
when comparing a ``VARCHAR`` of a smaller length to a string of greater
length regardless of operator in use (e.g. LIKE, MATCH, etc.). The
PostgreSQL dialect now omits the length from ``VARCHAR`` when rendering
these casts.
Mike Bayer [Sat, 18 Mar 2023 15:43:47 +0000 (11:43 -0400)]
implement content hashing for custom_op, not identity
Fixed critical SQL caching issue where use of the :meth:`_sql.Operators.op`
custom operator function would not produce an appropriate cache key,
leading to reduce the effectiveness of the SQL cache.
Stephen Rosen [Tue, 14 Mar 2023 19:18:14 +0000 (14:18 -0500)]
Document generic type parameters to FunctionElement and GenericFunction (#9079)
* Document type parameters to FunctionElement
Add a note to FunctionElement which indicates that the type is a
typing.Generic class and points at GenericFunction examples for a
specific example usage.
A minimal reference is made to type checkers and IDEs as use-cases in
order to try to contextualize this as an optional feature which
supports particular use cases.
Append to the GenericFunction examples a case which uses `DateTime`
but also includes the generic type parameter (`datetime.datetime`).
### Description
Refactor out the lines in `PGDialect.initialize()` that set backslash escapes into their own method to provide an override hook for [`sqlalchemy-redshift`](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift) to use.
Fixes #9442
### Checklist
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Mon, 13 Mar 2023 16:19:03 +0000 (12:19 -0400)]
simplify AttachedDB test for SQLite
This test is failing on windows due to the new SQlite
provisioning that seemed to be failing to delete schema files
as they are still used by the main connection.
Mike Bayer [Sat, 11 Mar 2023 15:38:44 +0000 (10:38 -0500)]
document no-pep681 workarounds
Mypy 1.1.1 has been released which includes a non-compliant pep-681
implementation that fails with SQLAlchemy's :class:`.MappedAsDataclass` and
similar features. In order to work around this issue until Mypy is able to
release a fix, as well as to support other typing tools which may have
non-compliant pep-681 implementations, document a workaround class
for :class:`.MappedAsDataclass`.
Including this class as well as a decorator was considered, but overall
this is an issue with typing tools that they will have to resolve
and I'm not ready to set up for this issue going on long term. There's
also no good solution for the decorator version since you have to
have an ``__init__`` method indicated somewhere.
Mike Bayer [Thu, 9 Mar 2023 18:54:07 +0000 (13:54 -0500)]
repair broken lambda patch
in I4e0b627bfa187f1780dc68ec81b94db1c78f846a the 1.4 version has more
changes than the main version, which failed to get the entire change,
yet the whole thing was merged. Restore the missing mutex related
code to the main version.
Fixed regression where the fix for :ticket:`8098`, which was released in
the 1.4 series and provided a layer of concurrency-safe checks for the
lambda SQL API, included additional fixes in the patch that failed to be
applied to the main branch. These additional fixes have been applied.
Fix regression when deserializing python rows into cython
Fixed regression involving pickling of Python rows between the cython and
pure Python implementations of :class:`.Row`, which occurred as part of
refactoring code for version 2.0 with typing. A particular constant were
turned into a string based ``Enum`` for the pure Python version of
:class:`.Row` whereas the cython version continued to use an integer
constant, leading to deserialization failures.
Mike Bayer [Thu, 9 Mar 2023 16:49:46 +0000 (11:49 -0500)]
denormalize "public" schema to "PUBLIC"
Fixed reflection bug where Oracle "name normalize" would not work correctly
for reflection of symbols that are in the "PUBLIC" schema, such as
synonyms, meaning the PUBLIC name could not be indicated as lower case on
the Python side for the :paramref:`_schema.Table.schema` argument. Using
uppercase "PUBLIC" would work, but would then lead to awkward SQL queries
including a quoted ``"PUBLIC"`` name as well as indexing the table under
uppercase "PUBLIC", which was inconsistent.
Mike Bayer [Thu, 9 Mar 2023 17:41:03 +0000 (12:41 -0500)]
implement active_history for composites
Fixed bug where the "active history" feature was not fully
implemented for composite attributes, making it impossible to receive
events that included the "old" value. This seems to have been the case
with older SQLAlchemy versions as well, where "active_history" would
be propagated to the underlying column-based attributes, but an event
handler listening to the composite attribute itself would not be given
the "old" value being replaced, even if the composite() were set up
with active_history=True.
Additionally, fixed a regression that's local to 2.0 which disallowed
active_history on composite from being assigned to the impl with
``attr.impl.active_history=True``.
Mike Bayer [Wed, 8 Mar 2023 22:23:31 +0000 (17:23 -0500)]
clarify "selecting individual columns" doc
Just went to refer to this and it was full of difficult terminology
for no good reason. What's troubling is that this doc is like the
tenth time I've rewritten this and it still was loaded with too
much jargon and not clear about the behavior.
Mike Bayer [Wed, 8 Mar 2023 17:35:27 +0000 (12:35 -0500)]
Use independent TypeVar for ColumnElement.cast
Fixed typing issue where :meth:`.ColumnElement.cast` did not allow a
:class:`.TypeEngine` argument independent of the type of the
:class:`.ColumnElement` itself, which is the purpose of
:meth:`.ColumnElement.cast`.
Mike Bayer [Wed, 8 Mar 2023 15:10:14 +0000 (10:10 -0500)]
additional consistency for ORM/Core in tutorial
* Make sure we have blue borders for all sections
* rewrite "blue border" text, refer to textual means of determining
subject matter for a section; "blue borders" are not a primary
source of information
* Add some more intro text that was missing
Mike Bayer [Tue, 7 Mar 2023 14:03:07 +0000 (09:03 -0500)]
resolve select to NULLTYPE if no columns
Fixed regression where the :func:`_sql.select` construct would not be able
to render if it were given no columns and then used in the context of an
EXISTS, raising an internal exception instead. While an empty "SELECT" is
not typically valid SQL, in the context of EXISTS databases such as
PostgreSQL allow it, and in any case the condition now no longer raises
an internal exception.
For this case, also add an extra whitespace trim step for the unusual
case that there are no columns to render. This is done in such a
way as to not interfere with other test cases that are involving
custom compilation schemes.
Mike Bayer [Mon, 6 Mar 2023 03:25:46 +0000 (22:25 -0500)]
comment out pickle/ process test and attempt 2.0.5.post1
if .post1 fails to work out, we'll just do 2.0.6
The test_pickle_rows_other_process test is failing during wheel
builds as it seems that the "subprocess" run is not using the
cython extensions, leading to a pickle mismatch between the
cythonized and the pure python version of a row. comment
out this test and attempt to release as 2.0.5.post1 so that
wheels can build.
Mike Bayer [Sat, 4 Mar 2023 20:31:41 +0000 (15:31 -0500)]
KeyFuncDict regression fixes and dataclass fixes
adapt None-key warning for non-mapped attributes
Fixed multiple regressions due to :ticket:`8372`, involving
:func:`_orm.attribute_mapped_collection` (now called
:func:`_orm.attribute_keyed_dict`).
First, the collection was no longer usable with "key" attributes that were
not themselves ordinary mapped attributes; attributes linked to descriptors
and/or association proxy attributes have been fixed.
Second, if an event or other operation needed access to the "key" in order
to populate the dictionary from an mapped attribute that was not
loaded, this also would raise an error inappropriately, rather than
trying to load the attribute as was the behavior in 1.4. This is also
fixed.
For both cases, the behavior of :ticket:`8372` has been expanded.
:ticket:`8372` introduced an error that raises when the derived key that
would be used as a mapped dictionary key is effectively unassigned. In this
change, a warning only is emitted if the effective value of the ".key"
attribute is ``None``, where it cannot be unambiguously determined if this
``None`` was intentional or not. ``None`` will be not supported as mapped
collection dictionary keys going forward (as it typically refers to NULL
which means "unknown"). Setting
:paramref:`_orm.attribute_keyed_dict.ignore_unpopulated_attribute` will now
cause such ``None`` keys to be ignored as well.
Add value constructors to dictionary collections
Added constructor arguments to the built-in mapping collection types
including :class:`.KeyFuncDict`, :func:`_orm.attribute_keyed_dict`,
:func:`_orm.column_keyed_dict` so that these dictionary types may be
constructed in place given the data up front; this provides further
compatibility with tools such as Python dataclasses ``.asdict()`` which
relies upon invoking these classes directly as ordinary dictionary classes.
Mike Bayer [Fri, 3 Mar 2023 14:30:58 +0000 (09:30 -0500)]
audition pymssql once more; retire sane_rowcount_returning
pymssql seems to be maintained again and seems to be working
completely, so let's try re-enabling it.
Fixed issue in the new :class:`.Uuid` datatype which prevented it from
working with the pymssql driver. As pymssql seems to be maintained again,
restored testing support for pymssql.
Tweaked the pymssql dialect to take better advantage of
RETURNING for INSERT statements in order to retrieve last inserted primary
key values, in the same way as occurs for the mssql+pyodbc dialect right
now.
Identified that the ``sqlite`` and ``mssql+pyodbc`` dialects are now
compatible with the SQLAlchemy ORM's "versioned rows" feature, since
SQLAlchemy now computes rowcount for a RETURNING statement in this specific
case by counting the rows returned, rather than relying upon
``cursor.rowcount``. In particular, the ORM versioned rows use case
(documented at :ref:`mapper_version_counter`) should now be fully
supported with the SQL Server pyodbc dialect.
Nils Philippsen [Tue, 28 Feb 2023 21:04:54 +0000 (16:04 -0500)]
restore old *args approach for MutableDict.pop()
The typing change in ba0e508141206efc55cdab91df21c1
changed the semantics of pop() and possibly setdefault() in order to
try working at runtime with a two-argument signature. however
the implementation for this in cpython likely uses a strict `*args`
approach where the lack of the second parameter is explicit, rather
than matching to a constant. Restore the old implementation inside
of a "not TYPE_CHECKING" block while keeping the type annotated forms
intact for typing only.
Fixed regression caused by typing added to ``sqlalchemy.ext.mutable`` for
:ticket:`8667`, where the semantics of the ``.pop()`` method changed such
that the method was non-working. Pull request courtesy Nils Philippsen.
Mike Bayer [Fri, 24 Feb 2023 21:15:21 +0000 (16:15 -0500)]
ensure event handlers called for all do_ping
The support for pool ping listeners to receive exception events via the
:meth:`.ConnectionEvents.handle_error` event added in 2.0.0b1 for
:ticket:`5648` failed to take into account dialect-specific ping routines
such as that of MySQL and PostgreSQL. The dialect feature has been reworked
so that all dialects participate within event handling. Additionally,
a new boolean element :attr:`.ExceptionContext.is_pre_ping` is added
which identifies if this operation is occurring within the pre-ping
operation.
For this release, third party dialects which implement a custom
:meth:`_engine.Dialect.do_ping` method can opt in to the newly improved
behavior by having their method no longer catch exceptions or check
exceptions for "is_disconnect", instead just propagating all exceptions
outwards. Checking the exception for "is_disconnect" is now done by an
enclosing method on the default dialect, which ensures that the event hook
is invoked for all exception scenarios before testing the exception as a
"disconnect" exception. If an existing ``do_ping()`` method continues to
catch exceptions and check "is_disconnect", it will continue to work as it
did previously, but ``handle_error`` hooks will not have access to the
exception if it isn't propagated outwards.
Mike Bayer [Wed, 1 Mar 2023 16:07:25 +0000 (11:07 -0500)]
TextualSelect is ReturnsRowsRole
Fixed typing bug where :meth:`_sql.Select.from_statement` would not accept
:func:`_sql.text` or :class:`.TextualSelect` objects as a valid type.
Additionally repaired the :class:`.TextClause.columns` method to have a
return type, which was missing.
Mike Bayer [Thu, 2 Mar 2023 01:44:49 +0000 (20:44 -0500)]
allow multiparams with scalars
Fixed bug where the :meth:`_engine.Connection.scalars` method was not typed
as allowing a multiple-parameters list, which is now supported using
insertmanyvalues operations.
easy_markie_tee [Wed, 1 Mar 2023 21:51:43 +0000 (16:51 -0500)]
Fix Typo In Tutorial (#9399)
Missing a word on the page `Working with Database Metadata`.
First paragraph under section 'Setting up MetaData with Table objects'.
"...the database which we query from is know [as] a table."
Co-authored-by: markie tee <cassette.head@gmail.com>
Mike Bayer [Tue, 28 Feb 2023 16:05:48 +0000 (11:05 -0500)]
ensure single import per line
This adds the very small plugin flake8-import-single which
will prevent us from having an import with more than one symbol
on a line.
Flake8 by itself prevents this pattern with E401:
import collections, os, sys
However does not do anything with this:
from sqlalchemy import Column, text
Both statements have the same issues generating merge artifacts
as well as presenting a manual decision to be made. While
zimports generally cleans up such imports at the top level, we
don't enforce zimports / pre-commit use.
the plugin finds the same issue for imports that are inside of
test methods. We shouldn't usually have imports in test methods
so most of them here are moved to be top level.
The version is pinned at 0.1.5; the project seems to have no
activity since 2019, however there are three 0.1.6dev releases
on pypi which stopped in September 2019, they seem to be
experiments with packaging. The source for 0.1.5
is extremely simple and only reveals one method to flake8
(the run() method).
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
