Stephen Rosen [Tue, 14 Mar 2023 19:18:14 +0000 (14:18 -0500)]
Document generic type parameters to FunctionElement and GenericFunction (#9079)
* Document type parameters to FunctionElement
Add a note to FunctionElement which indicates that the type is a
typing.Generic class and points at GenericFunction examples for a
specific example usage.
A minimal reference is made to type checkers and IDEs as use-cases in
order to try to contextualize this as an optional feature which
supports particular use cases.
Append to the GenericFunction examples a case which uses `DateTime`
but also includes the generic type parameter (`datetime.datetime`).
### Description
Refactor out the lines in `PGDialect.initialize()` that set backslash escapes into their own method to provide an override hook for [`sqlalchemy-redshift`](https://github.com/sqlalchemy-redshift/sqlalchemy-redshift) to use.
Fixes #9442
### Checklist
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Mon, 13 Mar 2023 16:19:03 +0000 (12:19 -0400)]
simplify AttachedDB test for SQLite
This test is failing on windows due to the new SQlite
provisioning that seemed to be failing to delete schema files
as they are still used by the main connection.
Mike Bayer [Sat, 11 Mar 2023 15:38:44 +0000 (10:38 -0500)]
document no-pep681 workarounds
Mypy 1.1.1 has been released which includes a non-compliant pep-681
implementation that fails with SQLAlchemy's :class:`.MappedAsDataclass` and
similar features. In order to work around this issue until Mypy is able to
release a fix, as well as to support other typing tools which may have
non-compliant pep-681 implementations, document a workaround class
for :class:`.MappedAsDataclass`.
Including this class as well as a decorator was considered, but overall
this is an issue with typing tools that they will have to resolve
and I'm not ready to set up for this issue going on long term. There's
also no good solution for the decorator version since you have to
have an ``__init__`` method indicated somewhere.
Mike Bayer [Thu, 9 Mar 2023 18:54:07 +0000 (13:54 -0500)]
repair broken lambda patch
in I4e0b627bfa187f1780dc68ec81b94db1c78f846a the 1.4 version has more
changes than the main version, which failed to get the entire change,
yet the whole thing was merged. Restore the missing mutex related
code to the main version.
Fixed regression where the fix for :ticket:`8098`, which was released in
the 1.4 series and provided a layer of concurrency-safe checks for the
lambda SQL API, included additional fixes in the patch that failed to be
applied to the main branch. These additional fixes have been applied.
Fix regression when deserializing python rows into cython
Fixed regression involving pickling of Python rows between the cython and
pure Python implementations of :class:`.Row`, which occurred as part of
refactoring code for version 2.0 with typing. A particular constant were
turned into a string based ``Enum`` for the pure Python version of
:class:`.Row` whereas the cython version continued to use an integer
constant, leading to deserialization failures.
Mike Bayer [Thu, 9 Mar 2023 16:49:46 +0000 (11:49 -0500)]
denormalize "public" schema to "PUBLIC"
Fixed reflection bug where Oracle "name normalize" would not work correctly
for reflection of symbols that are in the "PUBLIC" schema, such as
synonyms, meaning the PUBLIC name could not be indicated as lower case on
the Python side for the :paramref:`_schema.Table.schema` argument. Using
uppercase "PUBLIC" would work, but would then lead to awkward SQL queries
including a quoted ``"PUBLIC"`` name as well as indexing the table under
uppercase "PUBLIC", which was inconsistent.
Mike Bayer [Thu, 9 Mar 2023 17:41:03 +0000 (12:41 -0500)]
implement active_history for composites
Fixed bug where the "active history" feature was not fully
implemented for composite attributes, making it impossible to receive
events that included the "old" value. This seems to have been the case
with older SQLAlchemy versions as well, where "active_history" would
be propagated to the underlying column-based attributes, but an event
handler listening to the composite attribute itself would not be given
the "old" value being replaced, even if the composite() were set up
with active_history=True.
Additionally, fixed a regression that's local to 2.0 which disallowed
active_history on composite from being assigned to the impl with
``attr.impl.active_history=True``.
Mike Bayer [Wed, 8 Mar 2023 22:23:31 +0000 (17:23 -0500)]
clarify "selecting individual columns" doc
Just went to refer to this and it was full of difficult terminology
for no good reason. What's troubling is that this doc is like the
tenth time I've rewritten this and it still was loaded with too
much jargon and not clear about the behavior.
Mike Bayer [Wed, 8 Mar 2023 17:35:27 +0000 (12:35 -0500)]
Use independent TypeVar for ColumnElement.cast
Fixed typing issue where :meth:`.ColumnElement.cast` did not allow a
:class:`.TypeEngine` argument independent of the type of the
:class:`.ColumnElement` itself, which is the purpose of
:meth:`.ColumnElement.cast`.
Mike Bayer [Wed, 8 Mar 2023 15:10:14 +0000 (10:10 -0500)]
additional consistency for ORM/Core in tutorial
* Make sure we have blue borders for all sections
* rewrite "blue border" text, refer to textual means of determining
subject matter for a section; "blue borders" are not a primary
source of information
* Add some more intro text that was missing
Mike Bayer [Tue, 7 Mar 2023 14:03:07 +0000 (09:03 -0500)]
resolve select to NULLTYPE if no columns
Fixed regression where the :func:`_sql.select` construct would not be able
to render if it were given no columns and then used in the context of an
EXISTS, raising an internal exception instead. While an empty "SELECT" is
not typically valid SQL, in the context of EXISTS databases such as
PostgreSQL allow it, and in any case the condition now no longer raises
an internal exception.
For this case, also add an extra whitespace trim step for the unusual
case that there are no columns to render. This is done in such a
way as to not interfere with other test cases that are involving
custom compilation schemes.
Mike Bayer [Mon, 6 Mar 2023 03:25:46 +0000 (22:25 -0500)]
comment out pickle/ process test and attempt 2.0.5.post1
if .post1 fails to work out, we'll just do 2.0.6
The test_pickle_rows_other_process test is failing during wheel
builds as it seems that the "subprocess" run is not using the
cython extensions, leading to a pickle mismatch between the
cythonized and the pure python version of a row. comment
out this test and attempt to release as 2.0.5.post1 so that
wheels can build.
Mike Bayer [Sat, 4 Mar 2023 20:31:41 +0000 (15:31 -0500)]
KeyFuncDict regression fixes and dataclass fixes
adapt None-key warning for non-mapped attributes
Fixed multiple regressions due to :ticket:`8372`, involving
:func:`_orm.attribute_mapped_collection` (now called
:func:`_orm.attribute_keyed_dict`).
First, the collection was no longer usable with "key" attributes that were
not themselves ordinary mapped attributes; attributes linked to descriptors
and/or association proxy attributes have been fixed.
Second, if an event or other operation needed access to the "key" in order
to populate the dictionary from an mapped attribute that was not
loaded, this also would raise an error inappropriately, rather than
trying to load the attribute as was the behavior in 1.4. This is also
fixed.
For both cases, the behavior of :ticket:`8372` has been expanded.
:ticket:`8372` introduced an error that raises when the derived key that
would be used as a mapped dictionary key is effectively unassigned. In this
change, a warning only is emitted if the effective value of the ".key"
attribute is ``None``, where it cannot be unambiguously determined if this
``None`` was intentional or not. ``None`` will be not supported as mapped
collection dictionary keys going forward (as it typically refers to NULL
which means "unknown"). Setting
:paramref:`_orm.attribute_keyed_dict.ignore_unpopulated_attribute` will now
cause such ``None`` keys to be ignored as well.
Add value constructors to dictionary collections
Added constructor arguments to the built-in mapping collection types
including :class:`.KeyFuncDict`, :func:`_orm.attribute_keyed_dict`,
:func:`_orm.column_keyed_dict` so that these dictionary types may be
constructed in place given the data up front; this provides further
compatibility with tools such as Python dataclasses ``.asdict()`` which
relies upon invoking these classes directly as ordinary dictionary classes.
Mike Bayer [Fri, 3 Mar 2023 14:30:58 +0000 (09:30 -0500)]
audition pymssql once more; retire sane_rowcount_returning
pymssql seems to be maintained again and seems to be working
completely, so let's try re-enabling it.
Fixed issue in the new :class:`.Uuid` datatype which prevented it from
working with the pymssql driver. As pymssql seems to be maintained again,
restored testing support for pymssql.
Tweaked the pymssql dialect to take better advantage of
RETURNING for INSERT statements in order to retrieve last inserted primary
key values, in the same way as occurs for the mssql+pyodbc dialect right
now.
Identified that the ``sqlite`` and ``mssql+pyodbc`` dialects are now
compatible with the SQLAlchemy ORM's "versioned rows" feature, since
SQLAlchemy now computes rowcount for a RETURNING statement in this specific
case by counting the rows returned, rather than relying upon
``cursor.rowcount``. In particular, the ORM versioned rows use case
(documented at :ref:`mapper_version_counter`) should now be fully
supported with the SQL Server pyodbc dialect.
Nils Philippsen [Tue, 28 Feb 2023 21:04:54 +0000 (16:04 -0500)]
restore old *args approach for MutableDict.pop()
The typing change in ba0e508141206efc55cdab91df21c1
changed the semantics of pop() and possibly setdefault() in order to
try working at runtime with a two-argument signature. however
the implementation for this in cpython likely uses a strict `*args`
approach where the lack of the second parameter is explicit, rather
than matching to a constant. Restore the old implementation inside
of a "not TYPE_CHECKING" block while keeping the type annotated forms
intact for typing only.
Fixed regression caused by typing added to ``sqlalchemy.ext.mutable`` for
:ticket:`8667`, where the semantics of the ``.pop()`` method changed such
that the method was non-working. Pull request courtesy Nils Philippsen.
Mike Bayer [Fri, 24 Feb 2023 21:15:21 +0000 (16:15 -0500)]
ensure event handlers called for all do_ping
The support for pool ping listeners to receive exception events via the
:meth:`.ConnectionEvents.handle_error` event added in 2.0.0b1 for
:ticket:`5648` failed to take into account dialect-specific ping routines
such as that of MySQL and PostgreSQL. The dialect feature has been reworked
so that all dialects participate within event handling. Additionally,
a new boolean element :attr:`.ExceptionContext.is_pre_ping` is added
which identifies if this operation is occurring within the pre-ping
operation.
For this release, third party dialects which implement a custom
:meth:`_engine.Dialect.do_ping` method can opt in to the newly improved
behavior by having their method no longer catch exceptions or check
exceptions for "is_disconnect", instead just propagating all exceptions
outwards. Checking the exception for "is_disconnect" is now done by an
enclosing method on the default dialect, which ensures that the event hook
is invoked for all exception scenarios before testing the exception as a
"disconnect" exception. If an existing ``do_ping()`` method continues to
catch exceptions and check "is_disconnect", it will continue to work as it
did previously, but ``handle_error`` hooks will not have access to the
exception if it isn't propagated outwards.
Mike Bayer [Wed, 1 Mar 2023 16:07:25 +0000 (11:07 -0500)]
TextualSelect is ReturnsRowsRole
Fixed typing bug where :meth:`_sql.Select.from_statement` would not accept
:func:`_sql.text` or :class:`.TextualSelect` objects as a valid type.
Additionally repaired the :class:`.TextClause.columns` method to have a
return type, which was missing.
Mike Bayer [Thu, 2 Mar 2023 01:44:49 +0000 (20:44 -0500)]
allow multiparams with scalars
Fixed bug where the :meth:`_engine.Connection.scalars` method was not typed
as allowing a multiple-parameters list, which is now supported using
insertmanyvalues operations.
easy_markie_tee [Wed, 1 Mar 2023 21:51:43 +0000 (16:51 -0500)]
Fix Typo In Tutorial (#9399)
Missing a word on the page `Working with Database Metadata`.
First paragraph under section 'Setting up MetaData with Table objects'.
"...the database which we query from is know [as] a table."
Co-authored-by: markie tee <cassette.head@gmail.com>
Mike Bayer [Tue, 28 Feb 2023 16:05:48 +0000 (11:05 -0500)]
ensure single import per line
This adds the very small plugin flake8-import-single which
will prevent us from having an import with more than one symbol
on a line.
Flake8 by itself prevents this pattern with E401:
import collections, os, sys
However does not do anything with this:
from sqlalchemy import Column, text
Both statements have the same issues generating merge artifacts
as well as presenting a manual decision to be made. While
zimports generally cleans up such imports at the top level, we
don't enforce zimports / pre-commit use.
the plugin finds the same issue for imports that are inside of
test methods. We shouldn't usually have imports in test methods
so most of them here are moved to be top level.
The version is pinned at 0.1.5; the project seems to have no
activity since 2019, however there are three 0.1.6dev releases
on pypi which stopped in September 2019, they seem to be
experiments with packaging. The source for 0.1.5
is extremely simple and only reveals one method to flake8
(the run() method).
Mike Bayer [Sun, 26 Feb 2023 14:31:36 +0000 (09:31 -0500)]
include columns from superclasses that indicate "selectin"
Added support for the :paramref:`_orm.Mapper.polymorphic_load` parameter to
be applied to each mapper in an inheritance hierarchy more than one level
deep, allowing columns to load for all classes in the hierarchy that
indicate ``"selectin"`` using a single statement, rather than ignoring
elements on those intermediary classes that nonetheless indicate they also
would participate in ``"selectin"`` loading and were not part of the
base-most SELECT statement.
Grey Li [Sun, 26 Feb 2023 10:56:37 +0000 (05:56 -0500)]
Add separate version notes for scalars
Add separate 1.4.24 and 1.4.26 version notes for the .scalars method;
this covers Session, scoped_session, AsyncSession, async_scoped_session
as the "scoped" versions did not have the method added until 1.4.26
as part of :ticket:`7103`.
Also indicate scoped_session as ``sqlalchemy.orm.scoped_session`` in
docs rather than ``sqlalchemy.orm.scoping.scoped_session``. This is
also happening in I77da54891860095edcb1f0625ead99fee89bd76f separately,
as both changesets refer to scoped_session without using ".scoping".
Matus Valo [Sun, 26 Feb 2023 20:10:23 +0000 (15:10 -0500)]
Declare KEY_OBJECTS_ONLY as cdef variable
A small optimization to the Cython implementation of :class:`.ResultProxy`
using a cdef for a particular int value to avoid Python overhead. Pull
request courtesy Matus Valo.
Mike Bayer [Tue, 21 Feb 2023 15:34:01 +0000 (10:34 -0500)]
apply a fixed locals w/ Mapped to all de-stringify
Continued the fix for :ticket:`8853`, allowing the :class:`_orm.Mapped`
name to be fully qualified regardless of whether or not
``from __annotations__ import future`` were present. This issue first fixed
in 2.0.0b3 confirmed that this case worked via the test suite, however the
test suite apparently was not testing the behavior for the name ``Mapped``
not being locally present at all; string resolution has been updated to
ensure the ``Mapped`` symbol is locatable as applies to how the ORM uses
these functions.
Mike Bayer [Sun, 26 Feb 2023 20:34:57 +0000 (15:34 -0500)]
use read-only Mapping for values dictionary type
Improved typing for the mapping passed to :meth:`.UpdateBase.values` to be
more open-ended about collection type, by indicating read-only ``Mapping``
instead of writeable ``Dict``, the latter of which would error out under
typing tools on too limited of a key type.
Mike Bayer [Fri, 24 Feb 2023 15:58:25 +0000 (10:58 -0500)]
locate automap base in hierarchy directly
Fixed issue in automap where calling ``.prepare()`` from one of the mapped
classes would not use the correct base class when automap detected new
tables, instead using the given class, leading to mappers trying to
configure inheritance. While one should normally call ``.prepare()`` from
the base in any case, it shouldn't misbehave that badly when called from a
subclass.
Mike Bayer [Sat, 18 Feb 2023 14:10:20 +0000 (09:10 -0500)]
consider column.name directly when evaluating use_existing_column
Fixed issue where new :paramref:`_orm.mapped_column.use_existing_column`
feature would not work if the two same-named columns were mapped under
attribute names that were differently-named from the explicit name given to
the column itself. The attribute names can now be differently named when
using this parameter.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
