Mike Bayer [Sun, 23 Jan 2022 14:28:22 +0000 (09:28 -0500)]
dont test squelched warnings against the filter
I spent days on Ibcf09af25228d39ee5a943fda82d8a9302433726
reading it over and over again and noticed this slight inaccuracy
10 seconds after I merged it.
the assert_warns_message() and assert_warns() functions should
not consider a mismatched warning class as valid for a match.
Mike Bayer [Sun, 23 Jan 2022 14:14:00 +0000 (09:14 -0500)]
make the autoincrement doc a lot more clear
I saw it saying "this parameter has no effect when
Identity is in use", which is not accurate, so I've expanded
this way out with bullets to make the current situation
as clear as possible.
Mike Bayer [Fri, 21 Jan 2022 23:46:37 +0000 (18:46 -0500)]
dont use exception catches for warnings; modernize xdist detection
Improvements to the test suite's integration with pytest such that the
"warnings" plugin, if manually enabled, will not interfere with the test
suite, such that third parties can enable the warnings plugin or make use
of the ``-W`` parameter and SQLAlchemy's test suite will continue to pass.
Additionally, modernized the detection of the "pytest-xdist" plugin so that
plugins can be globally disabled using PYTEST_DISABLE_PLUGIN_AUTOLOAD=1
without breaking the test suite if xdist were still installed. Warning
filters that promote deprecation warnings to errors are now localized to
SQLAlchemy-specific warnings, or within SQLAlchemy-specific sources for
general Python deprecation warnings, so that non-SQLAlchemy deprecation
warnings emitted from pytest plugins should also not impact the test suite.
Identified a bit of cleanup for the PostgreSQL provisioning
as a result.
Mike Bayer [Fri, 21 Jan 2022 15:19:02 +0000 (10:19 -0500)]
Skip PK returned as None for RETURNING, server side default
Fixed regression where the ORM exception that is to be raised when an
INSERT silently fails to actually insert a row (such as from a trigger)
would not be reached, due to a runtime exception raised ahead of time due
to the missing primary key value, thus raising an uninformative exception
rather than the correct one. For 1.4 and above, a new ``FlushError`` is
added for this case that's raised earlier than the previous "null identity"
exception was for 1.3, as a situation where the number of rows actually
INSERTed does not match what was expected is a more critical situation in
1.4 as it prevents batching of multiple objects from working correctly.
This is separate from the case where a newly fetched primary key is
fetched as NULL, which continues to raise the existing "null identity"
exception.
Mike Bayer [Thu, 20 Jan 2022 20:21:17 +0000 (15:21 -0500)]
re-enable tests for asyncmy; fix Binary
Fixed regression in asyncmy dialect caused by :ticket:`7567` where removal
of the PyMySQL dependency broke binary columns, due to the asyncmy dialect
not being properly included within CI tests.
Mike Bayer [Thu, 20 Jan 2022 20:01:47 +0000 (15:01 -0500)]
restore empty list logic to ARRAY of ENUM parsing
Fixed regression where the change in :ticket:`7148` to repair ENUM handling
in PostgreSQL broke the use case of an empty ARRAY of ENUM, preventing rows
that contained an empty array from being handled correctly when fetching
results.
Mike Bayer [Thu, 20 Jan 2022 14:31:42 +0000 (09:31 -0500)]
repair mapper sort
Fixed issue in :meth:`_orm.Session.bulk_save_mappings` where the sorting
that takes place when the ``preserve_order`` parameter is set to False
would sort partially on ``Mapper`` objects, which is rejected in Python
3.11.
Also uses typing_extensions for NotRequired as this symbol
does not seem to be in Python 3.11.0a4 yet. (2.0 only)
Mike Bayer [Wed, 19 Jan 2022 19:31:52 +0000 (14:31 -0500)]
Add AdaptedConnection.run_async
Added new method :meth:`.AdaptedConnection.run_async` to the DBAPI
connection interface used by asyncio drivers, which allows methods to be
called against the underlying "driver" connection directly within a
sync-style function where the ``await`` keyword can't be used, such as
within SQLAlchemy event handler functions. The method is analogous to the
:meth:`_asyncio.AsyncConnection.run_sync` method which translates
async-style calls to sync-style. The method is useful for things like
connection-pool on-connect handlers that need to invoke awaitable methods
on the driver connection when it's first created.
Mike Bayer [Tue, 18 Jan 2022 22:19:24 +0000 (17:19 -0500)]
reject methods as lambda SQL callables
Added an informative error message when a method object is passed to a SQL
construct. Previously, when such a callable were passed, as is a common
typographical error when dealing with method-chained SQL constructs, they
were interpreted as "lambda SQL" targets to be invoked at compilation time,
which would lead to silent failures. As this feature was not intended to be
used with methods, method objects are now rejected.
Mike Bayer [Sun, 16 Jan 2022 15:21:45 +0000 (10:21 -0500)]
enhance double-aliased table logic to handle more cases
Fixed ORM regression where calling the :func:`_orm.aliased` function
against an existing :func:`_orm.aliased` construct would fail to produce
correct SQL if the existing construct were against a fixed table. The fix
allows that the original :func:`_orm.aliased` construct is disregarded if
it were only against a table that's now being replaced. It also allows for
correct behavior when constructing a :func:`_orm.aliased` without a
selectable argument against a :func:`_orm.aliased` that's against a
subuquery, to create an alias of that subquery (i.e. to change its name).
The nesting behavior of :func:`_orm.aliased` remains in place for the case
where the outer :func:`_orm.aliased` object is against a subquery which in
turn refers to the inner :func:`_orm.aliased` object. This is a relatively
new 1.4 feature that helps to suit use cases that were previously served by
the deprecated ``Query.from_self()`` method.
Mike Bayer [Tue, 18 Jan 2022 16:02:57 +0000 (11:02 -0500)]
detect map_imperatively() called twice
Fixed issue where calling upon :meth:`_orm.regsitry.map_imperatively` more
than once for the same class would produce an unexpected error, rather than
an informative error that the target class is already mapped. This behavior
differed from that of the :func:`_orm.mapper` function which does report an
informative message already.
For 2.0, this change also cleans up the logic that detects against
`Mapper()` or `_mapper()` being invoked directly. 1.4's backport will
take on a different format as `mapper()` is still public API in that
release.
José Duarte [Thu, 13 Jan 2022 22:20:06 +0000 (17:20 -0500)]
Fixes(#7561) Add support for postgres.UUID literal_binds compilation
Added string rendering to the :class:`.postgresql.UUID` datatype, so that
stringifying a statement with "literal_binds" that uses this type will
render an appropriate string value for the PostgreSQL backend. Pull request
courtesy José Duarte.
Mike Bayer [Wed, 5 Jan 2022 17:20:46 +0000 (12:20 -0500)]
implement second-level type resolution for literals
Added additional rule to the system that determines ``TypeEngine``
implementations from Python literals to apply a second level of adjustment
to the type, so that a Python datetime with or without tzinfo can set the
``timezone=True`` parameter on the returned :class:`.DateTime` object, as
well as :class:`.Time`. This helps with some round-trip scenarios on
type-sensitive PostgreSQL dialects such as asyncpg, psycopg3 (2.0 only).
Improved support for asyncpg handling of TIME WITH TIMEZONE, which
was not fully implemented.
Mike Bayer [Mon, 10 Jan 2022 19:59:59 +0000 (14:59 -0500)]
ensure with_options not switched to a list
Fixed regression which appeared in 1.4.23 which could cause loader options
to be mis-handled in some cases, in particular when using joined table
inheritance in combination with the ``polymorphic_load="selectin"`` option
as well as relationship lazy loading, leading to a ``TypeError``.
Mike Bayer [Tue, 4 Jan 2022 19:08:10 +0000 (14:08 -0500)]
improve custom operator for SQL types docs
introduce here that "custom ops" first come from
the .op() method in the usual case. then only if one wants
such an op to be pre-assocaited with particular types,
then the comparator may be used. Also clarify the individual
points regarding the comparator.
Mike Bayer [Mon, 3 Jan 2022 22:28:52 +0000 (17:28 -0500)]
ensure correlate_except is checked for empty tuple
Fixed issue where :meth:`_sql.Select.correlate_except` method, when passed
either the ``None`` value or no arguments, would not correlate any elements
when used in an ORM context (that is, passing ORM entities as FROM
clauses), rather than causing all FROM elements to be considered as
"correlated" in the same way which occurs when using Core-only constructs.
Federico Caselli [Thu, 30 Dec 2021 22:29:55 +0000 (23:29 +0100)]
Imrpove MySQL/MariaDB dialect initialization.
Replace ``SHOW VARIABLES LIKE`` statement with equivalent
``SELECT @@variable`` in MySQL and MariaDB dialect initialization.
This should avoid mutex contention caused by ``SHOW VARIABLES``,
improving initialization performance.
Gord Thompson [Mon, 20 Dec 2021 21:37:13 +0000 (14:37 -0700)]
Reflect included columns as dialect_options
Fixed reflection of covering indexes to report ``include_columns`` as part
of the ``dialect_options`` entry in the reflected index dictionary, thereby
enabling round trips from reflection->create to be complete. Included
columns continue to also be present under the ``include_columns`` key for
backwards compatibility.
Mike Bayer [Sun, 26 Dec 2021 17:13:19 +0000 (12:13 -0500)]
include empty intermediary tables in optimized get
Fixed issue in joined-inheritance load of additional attributes
functionality in deep multi-level inheritance where an intermediary table
that contained no columns would not be included in the tables joined,
instead linking those tables to their primary key identifiers. While this
works fine, it nonetheless in 1.4 began producing the cartesian product
compiler warning. The logic has been changed so that these intermediary
tables are included regardless. While this does include additional tables
in the query that are not technically necessary, this only occurs for the
highly unusual case of deep 3+ level inheritance with intermediary tables
that have no non primary key columns, potential performance impact is
therefore expected to be negligible.
Mike Bayer [Sun, 26 Dec 2021 16:25:00 +0000 (11:25 -0500)]
restore graceful degrade of subqueryload w from_statement
Fixed regression from 1.3 where the "subqueryload" loader strategy would
fail with a stack trace if used against a query that made use of
:meth:`_orm.Query.from_statement` or :meth:`_sql.Select.from_statement`. As
subqueryload requires modifying the original statement, it's not compatible
with the "from_statement" use case, especially for statements made against
the :func:`_sql.text` construct. The behavior now is equivalent to that of
1.3 and previously, which is that the loader strategy silently degrades to
not be used for such statements, typically falling back to using the
lazyload strategy.
Kai Mueller [Tue, 21 Dec 2021 20:00:30 +0000 (15:00 -0500)]
Fix missing class attributes when using __class_getitem__
Fixed issue where the ``__class_getitem__()`` method of the generated
declarative base class by :func:`_orm.as_declarative` would lead to
inaccessible class attributes such as ``__table__``, for cases where a
``Generic[T]`` style typing declaration were used in the class hierarchy.
This is in continuation from the basic addition of ``__class_getitem__()``
in :ticket:`7368`. Pull request courtesy Kai Mueller.
Mike Bayer [Wed, 22 Dec 2021 20:33:11 +0000 (15:33 -0500)]
add recursion check for with_loader_criteria() option
Fixed recursion overflow which could occur within ORM statement compilation
when using either the :func:`_orm.with_loader_criteria` feature or the the
:meth:`_orm.PropComparator.and_` method within a loader strategy in
conjunction with a subquery which referred to the same entity being altered
by the criteria option, or loaded by the loader strategy. A check for
coming across the same loader criteria option in a recursive fashion has
been added to accommodate for this scenario.
Mike Bayer [Wed, 22 Dec 2021 21:21:33 +0000 (16:21 -0500)]
use fully qualified, locatable names for all use of api.named_type()
Fixed mypy regression where the release of mypy 0.930 added additional
internal checks to the format of "named types", requiring that they be
fully qualified and locatable. This broke the mypy plugin for SQLAlchemy,
raising an assertion error, as there was use of symbols such as
``__builtins__`` and other un-locatable or unqualified names that
previously had not raised any assertions.
Mike Bayer [Tue, 21 Dec 2021 23:08:33 +0000 (18:08 -0500)]
accommodate for "clone" of ColumnClause
for use with the ClauseElement.params() method,
altered ColumnClause._clone() so that while the element
stays immutable, if the column is associated with a subquery,
it returns a new version of itself as corresponding to a
clone of the subquery. this allows processing functions
to access the parameters in the subquery and produce a
copy of it. The use case here is the expanded use of
.params() within loader strategies that use
HasCacheKey._apply_params_to_element().
Fixed issue in new "loader criteria" method
:meth:`_orm.PropComparator.and_` where usage with a loader strategy like
:func:`_orm.selectinload` against a column that was a member of the ``.c.``
collection of a subquery object, where the subquery would be dynamically
added to the FROM clause of the statement, would be subject to stale
parameter values within the subquery in the SQL statement cache, as the
process used by the loader strategy to replace the parameters at execution
time would fail to accommodate the subquery when received in this form.
Mike Bayer [Tue, 14 Dec 2021 21:46:50 +0000 (16:46 -0500)]
include InterfaceError for mariadb disconnect check
Corrected the error classes inspected for the "is_disconnect" check for the
``mariadbconnector`` dialect, which was failing for disconnects that
occurred due to common MySQL/MariaDB error codes such as 2006; the DBAPI
appears to currently use the ``mariadb.InterfaceError`` exception class for
disconnect errors such as error code 2006, which has been added to the list
of classes checked.
For the current "real reconnect test", shutting down the mariadb
connection from the client side produces
ProgrammingError("Connection isn't valid anymore") which we also
continue to intercept.
Nils Philippsen [Sun, 12 Dec 2021 23:35:03 +0000 (18:35 -0500)]
Ignore ephemeral classes in test_all_present()
Fixed a regression in the test suite where the test called
``CompareAndCopyTest::test_all_present`` would fail on some platforms due
to additional testing artifacts being detected. Pull request courtesy Nils
Philippsen.
In some circumstances, ephemeral class objects that are created within
the scope of a test method don't seem to be garbage collected directly
on exit. Filter out classes created in test modules.
Mike Bayer [Sun, 12 Dec 2021 18:37:21 +0000 (13:37 -0500)]
use the options from the cached statement for propagate_options
Fixed caching-related issue where the use of a loader option of the form
``lazyload(aliased(A).bs).joinedload(B.cs)`` would fail to result in the
joinedload being invoked for runs subsequent to the query being cached, due
to a mismatch for the options / object path applied to the objects loaded
for a query with a lead entity that used ``aliased()``.
Mike Bayer [Sun, 12 Dec 2021 16:23:07 +0000 (11:23 -0500)]
check for string_types, not str, for py2 support
Fixed regression in the :func:`_engine.make_url` function used to parse URL
strings where the query string parsing would go into a recursion overflow
if a Python 2 ``u''`` string were used.
Nils Philippsen [Sat, 13 Nov 2021 16:11:32 +0000 (11:11 -0500)]
Add async_engine_from_config()
Added :func:`_asyncio.async_engine_config` function to create
an async engine from a configuration dict. This otherwise
behaves the same as :func:`_sa.engine_from_config`.
Mike Bayer [Thu, 9 Dec 2021 19:23:42 +0000 (14:23 -0500)]
implement correct errors for Row immutability
Corrected the error message for the ``AttributeError`` that's raised when
attempting to write to an attribute on the :class:`_result.Row` class,
which is immutable. The previous message claimed the column didn't exist
which is misleading.
Mike Bayer [Thu, 9 Dec 2021 17:51:43 +0000 (12:51 -0500)]
implement attributes.Proxy._clone()
Fixed issue where the internal cloning used by the
:meth:`_orm.PropComparator.any` method on a :func:`_orm.relationship` in
the case where the related class also makes use of ORM polymorphic loading,
would fail if a hybrid property on the related, polymorphic class were used
within the criteria for the ``any()`` operation.
Mike Bayer [Fri, 3 Dec 2021 19:04:05 +0000 (14:04 -0500)]
Warn when caching is disabled / document
This patch adds new warnings for all elements that
don't indicate their caching behavior, including user-defined
ClauseElement subclasses and third party dialects.
it additionally adds new documentation to discuss an apparent
performance degradation in 1.4 when caching is disabled as a
result in the significant expense incurred by ORM
lazy loaders, which in 1.3 used BakedQuery so were actually
cached.
As a result of adding the warnings, a fair degree of
lesser used SQL expression objects identified that they did not
define caching behavior so would have been producing
``[no key]``, including PostgreSQL constructs ``hstore``
and ``array``. These have been amended to use inherit
cache where appropriate. "on conflict" constructs in
PostgreSQL, MySQL, SQLite still explicitly don't generate
a cache key at this time.
The change also adds a test for all constructs via
assert_compile() to assert they will not generate cache
warnings.
Mike Bayer [Thu, 2 Dec 2021 00:27:25 +0000 (19:27 -0500)]
contextmanager skips rollback if trans says to skip it
Fixed issue where if an exception occurred when the :class:`_orm.Session`
were to close the connection within the :meth:`_orm.Session.commit` method,
when using a context manager for :meth:`_orm.Session.begin` , it would
attempt a rollback which would not be possible as the :class:`_orm.Session`
was in between where the transaction is committed and the connection is
then to be returned to the pool, raising the exception "this
sessiontransaction is in the committed state". This exception can occur
mostly in an asyncio context where CancelledError can be raised.
The where method of exists now accepts multiple cluase.
Support multiple clause elements in the :meth:`_sql.Exists.where` method,
unifying the api with the on presented by a normal :func:`_sql.select`
construct.
Mike Bayer [Thu, 2 Dec 2021 02:39:59 +0000 (21:39 -0500)]
copy list for __iadd__
Fixed issue where a list mapped with :func:`_orm.relationship` would go
into an endless loop if in-place added to itself, i.e. the ``+=`` operator
were used, as well as if ``.extend()`` were given the same list.
Kai Mueller [Wed, 1 Dec 2021 15:58:40 +0000 (10:58 -0500)]
Add __class_getitem__ to the declarative Base class
Fixed issue where the :func:`_orm.as_declarative` decorator and similar
functions used to generate the declarative base class would not copy the
``__class_getitem__()`` method from a given superclass, which prevented the
use of pep-484 generics in conjunction with the ``Base`` class. Pull
request courtesy Kai Mueller.
Federico Caselli [Sat, 27 Nov 2021 08:53:29 +0000 (09:53 +0100)]
adapt pytest plugin to support pytest v7
Implemented support for the test suite to run correctly under Pytest 7.
Previously, only Pytest 6.x was supported for Python 3, however the version
was not pinned on the upper bound in tox.ini. Pytest is not pinned in
tox.ini to be lower than version 8 so that SQLAlchemy versions released
with the current codebase will be able to be tested under tox without
changes to the environment. Much thanks to the Pytest developers for
their help with this issue.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
