Mike Bayer [Thu, 24 Jun 2021 13:12:31 +0000 (09:12 -0400)]
Use Py_TPFLAGS_HAVE_GC for Row
Fixed an issue in the C extension for the :class:`_result.Row` class which
could lead to a memory leak in the unlikely case of a :class:`_result.Row`
object which referred to an ORM object that then was mutated to refer back
to the ``Row`` itself, creating a cycle. The Python C APIs for tracking GC
cycles has been added to the native :class:`_result.Row` implementation to
accommodate for this case.
Mike Bayer [Wed, 23 Jun 2021 20:34:05 +0000 (16:34 -0400)]
Add Executable to DefaultGenerator
Fixed the class hierarchy for the :class:`_schema.Sequence` and the more
general :class:`_schema.DefaultGenerator` base, as these are "executable"
as statements they need to include :class:`_sql.Executable` in their
hierarchy, not just :class:`_roles.StatementRole` as was applied
arbitrarily to :class:`_schema.Sequence` previously. The fix allows
:class:`_schema.Sequence` to work in all ``.execute()`` methods including
with :meth:`_orm.Session.execute` which was not working in the case that a
``do_orm_execute()`` handler was also established.
Mike Bayer [Wed, 23 Jun 2021 12:50:48 +0000 (08:50 -0400)]
consider "*" col as textual ordered
Fixed old issue where a :func:`_sql.select()` made against the token "*",
which then yielded exactly one column, would fail to correctly organize the
``cursor.description`` column name into the keys of the result object.
MajorDallas [Tue, 22 Jun 2021 19:34:09 +0000 (15:34 -0400)]
Add impl property to PostgreSQL / Oracle INTERVAL class
Fixed issue where the ``INTERVAL`` datatype on PostgreSQL and Oracle would
produce an ``AttributeError`` when used in the context of a comparison
operation against a ``timedelta()`` object. Pull request courtesy
MajorDallas.
Mike Bayer [Tue, 22 Jun 2021 17:27:18 +0000 (13:27 -0400)]
Export deferred columns but not col props; fix CTE labeling
Refined the behavior of ORM subquery rendering with regards to deferred
columns and column properties to be more compatible with that of 1.3 while
also providing for 1.4's newer features. As a subquery in 1.4 does not make
use of loader options, including :func:`_orm.deferred`, a subquery that is
against an ORM entity with deferred attributes will now render those
deferred attributes that refer directly to mapped table columns, as these
are needed in the outer SELECT if that outer SELECT makes use of these
columns; however a deferred attribute that refers to a composed SQL
expression as we normally do with :func:`_orm.column_property` will not be
part of the subquery, as these can be selected explicitly if needed in the
subquery. If the entity is being SELECTed from this subquery, the column
expression can still render on "the outside" in terms of the derived
subquery columns. This produces essentially the same behavior as when
working with 1.3. However in this case the fix has to also make sure that
the ``.selected_columns`` collection of an ORM-enabled :func:`_sql.select`
also follows these rules, which in particular allows recursive CTEs to
render correctly in this scenario, which were previously failing to render
correctly due to this issue.
As part of this change the _exported_columns_iterator() method has been
removed and logic simplified to use ._all_selected_columns from any
SelectBase object where _exported_columns_iterator() was used before.
Additionally sets up UpdateBase to include ReturnsRows in its hierarchy;
the literal point of ReturnsRows was to be a common base for UpdateBase
and SelectBase so it was kind of weird it wasn't there.
Fixes: #6661
Fixed issue in CTE constructs mostly relevant to ORM use cases where a
recursive CTE against "anonymous" labels such as those seen in ORM
``column_property()`` mappings would render in the
``WITH RECURSIVE xyz(...)`` section as their raw internal label and not a
cleanly anonymized name.
Mike Bayer [Mon, 21 Jun 2021 22:13:55 +0000 (18:13 -0400)]
apply render_schema_translates to identity insert directives
Fixed bug where the "schema_translate_map" feature would fail to function
correctly in conjunction with an INSERT into a table that has an IDENTITY
column, where the value of the IDENTITY column were specified in the values
of the INSERT thus triggering SQLAlchemy's feature of setting IDENTITY
INSERT to "on"; it's in this directive where the schema translate map would
fail to be honored.
Anton Kovalevich [Fri, 18 Jun 2021 14:33:48 +0000 (10:33 -0400)]
Implement MySQL-specific MATCH
Added new construct :class:`_mysql.match`, which provides for the full
range of MySQL's MATCH operator including multiple column support and
modifiers. Pull request courtesy Anton Kovalevich.
Mike Bayer [Mon, 21 Jun 2021 17:08:59 +0000 (13:08 -0400)]
ensure test has deterministic FROM rendering
test_options_entities_replaced_with_equivs_three did not
have deterministic FROM ordering, so adding an inner
join from user->address should ensure there's a single
FROM element that is a series of joins.
Mike Bayer [Mon, 21 Jun 2021 16:52:42 +0000 (12:52 -0400)]
accommodate no cls info found in _scan_declarative
Fixed issue in mypy plugin where class info for a custom declarative base
would not be handled correctly on a cached mypy pass, leading to an
AssertionError being raised.
Mike Bayer [Fri, 18 Jun 2021 21:42:53 +0000 (17:42 -0400)]
ensure greenlet_spawn propagates BaseException
Fixed bug in asyncio implementation where the greenlet adaptation system
failed to propagate ``BaseException`` subclasses, most notably including
``asyncio.CancelledError``, to the exception handling logic used by the
engine to invalidate and clean up the connection, thus preventing
connections from being correctly disposed when a task was cancelled.
Abhishek Jog [Wed, 16 Jun 2021 19:15:24 +0000 (15:15 -0400)]
Added netezza in external dialects in documentation
### Description
IBM Netezza Performance Server (aka Netezza, NPS) now has fully function SQLAlchemy dialect. Need to add it in SQLAlchemy 1.3 documentation.
This PR fixes https://github.com/sqlalchemy/sqlalchemy/issues/6609
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 15 Jun 2021 19:13:34 +0000 (15:13 -0400)]
memoize current options and joins w with_entities/with_only_cols
Fixed further regressions in the same area as that of :ticket:`6052` where
loader options as well as invocations of methods like
:meth:`_orm.Query.join` would fail if the left side of the statement for
which the option/join depends upon were replaced by using the
:meth:`_orm.Query.with_entities` method, or when using 2.0 style queries
when using the :meth:`_sql.Select.with_only_columns` method. A new set of
state has been added to the objects which tracks the "left" entities that
the options / join were made against which is memoized when the lead
entities are changed.
jason3gb [Wed, 16 Jun 2021 14:18:08 +0000 (10:18 -0400)]
Implement async_scoped_session
Implemented :class:`_asyncio.async_scoped_session` to address some
asyncio-related incompatibilities between :class:`_orm.scoped_session` and
:class:`_asyncio.AsyncSession`, in which some methods (notably the
:meth:`_asyncio.async_scoped_session.remove` method) should be used with
the ``await`` keyword.
Mike Bayer [Fri, 11 Jun 2021 14:23:51 +0000 (10:23 -0400)]
set autocommit for psycopg2 pre-ping
Fixed issue where the pool "pre ping" feature would implicitly start a
transaction, which would then interfere with custom transactional flags
such as PostgreSQL's "read only" mode when used with the psycopg2 driver.
Mike Bayer [Tue, 8 Jun 2021 19:43:13 +0000 (15:43 -0400)]
simplify relationship caching options
Clarified the current purpose of the
:paramref:`_orm.relationship.bake_queries` flag, which in 1.4 is to enable
or disable "lambda caching" of statements within the "lazyload" and
"selectinload" loader strategies; this is separate from the more
foundational SQL query cache that is used for most statements.
Additionally, the lazy loader no longer uses its own cache for many-to-one
SQL queries, which was an implementation quirk that doesn't exist for any
other loader scenario. Finally, the "lru cache" warning that the lazyloader
and selectinloader strategies could emit when handling a wide array of
class/relationship combinations has been removed; based on analysis of some
end-user cases, this warning doesn't suggest any significant issue. While
setting ``bake_queries=False`` for such a relationship will remove this
cache from being used, there's no particular performance gain in this case
as using no caching vs. using a cache that needs to refresh often likely
still wins out on the caching being used side.
Propagate asyncio flag from the dialect to selected pool classes
Fixed an issue that presented itself when using the :class:`_pool.NullPool`
or the :class:`_pool.StaticPool` with an async engine. This mostly affected
the aiosqlite dialect.
Mike Bayer [Mon, 7 Jun 2021 22:49:04 +0000 (18:49 -0400)]
restore adapter logic in ORM loading
Fixed regression involving how the ORM would resolve a given mapped column
to a result row, where under cases such as joined eager loading, a slightly
more expensive "fallback" could take place to set up this resolution due to
some logic that was removed since 1.3. The issue could also cause
deprecation warnings involving column resolution to be emitted when using a
1.4 style query with joined eager loading.
In order to ensure we don't look up columns by string name in the ORM,
we've turned on future_result=True in all cases, which I thought was
already the assumption here, but apparently not. That in turn
led to the issue that Session autocommit relies on close_with_result=True,
which is legacy result only. This was also hard to figure out.
So a new exception is raised if one tries to use future_result=True
along with close_with_result, and the Session now has an explicit path
for "autocommit" that sets these flags to their legacy values.
This does leave the possibility of some of these fallback cases
emitting warnings for users using session in autocommit along with
joined inheritance and column properties, as this patch identifies
that joined inheritance + column properties produce the fallback
logic when looking up in the result via the adapted column, which
in those tests is actually a Label object that doesn't adapt
nicely.
Mike Bayer [Mon, 7 Jun 2021 21:47:07 +0000 (17:47 -0400)]
repair Join.is_derived_from() to not rely on simple identity
Fixed issue where query production for joinedload against a complex left
hand side involving joined-table inheritance could fail to produce a
correct query, due to a clause adaption issue.
Mike Bayer [Mon, 7 Jun 2021 15:28:49 +0000 (11:28 -0400)]
Add asyncio.TimeoutError as an exit exception
Added ``asyncio.exceptions.TimeoutError``,
``asyncio.exceptions.CancelledError`` as so-called "exit exceptions", a
class of exceptions that include things like ``GreenletExit`` and
``KeyboardInterrupt``, which are considered to be events that warrant
considering a DBAPI connection to be in an unusable state where it should
be recycled.
Mike Bayer [Mon, 7 Jun 2021 13:21:25 +0000 (09:21 -0400)]
init extra_criteria_entities in fromstatement w/ DML
Fixed issue in experimental "select ORM objects from INSERT/UPDATE" use
case where an error was raised if the statement were against a
single-table-inheritance subclass.
Additionally makes some adjustments in the SQL assertion
fixture to test a FromStatement w/ DML.
Mike Bayer [Sun, 6 Jun 2021 14:12:05 +0000 (10:12 -0400)]
Pass URL object, not the string, to on_connect_url
The fix for pysqlcipher released in version 1.4.3 :ticket:`5848` was
unfortunately non-working, in that the new ``on_connect_url`` hook was
erroneously not receiving a ``URL`` object under normal usage of
:func:`_sa.create_engine` and instead received a string that was unhandled;
the test suite failed to fully set up the actual conditions under which
this hook is called. This has been fixed.
Mike Bayer [Fri, 4 Jun 2021 23:45:45 +0000 (19:45 -0400)]
don't enable "fast insert executemany" for ON CONFLICT etc
Fixed issue where using the PostgreSQL "INSERT..ON CONFLICT" structure
would fail to work with the psycopg2 driver if it were used in an
"executemany" context along with bound parameters in the "SET" clause, due
to the implicit use of the psycopg2 fast execution helpers which are not
appropriate for this style of INSERT statement. Additional checks to
exclude this kind of statement from that particular extension have been
added.
Mike Bayer [Thu, 3 Jun 2021 13:37:27 +0000 (09:37 -0400)]
Adjust create_proxy_methods() to use kw arguments
Adjusted the means by which classes such as :class:`_orm.scoped_session`
and :class:`_asyncio.AsyncSession` are generated from the base
:class:`_orm.Session` class, such that custom :class:`_orm.Session`
subclasses such as that used by Flask-SQLAlchemy don't need to implement
positional arguments when they call into the superclass method, and can
continue using the same argument styles as in previous releases.
Mike Bayer [Thu, 3 Jun 2021 03:07:28 +0000 (23:07 -0400)]
delete exception traceback in _expect_raises
a new segfault is observed in python 3.10.0b2 in conjunction
with asyncio and possibly greenlet. Ensuring the
traceback object is deleted from scope here, which is a
good idea anyway, apparently seems to resolve the issue.
Mike Bayer [Wed, 2 Jun 2021 22:24:43 +0000 (18:24 -0400)]
temporarily disable test_no_attach_to_event_loop
this test currently causes the test suite to hang; it previously
was not actually running the worker thread
as the testing_engine() fixture
was rejecting the "transfer_staticpool" keyword argument.
as we seem to have a greenlet-related segfault in 3.10.0b2 I am
going to have to get the greenlet devs to run the test suite
so i want to get anything not totally smooth out of it for the
moment.
Mike Bayer [Wed, 2 Jun 2021 16:23:31 +0000 (12:23 -0400)]
Implement proxy back reference system for asyncio
Implemented a new registry architecture that allows the ``Async`` version
of an object, like ``AsyncSession``, ``AsyncConnection``, etc., to be
locatable given the proxied "sync" object, i.e. ``Session``,
``Connection``. Previously, to the degree such lookup functions were used,
an ``Async`` object would be re-created each time, which was less than
ideal as the identity and state of the "async" object would not be
preserved across calls.
From there, new helper functions :func:`_asyncio.async_object_session`,
:func:`_asyncio.async_session` as well as a new :class:`_orm.InstanceState`
attribute :attr:`_orm.InstanceState.asyncio_session` have been added, which
are used to retrieve the original :class:`_asyncio.AsyncSession` associated
with an ORM mapped object, a :class:`_orm.Session` associated with an
:class:`_asyncio.AsyncSession`, and an :class:`_asyncio.AsyncSession`
associated with an :class:`_orm.InstanceState`, respectively.
This patch also implements new methods
:meth:`_asyncio.AsyncSession.in_nested_transaction`,
:meth:`_asyncio.AsyncSession.get_transaction`,
:meth:`_asyncio.AsyncSession.get_nested_transaction`.
Mike Bayer [Tue, 1 Jun 2021 20:03:52 +0000 (16:03 -0400)]
improve "overlaps" warning; test for m2m
The warning that's emitted for :func:`_orm.relationship` when multiple
relationships would overlap with each other as far as foreign key
attributes written towards, now includes the specific "overlaps" argument
to use for each warning in order to silence the warning without changing
the mapping.
Mike Bayer [Sat, 29 May 2021 15:05:25 +0000 (11:05 -0400)]
skip dupe order by test on mssql
SQL Server wants to complain if ORDER BY specifies the same
expression more than once, so just skip this test added in aaba0650d7410f579b2c14f8f1b0680a1d1852c4 as this only needs
to test ORM functionality.
Mike Bayer [Sat, 29 May 2021 14:05:20 +0000 (10:05 -0400)]
Ensure propagate_attrs available on PropComparator
Fixed regression caused by just-released performance fix mentioned in #6550
where a query.join() to a relationship could produce an AttributeError if
the query were made against non-ORM structures only, a fairly unusual
calling pattern.
In this fix, since we are no longer going through the production
of ``__clause_element__()`` for Cls.some_relationship, I assumed we
just throw this object away completely but I missed the one little
bit where we might be getting ``_propagate_attrs`` from it.
So we implement ``_propagate_attrs`` on ``PropComparator`` as well,
since this is easy to define.
Mike Bayer [Fri, 28 May 2021 12:29:24 +0000 (08:29 -0400)]
Fix adaption in AnnotatedLabel; repair needless expense in coercion
Fixed regression involving clause adaption of labeled ORM compound
elements, such as single-table inheritance discriminator expressions with
conditionals or CASE expressions, which could cause aliased expressions
such as those used in ORM join / joinedload operations to not be adapted
correctly, such as referring to the wrong table in the ON clause in a join.
This change also improves a performance bump that was located within the
process of invoking :meth:`_sql.Select.join` given an ORM attribute
as a target.
Mike Bayer [Fri, 28 May 2021 01:15:01 +0000 (21:15 -0400)]
ensure relationship.order_by stored as a tuple; check in dynamic also
Fixed regression in dynamic loader strategy and :func:`_orm.relationship`
overall where the :paramref:`_orm.relationship.order_by` parameter were
stored as a mutable list, which could then be mutated when combined with
additional "order_by" methods used against the dynamic query object,
causing the ORDER BY criteria to continue to grow repetitively.
Federico Caselli [Thu, 27 May 2021 20:51:40 +0000 (22:51 +0200)]
Make other loop test compatible with py3.10
Python 3.10 fixes the issue where the loop would bind to a queue
on instantiation. Now an object binds to the loop only when first
needs it. I've looked at queues, but I'm assuming locks behave
the same,
jason3gb [Tue, 25 May 2021 19:40:48 +0000 (15:40 -0400)]
Removed automap deprecation warning
Fixed a deprecation warning that was emitted when using
:func:`_automap.automap_base` without passing an existing
``Base``.
Fixes: #6529
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Fixed the automap's declarative import from sqlalchemy.orm instead from the deprecated ext.declarative package
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Lele Gaifax [Tue, 25 May 2021 19:51:44 +0000 (21:51 +0200)]
Fix minor glitches in the visitors.py docstrings (#6522)
* Fix docstring typo
* Capitalize first sentence word
* Properly markup class name
This is just for consistency with nearby references: the HasCacheKey class has no docstring so
it does not appear in the documentation, moreover it's in the traversals module, so Sphinx
cannot render it as a link anyway.
Jim Fulton [Tue, 18 May 2021 18:09:07 +0000 (14:09 -0400)]
Provide primary key values for data in tests that aren't about primary keys.
(message written by Mike) some backends such as BigQuery have no
autoincrement mechanism at all.
while we would like to pursue a strategy where provisioning.py could
provide for an in-Python sequence generator, at least remove
the need for autoincrement in suite tests that don't need it.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
