Mike Bayer [Sat, 26 Nov 2022 16:03:45 +0000 (11:03 -0500)]
add new variation helper
I'm using a lot of @testing.combinations with either
a boolean True/False, or a series of string names, each indicating
some case to switch on. I want a descriptive name in the test
run (not True/False) and I don't want to compare strings.
So make a new helper around @combinations that provides an
object interface that has booleans inside of it, prints nicely
in the test output, raises an error if you name the case
incorrectly.
Michael Gorven [Thu, 24 Nov 2022 08:47:26 +0000 (03:47 -0500)]
Fix reflection of constraints in attached schemas
Backported a fix for SQLite reflection of unique constraints in attached
schemas, released in 2.0 as a small part of :ticket:`4379`. Previously,
unique constraints in attached schemas would be ignored by SQLite
reflection. Pull request courtesy Michael Gorven.
Mike Bayer [Wed, 23 Nov 2022 15:58:28 +0000 (10:58 -0500)]
add "merge" to viewonly cascades; propagate NO_RAISE when merging
Fixed bug where :meth:`_orm.Session.merge` would fail to preserve the
current loaded contents of relationship attributes that were indicated with
the :paramref:`_orm.relationship.viewonly` parameter, thus defeating
strategies that use :meth:`_orm.Session.merge` to pull fully loaded objects
from caches and other similar techniques. In a related change, fixed issue
where an object that contains a loaded relationship that was nonetheless
configured as ``lazy='raise'`` on the mapping would fail when passed to
:meth:`_orm.Session.merge`; checks for "raise" are now suspended within
the merge process assuming the :paramref:`_orm.Session.merge.load`
parameter remains at its default of ``True``.
Overall, this is a behavioral adjustment to a change introduced in the 1.4
series as of :ticket:`4994`, which took "merge" out of the set of cascades
applied by default to "viewonly" relationships. As "viewonly" relationships
aren't persisted under any circumstances, allowing their contents to
transfer during "merge" does not impact the persistence behavior of the
target object. This allows :meth:`_orm.Session.merge` to correctly suit one
of its use cases, that of adding objects to a :class:`.Session` that were
loaded elsewhere, often for the purposes of restoring from a cache.
Mike Bayer [Wed, 23 Nov 2022 22:44:30 +0000 (17:44 -0500)]
flake8 has very courteously referred us to another pyqca project and closed all comments, thank you! I'll get on that right away
pyqca/flake8-import-order does not seem to have a release or a commit
in the past two years, so while I have created an issue and PR [1] [2],
for now vendor our fork so we can get on with things.
Mike Bayer [Thu, 17 Nov 2022 01:11:18 +0000 (20:11 -0500)]
accommodate NULL format_type()
Made an adjustment to how the PostgreSQL dialect considers column types
when it reflects columns from a table, to accommodate for alternative
backends which may return NULL from the PG ``format_type()`` function.
Eitan Mosenkis [Mon, 14 Nov 2022 21:11:15 +0000 (23:11 +0200)]
Explicitly state what happens if `order_by` is called more than once. (#8791)
* Explicitly state what happens if `order_by` is called more than once.
The existing docs cover how to clear existing `order_by` clauses but don't actually describe the behavior of calling `order_by` multiple times with different clauses.
Mike Bayer [Mon, 14 Nov 2022 13:54:56 +0000 (08:54 -0500)]
add informative exception context for literal render
An informative re-raise is now thrown in the case where any "literal
bindparam" render operation fails, indicating the value itself and
the datatype in use, to assist in debugging when literal params
are being rendered in a statement.
Adjusted the test suite which tests the Mypy plugin to accommodate for
changes in Mypy 0.990 regarding how it handles message output, which affect
how sys.path is interpreted when determining if notes and errors should be
printed for particular files. The change broke the test suite as the files
within the test directory itself no longer produced messaging when run
under the mypy API.
Mike Bayer [Thu, 10 Nov 2022 22:01:58 +0000 (17:01 -0500)]
ensure anon_map is passed for most annotated traversals
We can cache the annotated cache key for Table, but
for selectables it's not safe, as it fails to pass the
anon_map along and creates many redudant structures in
observed test scenario. It is likely safe for a
Column that's mapped to a Table also, however this is
not implemented here. Will have to see if that part
needs adjusting.
Fixed critical memory issue identified in cache key generation, where for
very large and complex ORM statements that make use of lots of ORM aliases
with subqueries, cache key generation could produce excessively large keys
that were orders of magnitude bigger than the statement itself. Much thanks
to Rollo Konig Brock for their very patient, long term help in finally
identifying this issue.
Also within TypeEngine objects, when we generate elements
for instance variables, skip the None elements at least.
this also saves on tuple complexity.
Mike Bayer [Mon, 7 Nov 2022 23:40:03 +0000 (18:40 -0500)]
establish consistency for RETURNING column labels
For the PostgreSQL and SQL Server dialects only, adjusted the compiler so
that when rendering column expressions in the RETURNING clause, the "non
anon" label that's used in SELECT statements is suggested for SQL
expression elements that generate a label; the primary example is a SQL
function that may be emitting as part of the column's type, where the label
name should match the column's name by default. This restores a not-well
defined behavior that had changed in version 1.4.21 due to :ticket:`6718`,
:ticket:`6710`. The Oracle dialect has a different RETURNING implementation
and was not affected by this issue. Version 2.0 features an across the
board change for its widely expanded support of RETURNING on other
backends.
Fixed issue in the Oracle dialect where an INSERT statement that used
``insert(some_table).values(...).returning(some_table)`` against a full
:class:`.Table` object at once would fail to execute, raising an exception.
Mike Bayer [Fri, 11 Nov 2022 14:46:06 +0000 (09:46 -0500)]
repair --disable-asyncio parameter
Fixed issue where the ``--disable-asyncio`` parameter to the test suite
would fail to not actually run greenlet tests and would also not prevent
the suite from using a "wrapping" greenlet for the whole suite. This
parameter now ensures that no greenlet or asyncio use will occur within the
entire run when set.
Mike Bayer [Fri, 4 Nov 2022 16:48:43 +0000 (12:48 -0400)]
resolve synonyms in dictionary form of Session.get()
Improved "dictionary mode" for :meth:`_orm.Session.get` so that synonym
names which refer to primary key attribute names may be indicated in the
named dictionary.
Mike Bayer [Tue, 1 Nov 2022 19:09:25 +0000 (15:09 -0400)]
soft close cursor for Query direct iterator interrupted
Fixed issue where the underlying DBAPI cursor would not be closed when
using :class:`_orm.Query` and direct iteration, if a user-defined exception
case were raised within the iteration process, interrupting the iterator
which otherwise is not possible to re-use in this context. When using
:meth:`_orm.Query.yield_per` to create server-side cursors, this would lead
to the usual MySQL-related issues with server side cursors out of sync.
To resolve, a catch for ``GeneratorExit`` is applied within the default
iterator, which applies only in those cases where the interpreter is
calling ``.close()`` on the iterator in any case.
A similar scenario can occur when using :term:`2.x` executions with direct
use of :class:`.Result`, in that case the end-user code has access to the
:class:`.Result` itself and should call :meth:`.Result.close` directly.
Version 2.0 will feature context-manager calling patterns to address this
use case. However within the 1.4 scope, ensured that ``.close()`` methods
are available on all :class:`.Result` implementations including
:class:`.ScalarResult`, :class:`.MappingResult`.
Mike Bayer [Wed, 2 Nov 2022 02:57:16 +0000 (22:57 -0400)]
scale back warnings for 1.4's bulk methods
since these methods have been improved for 2.0, the general
idea is not going away, so remove the warnings indicating
that these features are being removed.
Mike Bayer [Mon, 31 Oct 2022 19:09:34 +0000 (15:09 -0400)]
use simple decimal query to detect decimal char
Fixed issue where the ``nls_session_parameters`` view queried on first
connect in order to get the default decimal point character may not be
available depending on Oracle connection modes, and would therefore raise
an error. The approach to detecting decimal char has been simplified to
test a decimal value directly, instead of reading system views, which
works on any backend / driver.
Mike Bayer [Tue, 25 Oct 2022 20:00:50 +0000 (16:00 -0400)]
ensure pool.reset event always called for reset
Fixed issue where the :meth:`.PoolEvents.reset` event hook would
not be called when a :class:`.Connection` were closed which already
closed its own transaction. Logic that bypasses the "rollback on return"
behavior of the pool was also skipping the event hook being emitted,
preventing custom pool reset schemes from being used within this hook.
This was a regression that appeared in version 1.4.
For version 1.4, the hook is still not called in the case of an asyncio
connection that is being discarded due to garbage collection. Version 2.0
will feature an improved version of :meth:`.PoolEvents.reset` which also
receives contextual information about the reset, so that comprehensive
"custom connection reset" schemes can be devised. Existing custom
reset schemes that make use of :meth:`.PoolEvents.checkin` remain
usable as they typically only need to deal with connections that are to
be re-used.
Mike Bayer [Sun, 30 Oct 2022 00:08:25 +0000 (20:08 -0400)]
fix test for same mapper to use "isa"
Fixed issue in joined eager loading where an assertion fail would occur
with a particular combination of outer/inner joined eager loads in
conjunction with an inherited subclass mapper as the middle target.
Mike Bayer [Fri, 28 Oct 2022 16:20:22 +0000 (12:20 -0400)]
update rel/fk FAQ entry
this entry still made the assumptions of behavior before
ticket #3061, that accessing a non-initialized scalar attribute
on a pending object would populate the attribute with None.
It also used the word "initialize" when referring to a persistent
object which is a misleading term, it's "loaded", even though
in this example it's "loading" the value of None.
Fix up the language to be more consistent with the #3061 change.
Mike Barry [Wed, 26 Oct 2022 17:40:32 +0000 (13:40 -0400)]
use only object_id() function for temp tables
Fixed issue with :meth:`.Inspector.has_table` when used against a temporary
table for the SQL Server dialect would fail an invalid object name error on
some Azure variants, due to an unnecessary information schema query that is
not supported on those server versions. Pull request courtesy Mike Barry.
the patch also fills out test support for has_table()
against temp tables, temp views, adding to the has_table() support just
added for views in #8700.
Mike Bayer [Thu, 27 Oct 2022 13:28:02 +0000 (09:28 -0400)]
apply basic escaping to anon_labels unconditionally
Fixed issue which prevented the :func:`_sql.literal_column` construct from
working properly within the context of a :class:`.Select` construct as well
as other potential places where "anonymized labels" might be generated, if
the literal expression contained characters which could interfere with
format strings, such as open parenthesis, due to an implementation detail
of the "anonymous label" structure.
Mike Bayer [Thu, 27 Oct 2022 02:59:51 +0000 (22:59 -0400)]
ensure _ORMJoin transfers parententity from left side
Fixed bug involving :class:`.Select` constructs which used a combination of
:meth:`.Select.select_from` with an ORM entity followed by
:meth:`.Select.join` against the entity sent in
:meth:`.Select.select_from`, as well as using plain
:meth:`.Select.join_from`, which when combined with a columns clause that
didn't explicitly include that entity would then cause "automatic WHERE
criteria" features such as the IN expression required for a single-table
inheritance subclass, as well as the criteria set up by the
:func:`_orm.with_loader_criteria` option, to not be rendered for that
entity. The correct entity is now transferred to the :class:`.Join` object
that's generated internally, so that the criteria against the left
side entity is correctly added.
Mike Bayer [Tue, 25 Oct 2022 14:22:14 +0000 (10:22 -0400)]
raise for non-Load opt passed to options()
Fixed the exception that's raised when the
:func:`_orm.with_loader_criteria` option is attempted to be used within a
specific loader path, like in loader.options().
:func:`_orm.with_loader_criteria` is only intended to be used at the top
level.
Mike Bayer [Mon, 24 Oct 2022 23:24:11 +0000 (19:24 -0400)]
add Oracle-specific parameter escapes for expanding params
Fixed issue where bound parameter names, including those automatically
derived from similarly-named database columns, which contained characters
that normally require quoting with Oracle would not be escaped when using
"expanding parameters" with the Oracle dialect, causing execution errors.
The usual "quoting" for bound parameters used by the Oracle dialect is not
used with the "expanding parameters" architecture, so escaping for a large
range of characters is used instead, now using a list of characters/escapes
that are specific to Oracle.
Mike Bayer [Sun, 23 Oct 2022 23:24:54 +0000 (19:24 -0400)]
skip ad-hoc properties within subclass_load_via_in
Fixed issue where "selectin_polymorphic" loading for inheritance mappers
would not function correctly if the :param:`_orm.Mapper.polymorphic_on`
parameter referred to a SQL expression that was not directly mapped on the
class.
Mike Bayer [Sun, 23 Oct 2022 14:34:33 +0000 (10:34 -0400)]
test support for has_table()->view; backport to 1.4
For 1.4 only; in 2.0 this just refines the test suite a bit.
Fixed regression which occurred throughout the 1.4 series where the
:meth:`.Inspector.has_table` method, which historically reported on views
as well, stopped working for SQL Server. The method never worked for
Oracle in this way, so for compatibility within the 1.4 series,
Oracle's dialect remains returning False for ``has_table()`` against a
view within the 1.4 series.
The issue is not present in the 2.0 series which uses a different
reflection architecture, where has_table() reports True for views on all
backends including SQL Server and Oracle. Test support is added within the
1.4 series to ensure ``has_table()`` remains working per spec re: views.
Mike Bayer [Thu, 20 Oct 2022 16:28:29 +0000 (12:28 -0400)]
move API docs downwards
Sphinx 5.3 (compared to 5.1.1) is now putting all the autodoc
names into the TOC. So we have to start being more careful
to make sure API docs are well below narrative docs, because
this new style is a wall of text. i dont yet see any options
to turn it off, but it does seem like a good improvement, just makes
doc organization a more difficult endeavor.
Mike Bayer [Wed, 19 Oct 2022 13:23:21 +0000 (09:23 -0400)]
rename tables to _table for basic relationships
the names "parent" / "child" are confusing for new users
in that they are used for table names as well as
"back_populates='parent'", use a disambiguated name.
In this change, there's now overlap between the
variable named "association_table" and the table name
"association_table". not sure of a better naming system.
Mike Bayer [Mon, 17 Oct 2022 19:09:01 +0000 (15:09 -0400)]
update SEQUENCE docs ahead of default change
for backport to 1.4 as well, remove references to
Firebird, and also revert "associate Sequence with MetaData"
step as this is not needed usually, just note that schema
is not shared. encourage users to use IDENTITY instead.
Mike Bayer [Sat, 15 Oct 2022 15:12:25 +0000 (11:12 -0400)]
disable isort in pyproject.toml
disable isort, for IDEs that just default isort to be turned on, e.g. vscode.
we use flake8-import-order for import sorting, using zimports to actually
reformat code. isort is nicer in many ways but doesn't have our
"import *" fixer and also is not 100% compatible with flake8-import-order.
Mike Bayer [Mon, 10 Oct 2022 18:03:04 +0000 (14:03 -0400)]
warn for no polymorphic identity w/ poly hierarchy
A warning is emitted when attempting to configure a mapped class within an
inheritance hierarchy where the mapper is not given any polymorphic
identity, however there is a polymorphic discriminator column assigned.
Such classes should be abstract if they never intend to load directly.
Mike Bayer [Fri, 7 Oct 2022 15:25:08 +0000 (11:25 -0400)]
dont mutate bind_arguments incoming dictionary
The :paramref:`_orm.Session.execute.bind_arguments` dictionary is no longer
mutated when passed to :meth:`_orm.Session.execute` and similar; instead,
it's copied to an internal dictionary for state changes. Among other
things, this fixes and issue where the "clause" passed to the
:meth:`_orm.Session.get_bind` method would be incorrectly referring to the
:class:`_sql.Select` construct used for the "fetch" synchronization
strategy, when the actual query being emitted was a :class:`_dml.Delete` or
:class:`_dml.Update`. This would interfere with recipes for "routing
sessions".
John Bodley [Fri, 30 Sep 2022 01:58:58 +0000 (21:58 -0400)]
adjust MySQL view reflection for non-standard MySQL variants
Adjusted the regular expression used to match "CREATE VIEW" when
testing for views to work more flexibly, no longer requiring the
special keyword "ALGORITHM" in the middle, which was intended to be
optional but was not working correctly. The change allows view reflection
to work more completely on MySQL-compatible variants such as StarRocks.
Pull request courtesy John Bodley.
Mike Bayer [Sun, 2 Oct 2022 18:04:43 +0000 (14:04 -0400)]
experiment w/ docs formatter on SQLA 1.4
Enhanced the "{sql}" thing some more so that it maintains
these tags exactly as they were.
Note that the "{sql}" and "{stop}" tags are intended
to be on the Python code lines, not the SQL lines, so special
handling to find these, preserve them, then add them back
after python code is formatted is added here.
Mike Bayer [Sun, 25 Sep 2022 18:56:22 +0000 (14:56 -0400)]
warn for local-only column in remote side
A warning is emitted in ORM configurations when an explicit
:func:`_orm.remote` annotation is applied to columns that are local to the
immediate mapped class, when the referenced class does not include any of
the same table columns. Ideally this would raise an error at some point as
it's not correct from a mapping point of view.
Mike Bayer [Fri, 23 Sep 2022 19:17:57 +0000 (15:17 -0400)]
remove should_nest behavior for contains_eager()
Fixed regression for 1.4 in :func:`_orm.contains_eager` where the "wrap in
subquery" logic of :func:`_orm.joinedload` would be inadvertently triggered
for use of the :func:`_orm.contains_eager` function with similar statements
(e.g. those that use ``distinct()``, ``limit()`` or ``offset()``). This is
not appropriate for :func:`_orm.contains_eager` which has always had the
contract that the user-defined SQL statement is unmodified with the
exception of adding the appropriate columns.
Also includes an adjustment to the assertion in Label._make_proxy()
which was there to prevent a fixed label name from being anonymized;
if the label is already anonymous, the change should proceed.
This logic was being hit before the contains_eager behavior was
adjusted. With the adjustment, this code is not used.
Mike Bayer [Mon, 19 Sep 2022 13:40:40 +0000 (09:40 -0400)]
break out text() from TextualSelect for col matching
Fixed issue where mixing "*" with additional explicitly-named column
expressions within the columns clause of a :func:`_sql.select` construct
would cause result-column targeting to sometimes consider the label name or
other non-repeated names to be an ambiguous target.
Mike Bayer [Sat, 17 Sep 2022 14:33:55 +0000 (10:33 -0400)]
change verbiage stating exact compliance with RFC-1738
As long as we aren't using urlparse() to parse URLs,
we are not RFC-1738 compliant. As we accept underscores
in the scheme and not dashes or dots, we are not
RFC-1738 compliant, so emulate language like
that of PostgreSQL [1] that we "generally follow" this
scheme but include some exceptions.
Mike Bayer [Sat, 17 Sep 2022 14:18:56 +0000 (10:18 -0400)]
remove obtuse section about "bundled bind parameters"
Just looking for basics on insert in the first pages
of the tutorial I see this weird detour into something that
nobody ever uses and definitely isn't going to make sense
to the people I see complaining about our docs on twitter,
remove this. the tutorial probably needs a big sweep
for wordy obtuse things. the userbase is changing and
we really have a lot of brand-new-to-programming types coming
in.
Mike Bayer [Thu, 15 Sep 2022 22:54:31 +0000 (18:54 -0400)]
use get_event_loop() for python 3.6
per https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.get_running_loop
the get_event_loop method will eventually be an alias for
get_running_loop. the latter is not present in python 3.6
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
