Mike Bayer [Mon, 26 Jul 2021 22:06:41 +0000 (18:06 -0400)]
limit greenlet dependency to pypi-listed platforms
The setup requirements have been modified such ``greenlet`` is a default
requirement only for those platforms that are well known for ``greenlet``
to be installable and for which there is already a pre-built binary on
pypi; the current list is ``x86_64 aarch64 ppc64le amd64 win32``. For other
platforms, greenlet will not install by default, which should enable
installation and test suite running of SQLAlchemy 1.4 on platforms that
don't support ``greenlet``, excluding any asyncio features. In order to
install with the ``greenlet`` dependency included on a machine architecture
outside of the above list, the ``[asyncio]`` extra may be included by
running ``pip install sqlalchemy[asyncio]`` which will then attempt to
install ``greenlet``.
Additionally, the test suite has been repaired so that tests can complete
fully when greenlet is not installed, with appropriate skips for
asyncio-related tests.
João Sampaio [Wed, 11 Aug 2021 14:36:49 +0000 (11:36 -0300)]
Fix example with deprecated code
Just a few lines above, it is said that creating a mapping with `mapper()` is deprecated in favor of `mapper_registry.map_imperatively()`. This PR fixes an example in the documentation that still uses the old function.
Mike Bayer [Mon, 9 Aug 2021 18:43:53 +0000 (14:43 -0400)]
create concise + deterministic cache key for unboundload.options
Fixed issue in loader strategies where the use of the
:meth:`_orm.Load.options` method, particularly when nesting multiple calls,
would generate an overly long and more importantly non-deterministic cache
key, leading to very large cache keys which were also not allowing
efficient cache usage, both in terms of total memory used as well as number
of entries used in the cache itself.
Eric Masseran [Fri, 6 Aug 2021 08:11:09 +0000 (04:11 -0400)]
Dispatch independent ctes on compound select
Fix issue in :class:`_sql.CTE` where new :meth:`_sql.HasCTE.add_cte` method
added in version 1.4.21 / :ticket:`6752` failed to function correctly for
"compound select" structures such as :func:`_sql.union`,
:func:`_sql.union_all`, :func:`_sql.except`, etc. Pull request courtesy
Eric Masseran.
Mike Bayer [Fri, 23 Jul 2021 20:07:50 +0000 (16:07 -0400)]
add columns_clause_froms and related use cases
Added new attribute :attr:`_sql.Select.columns_clause_froms` that will
retrieve the FROM list implied by the columns clause of the
:class:`_sql.Select` statement. This differs from the old
:attr:`_sql.Select.froms` collection in that it does not perform any ORM
compilation steps, which necessarily deannotate the FROM elements and do
things like compute joinedloads etc., which makes it not an appropriate
candidate for the :meth:`_sql.Select.select_from` method. Additionally adds
a new parameter
:paramref:`_sql.Select.with_only_columns.maintain_column_froms` that
transfers this collection to :meth:`_sql.Select.select_from` before
replacing the columns collection.
In addition, the :attr:`_sql.Select.froms` is renamed to
:meth:`_sql.Select.get_final_froms`, to stress that this collection is not
a simple accessor and is instead calculated given the full state of the
object, which can be an expensive call when used in an ORM context.
Additionally fixes a regression involving the
:func:`_orm.with_only_columns` function to support applying criteria to
column elements that were replaced with either
:meth:`_sql.Select.with_only_columns` or :meth:`_orm.Query.with_entities` ,
which had broken as part of :ticket:`6503` released in 1.4.19.
Mike Bayer [Sat, 7 Aug 2021 15:02:59 +0000 (11:02 -0400)]
dont qualify literal_binds with literal_execute
this appears to be unnecessary and prevents end-user
literal_binds case from working.
Fixed issue where the ``literal_binds`` compiler flag, as used externally
to render bound parameters inline, would fail to work when used with a
certain class of parameters known as "literal_execute", which covers things
like LIMIT and OFFSET values for dialects where the drivers don't allow a
bound parameter, such as SQL Server's "TOP" clause. The issue locally
seemed to affect only the MSSQL dialect.
Mike Bayer [Thu, 5 Aug 2021 23:17:07 +0000 (19:17 -0400)]
accommodate for untracked boundparam lambda in offline_string
Fixed an issue in the ``CacheKey.to_offline_string()`` method used by the
dogpile.caching example where attempting to create a proper cache key from
the special "lambda" query generated by the lazy loader would fail to
include the parameter values, leading to an incorrect cache key.
Mike Bayer [Thu, 29 Jul 2021 14:10:28 +0000 (10:10 -0400)]
accommodate for cloned bindparams w/ maintain_key
Fixed issue where a bound parameter object that was "cloned" would cause a
name conflict in the compiler, if more than one clone of this parameter
were used at the same time in a single statement. This could occur in
particular with things like ORM single table inheritance queries that
indicated the same "discriminator" value multiple times in one query.
Mike Bayer [Tue, 27 Jul 2021 15:35:41 +0000 (11:35 -0400)]
accommodate plain core textual statements
Fixed issue where the horizontal sharding extension would not correctly
accommodate for a plain textual SQL statement passed to
:meth:`_orm.Session.execute`.
Fixed issue where the unit of work would internally use a 2.0-deprecated
SQL expression form, emitting a deprecation warning when SQLALCHEMY_WARN_20
were enabled.
Noting fancy. we could add checkbox and/or dropdowns if needed
**To test**: to go my fork. Link https://github.com/CaselIT/sqlalchemy/issues/new/choose
See docs at
https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-issue-forms
https://docs.github.com/en/communities/using-templates-to-encourage-useful-issues-and-pull-requests/syntax-for-githubs-form-schema
Mike Bayer [Wed, 21 Jul 2021 19:44:27 +0000 (15:44 -0400)]
Apply new uniquing rules for future ORM selects
Fixed issue where usage of the :meth:`_result.Result.unique` method with an
ORM result that included column expressions with unhashable types, such as
``JSON`` or ``ARRAY`` using non-tuples would silently fall back to using
the ``id()`` function, rather than raising an error. This now raises an
error when the :meth:`_result.Result.unique` method is used in a 2.0 style
ORM query. Additionally, hashability is assumed to be True for result
values of unknown type, such as often happens when using SQL functions of
unknown return type; if values are truly not hashable then the ``hash()``
itself will raise.
For legacy ORM queries, since the legacy :class:`_orm.Query` object
uniquifies in all cases, the old rules remain in place, which is to use
``id()`` for result values of unknown type as this legacy uniquing is
mostly for the purpose of uniquing ORM entities and not column values.
Mike Bayer [Tue, 20 Jul 2021 15:03:08 +0000 (11:03 -0400)]
dont warn for dictionary passed positionally
Fixed issue where use of the :paramref:`_sql.case.whens` parameter passing
a dictionary positionally and not as a keyword argument would emit a 2.0
deprecation warning, referring to the deprecation of passing a list
positionally. The dictionary format of "whens", passed positionally, is
still supported and was accidentally marked as deprecated.
Mike Bayer [Wed, 21 Jul 2021 15:18:01 +0000 (11:18 -0400)]
implement cache key for return_defaults token
Fixed critical caching issue where the ORM's persistence feature using
INSERT..RETURNING would cache an incorrect query when mixing the "bulk
save" and standard "flush" forms of INSERT.
Mike Bayer [Fri, 16 Jul 2021 15:34:13 +0000 (11:34 -0400)]
guard against unexpected weakref cleanups
Added some guards against ``KeyError`` in the event system to accommodate
the case that the interpreter is shutting down at the same time
:meth:`_engine.Engine.dispose` is being called, which would cause stack
trace warnings.
Mike Bayer [Fri, 16 Jul 2021 14:14:56 +0000 (10:14 -0400)]
reset key/name when TableValuedColumn is adapted
Fixed issue in new :meth:`_schema.Table.table_valued` method where the
resulting :class:`_sql.TableValuedColumn` construct would not respond
correctly to alias adaptation as is used throughout the ORM, such as for
eager loading, polymorphic loading, etc.
Mike Bayer [Thu, 15 Jul 2021 15:25:31 +0000 (11:25 -0400)]
limit None->null coercion to not occur with crud
Fixed issue where type-specific bound parameter handlers would not be
called upon in the case of using the :meth:`_sql.Insert.values` method with
the Python ``None`` value; in particular, this would be noticed when using
the :class:`_types.JSON` datatype as well as related PostgreSQL specific
types such as :class:`_postgresql.JSONB` which would fail to encode the
Python ``None`` value into JSON null, however the issue was generalized to
any bound parameter handler in conjunction with this specific method of
:class:`_sql.Insert`.
The issue with coercions forcing out ``null()`` may still impact
SQL expression usage as well; the change here is limited to crud
as the behavior there is relevant to some use cases, which may
need to be evaluated separately.
Mike Bayer [Thu, 15 Jul 2021 14:36:53 +0000 (10:36 -0400)]
apply list() around weakkeydictionary iteration
Fixed an issue where clearing of mappers during things like test suite
teardowns could cause a "dictionary changed size" warning during garbage
collection, due to iteration of a weak-referencing dictionary. A ``list()``
has been applied to prevent concurrent GC from affecting this operation.
Mike Bayer [Wed, 14 Jul 2021 14:29:50 +0000 (10:29 -0400)]
Ensure alias traversal block works when adapt_from_selectables present
Fixed regression which appeared in version 1.4.3 due to :ticket:`6060`
where rules that limit ORM adaptation of derived selectables interfered
with other ORM-adaptation based cases, in this case when applying
adaptations for a :func:`_orm.with_polymorphic` against a mapping which
uses a :func:`_orm.column_property` which in turn makes use of a scalar
select that includes a :func:`_orm.aliased` object of the mapped table.
Mike Bayer [Sat, 3 Jul 2021 23:48:55 +0000 (19:48 -0400)]
Adjust CTE recrusive col list to accommodate dupe col names
Fixed issue in CTE constructs where a recursive CTE that referred to a
SELECT that has duplicate column names, which are typically deduplicated
using labeling logic in 1.4, would fail to refer to the deduplicated label
name correctly within the WITH clause.
As part of this change we are also attempting to remove the
behavior of SelectStatementGrouping forcing off the "asfrom"
contextual flag, which will have the result of additional labeling
being applied to some UNION and similar statements when they are
interpreted as subqueries. To maintain compatibility with
"grouping", the Grouping/SelectStatementGrouping are now broken
out into two separate compiler cases, as the "asfrom" logic appears
to be tailored towards table valued SELECTS as column expressions.
Mike Bayer [Mon, 12 Jul 2021 22:19:08 +0000 (18:19 -0400)]
Extract format_constraint truncation rules to ON CONFLICT
Fixed issue where a too-long constraint name rendered as part of the "ON
CONFLICT ON CONSTRAINT" element of the :class:`_postgresql.Insert`
construct due to naming convention generation would not correctly truncate
the name in the same way that it normally renders within a CREATE TABLE
statement, thus producing a non-matching and too-long constraint name.
Mike Bayer [Tue, 6 Jul 2021 15:26:53 +0000 (11:26 -0400)]
labeling refactor
To service #6718 and #6710, the system by which columns are
given labels in a SELECT statement as well as the system that
gives them keys in a .c or .selected_columns collection have
been refactored to provide a single source of truth for
both, in constrast to the previous approach that included
similar logic repeated in slightly different ways.
Main ideas:
1. ColumnElement attributes ._label, ._anon_label, ._key_label
are renamed to include the letters "tq", meaning
"table-qualified" - these labels are only used when rendering
a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its
label style; as this label style is primarily legacy, the
"tq" names should be isolated so that in a 2.0 style application
these aren't being used at all
2. The means by which the "labels" and "proxy keys" for the elements
of a SELECT has been centralized to a single source of truth;
previously, the three of _generate_columns_plus_names,
_generate_fromclause_column_proxies, and _column_naming_convention
all had duplicated rules between them, as well as that there
were a little bit of labeling rules in compiler._label_select_column
as well; by this we mean that the various "anon_label" "anon_key"
methods on ColumnElement were called by all four of these methods,
where there were many cases where it was necessary that one method
comes up with the same answer as another of the methods. This
has all been centralized into _generate_columns_plus_names
for all the names except the "proxy key", which is generated
by _column_naming_convention.
3. compiler._label_select_column has been rewritten to both not make
any naming decisions nor any "proxy key" decisions, only whether
to label or not to label; the _generate_columns_plus_names method
gives it the information, where the proxy keys come from
_column_naming_convention; previously, these proxy keys were matched
based on restatement of similar (but not really the same) logic in
two places. The heuristics of "whether to label or not to label"
are also reorganized to be much easier to read and understand.
4. a new method compiler._label_returning_column is added for dialects
to use in their "generate returning columns" methods. A
github search reveals a small number of third party dialects also
doing this using the prior _label_select_column method so we
try to make sure _label_select_column continues to work the
exact same way for that specific use case; for the "SELECT" use
case it now needs
5. After some attempts to do it different ways, for the case where
_proxy_key is giving us some kind of anon label, we are hard
changing it to "_no_label" right now, as there's not currently
a way to fully match anonymized labels from stmt.c or
stmt.selected_columns to what will be in the result map. The
idea of "_no_label" is to encourage the user to use label('name')
for columns they want to be able to target by string name that
don't have a natural name.
Mike Bayer [Mon, 12 Jul 2021 18:28:19 +0000 (14:28 -0400)]
implement independent CTEs
Added new method :meth:`_sql.HasCTE.add_cte` to each of the
:func:`_sql.select`, :func:`_sql.insert`, :func:`_sql.update` and
:func:`_sql.delete` constructs. This method will add the given
:class:`_sql.CTE` as an "independent" CTE of the statement, meaning it
renders in the WITH clause above the statement unconditionally even if it
is not otherwise referenced in the primary statement. This is a popular use
case on the PostgreSQL database where a CTE is used for a DML statement
that runs against database rows independently of the primary statement.
Mike Bayer [Sun, 11 Jul 2021 23:40:59 +0000 (19:40 -0400)]
represent tablesample.sampling as FunctionElement in all cases
Fixed regression where the :func:`_sql.tablesample` construct would fail to
be executable when constructed given a floating-point sampling value not
embedded within a SQL function.
Mike Bayer [Sun, 11 Jul 2021 23:23:40 +0000 (19:23 -0400)]
repair schema_translate_map for schema type use cases
Fixed issue where the PostgreSQL ``ENUM`` datatype as embedded in the
``ARRAY`` datatype would fail to emit correctly in create/drop when the
``schema_translate_map`` feature were also in use. Additionally repairs a
related issue where the same ``schema_translate_map`` feature would not
work for the ``ENUM`` datatype in combination with a ``CAST``, that's also
intrinsic to how the ``ARRAY(ENUM)`` combination works on the PostgreSQL
dialect.
Mike Bayer [Fri, 2 Jul 2021 15:23:20 +0000 (11:23 -0400)]
implement deferred scalarobject history load
Modified the approach used for history tracking of scalar object
relationships that are not many-to-one, i.e. one-to-one relationships that
would otherwise be one-to-many. When replacing a one-to-one value, the
"old" value that would be replaced is no longer loaded immediately, and is
instead handled during the flush process. This eliminates an historically
troublesome lazy load that otherwise often occurs when assigning to a
one-to-one attribute, and is particularly troublesome when using
"lazy='raise'" as well as asyncio use cases.
This change does cause a behavioral change within the
:meth:`_orm.AttributeEvents.set` event, which is nonetheless currently
documented, which is that the event applied to such a one-to-one attribute
will no longer receive the "old" parameter if it is unloaded and the
:paramref:`_orm.relationship.active_history` flag is not set. As is
documented in :meth:`_orm.AttributeEvents.set`, if the event handler needs
to receive the "old" value when the event fires off, the active_history
flag must be established either with the event listener or with the
relationship. This is already the behavior with other kinds of attributes
such as many-to-one and column value references.
The change additionally will defer updating a backref on the "old" value
in the less common case that the "old" value is locally present in the
session, but isn't loaded on the relationship in question, until the
next flush occurs. If this causes an issue, again the normal
:paramref:`_orm.relationship.active_history` flag can be set to ``True``
on the relationship.
A private flag which restores the old value is retained for now,
as support within relevant test suites to exercise the old and
new behaviors together. This is so that if the behavioral change
produces problems we have test harnesses set up to further examine these
behaviors. The "legacy" style can go away in 2.0 or in a much later
1.4 release.
Mike Bayer [Mon, 5 Jul 2021 18:37:58 +0000 (14:37 -0400)]
Add additional support to honor _proxy_key in Core selects
Fixed ORM regression where ad-hoc label names generated for hybrid
properties and potentially other similar types of ORM-enabled expressions
would usually be propagated outwards through subqueries, allowing the name
to be retained in the final keys of the result set even when selecting from
subqueries. Additional state is now tracked in this case that isn't lost
when a hybrid is selected out of a Core select / subquery.
as we have removed things like column.label() from
ORM, since we now have to export the cols with the same names
as what we will render, experiment with giving a greater role
to the _proxy_key annotation so that a desired name can be
carried through more transarently.
Mike Bayer [Wed, 30 Jun 2021 20:04:07 +0000 (16:04 -0400)]
clear new Query._memoized_select_entities in _from_selectable
Fixed regression caused in 1.4.19 due to #6503 and related involving
:meth:`_orm.Query.with_entities` where the new structure used would be
inappropriately transferred to an enclosing :class:`_orm.Query` when making
use of set operations such as :meth:`_orm.Query.union`, causing the JOIN
instructions within to be applied to the outside query as well.
Mike Bayer [Wed, 30 Jun 2021 14:52:09 +0000 (10:52 -0400)]
Ensure compiler uses quote_schema hook for translates renders
Fixed regression where the special dotted-schema name handling for the SQL
Server dialect would not function correctly if the dotted schema name were
used within the ``schema_translate_map`` feature.
Mike Bayer [Wed, 30 Jun 2021 13:22:00 +0000 (09:22 -0400)]
apply quoting to "ON CONSTRAINT" symbol
Fixed issue in :meth:`_postgresql.Insert.on_conflict_do_nothing` and
:meth:`_postgresql.Insert.on_conflict_do_update` where the name of a unique
constraint passed as the ``constraint`` parameter would not be properly
quoted if it contained characters which required quoting.
Miłosz Stypiński [Thu, 24 Jun 2021 16:21:30 +0000 (12:21 -0400)]
Be less fiscal regarding validators functions
Adjusted the check in the mapper for a callable object that is used as a
``@validates`` validator function or a ``@reconstructor`` reconstruction
function, to check for "callable" more liberally such as to accommodate
objects based on fundamental attributes like ``__func__`` and
``__call___``, rather than testing for ``MethodType`` / ``FunctionType``,
allowing things like cython functions to work properly. Pull request
courtesy Miłosz Stypiński.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
