Mike Bayer [Fri, 5 Aug 2022 21:25:05 +0000 (17:25 -0400)]
deep compare CTEs before considering them conflicting
Fixed issue where referencing a CTE multiple times in conjunction with a
polymorphic SELECT could result in multiple "clones" of the same CTE being
constructed, which would then trigger these two CTEs as duplicates. To
resolve, the two CTEs are deep-compared when this occurs to ensure that
they are equivalent, then are treated as equivalent.
Mike Bayer [Wed, 3 Aug 2022 16:08:54 +0000 (12:08 -0400)]
send in the dragons on async_scoped_session
make it clear that async_scoped_session.remove() must
be called, else memory will build up. Generally
discourage the whole pattern as well, as this is a
"framework" pattern and we don't really want to be supporting
frameworks. Also indicate that scopefunc must be idempotent
and lightweight.
Fixed issue where the SQL Server dialect's query for the current isolation
level would fail on Azure Synapse Analytics, due to the way in which this
database handles transaction rollbacks after an error has occurred. The
initial query has been modified to no longer rely upon catching an error
when attempting to detect the appropriate system view. Additionally, to
better support this database's very specific "rollback" behavior,
implemented new parameter ``ignore_no_transaction_on_rollback`` indicating
that a rollback should ignore Azure Synapse error 'No corresponding
transaction found. (111214)', which is raised if no transaction is present
in conflict with the Python DBAPI.
Mike Bayer [Tue, 2 Aug 2022 18:51:49 +0000 (14:51 -0400)]
reword yield_per a bit more
I'm still not satisified with this section as it is still
too wordy and dense, but at least let's put a better description
of what yield_per actually is and why one might use it at the top.
Mike Bayer [Tue, 2 Aug 2022 15:34:35 +0000 (11:34 -0400)]
fix up SAVEPOINT docs
these contained a factual error that the entire session is
expired, which is no longer the case (I can't find exactly
when this was changed). Additionally, added a PostgreSQL
specific example w/ IntegrityError as this is the most
common case for this. Tried to tighten up other language
and make it as clear as possible.
Mike Bayer [Mon, 1 Aug 2022 14:29:13 +0000 (10:29 -0400)]
repair psycopg2 (and psycopg) multiple hosts format
Fixed issue in psycopg2 dialect where the "multiple hosts" feature
implemented for :ticket:`4392`, where multiple ``host:port`` pairs could be
passed in the query string as
``?host=host1:port1&host=host2:port2&host=host3:port3`` was not implemented
correctly, as it did not propagate the "port" parameter appropriately.
Connections that didn't use a different "port" likely worked without issue,
and connections that had "port" for some of the entries may have
incorrectly passed on that hostname. The format is now corrected to pass
hosts/ports appropriately.
As part of this change, maintained support for another multihost style that
worked unintentionally, which is comma-separated
``?host=h1,h2,h3&port=p1,p2,p3``. This format is more consistent with
libpq's query-string format, whereas the previous format is inspired by a
different aspect of libpq's URI format but is not quite the same thing.
If the two styles are mixed together, an error is raised as this is
ambiguous.
Mike Bayer [Tue, 19 Jul 2022 14:50:05 +0000 (10:50 -0400)]
check for TypeDecorator when handling getitem
Fixed issue where :class:`.TypeDecorator` would not correctly proxy the
``__getitem__()`` operator when decorating the :class:`.ARRAY` datatype,
without explicit workarounds.
Federico Caselli [Fri, 17 Jun 2022 21:12:39 +0000 (23:12 +0200)]
add shield() in aexit
Added ``asyncio.shield()`` to the connection and session release process
specifically within the ``__aexit__()`` context manager exit, when using
:class:`.AsyncConnection` or :class:`.AsyncSession` as a context manager
that releases the object when the context manager is complete. This appears
to help with task cancellation when using alternate concurrency libraries
such as ``anyio``, ``uvloop`` that otherwise don't provide an async context
for the connection pool to release the connection properly during task
cancellation.
Mike Bayer [Sun, 17 Jul 2022 15:32:27 +0000 (11:32 -0400)]
use concat() directly for contains, startswith, endswith
Adjusted the SQL compilation for string containment functions
``.contains()``, ``.startswith()``, ``.endswith()`` to force the use of the
string concatenation operator, rather than relying upon the overload of the
addition operator, so that non-standard use of these operators with for
example bytestrings still produces string concatenation operators.
To accommodate this, needed to add a new _rconcat operator function,
which is private, as well as a fallback in concat_op() that works
similarly to Python builtin ops.
Mike Bayer [Mon, 11 Jul 2022 01:24:17 +0000 (21:24 -0400)]
support "SELECT *" for ORM queries
A :func:`_sql.select` construct that is passed a sole '*' argument for
``SELECT *``, either via string, :func:`_sql.text`, or
:func:`_sql.literal_column`, will be interpreted as a Core-level SQL
statement rather than as an ORM level statement. This is so that the ``*``,
when expanded to match any number of columns, will result in all columns
returned in the result. the ORM- level interpretation of
:func:`_sql.select` needs to know the names and types of all ORM columns up
front which can't be achieved when ``'*'`` is used.
If ``'*`` is used amongst other expressions simultaneously with an ORM
statement, an error is raised as this can't be interpreted correctly by the
ORM.
Mike Bayer [Thu, 7 Jul 2022 15:44:09 +0000 (11:44 -0400)]
document using fetch() with Oracle
We implemented working FETCH support, but it's not
yet implied by limit/offset. The docs make no mention
that this is available which is very misleading including
to maintainers. Make it clear that fetch() support is
there right now, it's just not yet implicit with
limit/offset.
Mike Bayer [Wed, 6 Jul 2022 01:05:18 +0000 (21:05 -0400)]
generalize sql server check for id col to accommodate ORM cases
Fixed issues that prevented the new usage patterns for using DML with ORM
objects presented at :ref:`orm_dml_returning_objects` from working
correctly with the SQL Server pyodbc dialect.
Here we add a step to look in compile_state._dict_values more thoroughly
for the keys we need to determine "identity insert" or not, and also
add a new compiler variable dml_compile_state so that we can skip the
ORM's compile_state if present.
Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions
[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
Mike Bayer [Thu, 30 Jun 2022 23:10:06 +0000 (19:10 -0400)]
repair yield_per for non-SS dialects and add new options
Implemented new :paramref:`_engine.Connection.execution_options.yield_per`
execution option for :class:`_engine.Connection` in Core, to mirror that of
the same :ref:`yield_per <orm_queryguide_yield_per>` option available in
the ORM. The option sets both the
:paramref:`_engine.Connection.execution_options.stream_results` option at
the same time as invoking :meth:`_engine.Result.yield_per`, to provide the
most common streaming result configuration which also mirrors that of the
ORM use case in its usage pattern.
Fixed bug in :class:`_engine.Result` where the usage of a buffered result
strategy would not be used if the dialect in use did not support an
explicit "server side cursor" setting, when using
:paramref:`_engine.Connection.execution_options.stream_results`. This is in
error as DBAPIs such as that of SQLite and Oracle already use a
non-buffered result fetching scheme, which still benefits from usage of
partial result fetching. The "buffered" strategy is now used in all
cases where :paramref:`_engine.Connection.execution_options.stream_results`
is set.
Added :meth:`.FilterResult.yield_per` so that result implementations
such as :class:`.MappingResult`, :class:`.ScalarResult` and
:class:`.AsyncResult` have access to this method.
Mike Bayer [Tue, 28 Jun 2022 22:55:19 +0000 (18:55 -0400)]
produce column copies up the whole hierarchy first
Fixed issue where a hierarchy of classes set up as an abstract or mixin
declarative classes could not declare standalone columns on a superclass
that would then be copied correctly to a :class:`_orm.declared_attr`
callable that wanted to make use of them on a descendant class.
Originally it looked like this would produce an ordering change,
however an adjustment to the flow for produce_column_copies
has avoided that for now.
Mike Bayer [Fri, 24 Jun 2022 14:31:46 +0000 (10:31 -0400)]
add fallback for old mutable format
Fixed regression caused by :ticket:`8133` where the pickle format for
mutable attributes was changed, without a fallback to recognize the old
format, causing in-place upgrades of SQLAlchemy to no longer be able to
read pickled data from previous versions. A check plus a fallback for the
old format is now in place.
Mike Bayer [Thu, 23 Jun 2022 15:15:19 +0000 (11:15 -0400)]
refine _include_fn to not include sibling mappers
Fixed regression caused by :ticket:`8064` where a particular check for
column correspondence was made too liberal, resulting in incorrect
rendering for some ORM subqueries such as those using
:meth:`.PropComparator.has` or :meth:`.PropComparator.any` in conjunction
with joined-inheritance queries that also use legacy aliasing features.
Mike Bayer [Wed, 22 Jun 2022 22:50:35 +0000 (18:50 -0400)]
sub-categorize special function forms
this is the tutorial, which should have some semblence of
not getting too far into the weeds. however, as we dont
really have other places to explain SQL concepts, and SQL
functions have a lot of them, we dont have another home right
now. so at least further sub-categorize window functions,
table/column valued functions, and WITHIN GROUP into an
"advanced function techniques" section with a disclaimer that
these are less common use cases.
Mike Bayer [Mon, 13 Jun 2022 15:46:28 +0000 (11:46 -0400)]
rework ORM mapping docs
prepare docs for newly incoming mapper styles, including
new dataclass mapping. move the existing dataclass/attrs
docs all into their own section and try to improve organization
and wording into the relatively recent "mapping styles"
document.
Gord Thompson [Tue, 14 Jun 2022 16:09:04 +0000 (10:09 -0600)]
Allow NUMERIC()/DECIMAL() IDENTITY columns
Fixed issue where :class:`.Table` objects that made use of IDENTITY columns
with a :class:`.Numeric` datatype would produce errors when attempting to
reconcile the "autoincrement" column, preventing construction of the
:class:`.Column` from using the :paramref:`.Column.autoincrement` parameter
as well as emitting errors when attempting to invoke an :class:`.Insert`
construct.
Mike Bayer [Tue, 14 Jun 2022 19:41:31 +0000 (15:41 -0400)]
pickle mutable parents according to key
Fixed bug in :class:`.Mutable` where pickling and unpickling of an ORM
mapped instance would not correctly restore state for mappings that
contained multiple :class:`.Mutable`-enabled attributes.
Mike Bayer [Thu, 9 Jun 2022 01:35:02 +0000 (21:35 -0400)]
restore parameter escaping for public methods
Adjusted the fix made for :ticket:`8056` which adjusted the escaping of
bound parameter names with special characters such that the escaped names
were translated after the SQL compilation step, which broke a published
recipe on the FAQ illustrating how to merge parameter names into the string
output of a compiled SQL string. The change restores the escaped names that
come from ``compiled.params`` and adds a conditional parameter to
:meth:`.SQLCompiler.construct_params` named ``escape_names`` that defaults
to ``True``, restoring the old behavior by default.
Mike Bayer [Thu, 9 Jun 2022 13:53:43 +0000 (09:53 -0400)]
dont transfer __weakref__ to regenerated class
Repaired a deprecation warning class decorator that was preventing key
objects such as :class:`_engine.Connection` from having a proper
``__weakref__`` attribute, causing operations like Python standard library
``inspect.getmembers()`` to fail.
Fixed issue where a :func:`_orm.with_loader_criteria` option could not be
pickled, as is necessary when it is carried along for propagation to lazy
loaders in conjunction with a caching scheme. Currently, the only form that
is supported as picklable is to pass the "where criteria" as a fixed
module-level callable function that produces a SQL expression. An ad-hoc
"lambda" can't be pickled, and a SQL expression object is usually not fully
picklable directly.
Mike Bayer [Tue, 7 Jun 2022 19:00:20 +0000 (15:00 -0400)]
fix race conditions in lambda statements
Fixed multiple observed race conditions related to :func:`.lambda_stmt`,
including an initial "dogpile" issue when a new Python code object is
initially analyzed among multiple simultaneous threads which created both a
performance issue as well as some internal corruption of state.
Additionally repaired observed race condition which could occur when
"cloning" an expression construct that is also in the process of being
compiled or otherwise accessed in a different thread due to memoized
attributes altering the ``__dict__`` while iterated, for Python versions
prior to 3.10; in particular the lambda SQL construct is sensitive to this
as it holds onto a single statement object persistently. The iteration has
been refined to use ``dict.copy()`` with or without an additional iteration
instead.
Mike Bayer [Tue, 7 Jun 2022 13:40:26 +0000 (09:40 -0400)]
graceful degrade for FKs not reflectable
Fixed bugs involving the :paramref:`.Table.include_columns` and the
:paramref:`.Table.resolve_fks` parameters on :class:`.Table`; these
little-used parameters were apparently not working for columns that refer
to foreign key constraints.
In the first case, not-included columns that refer to foreign keys would
still attempt to create a :class:`.ForeignKey` object, producing errors
when attempting to resolve the columns for the foreign key constraint
within reflection; foreign key constraints that refer to skipped columns
are now omitted from the table reflection process in the same way as
occurs for :class:`.Index` and :class:`.UniqueConstraint` objects with the
same conditions. No warning is produced however, as we likely want to
remove the include_columns warnings for all constraints in 2.0.
In the latter case, the production of table aliases or subqueries would
fail on an FK related table not found despite the presence of
``resolve_fks=False``; the logic has been repaired so that if a related
table is not found, the :class:`.ForeignKey` object is still proxied to the
aliased table or subquery (these :class:`.ForeignKey` objects are normally
used in the production of join conditions), but it is sent with a flag that
it's not resolvable. The aliased table / subquery will then work normally,
with the exception that it cannot be used to generate a join condition
automatically, as the foreign key information is missing. This was already
the behavior for such foreign key constraints produced using non-reflection
methods, such as joining :class:`.Table` objects from different
:class:`.MetaData` collections.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
