Mike Bayer [Thu, 12 Jan 2023 16:25:39 +0000 (11:25 -0500)]
add with_loader_criteria() test for #8064 / #9091
test related to #8064, added after discussion #9091 which
requested this behavior for with_loader_criteria() where it was
found to be working as of this issue, just not tested
Mike Bayer [Tue, 10 Jan 2023 14:51:23 +0000 (09:51 -0500)]
fix ORM support for column-named bindparam() in crud .values()
Fixed bug / regression where using :func:`.bindparam()` with the same name
as a column in the :meth:`.Update.values` method of :class:`.Update`, as
well as the :meth:`.Insert.values` method of :class:`.Insert` in 2.0 only,
would in some cases silently fail to honor the SQL expression in which the
parameter were presented, replacing the expression with a new parameter of
the same name and discarding any other elements of the SQL expression, such
as SQL functions, etc. The specific case would be statements that were
constructed against ORM entities rather than plain :class:`.Table`
instances, but would occur if the statement were invoked with a
:class:`.Session` or a :class:`.Connection`.
:class:`.Update` part of the issue was present in both 2.0 and 1.4 and is
backported to 1.4.
For 1.4, also backports the sqlalchemy.testing.Variation update
to the variation() API.
Michael Gorven [Wed, 4 Jan 2023 17:30:42 +0000 (12:30 -0500)]
[asyncpg] Extract rowcount for SELECT statements
Added support to the asyncpg dialect to return the ``cursor.rowcount``
value for SELECT statements when available. While this is not a typical use
for ``cursor.rowcount``, the other PostgreSQL dialects generally provide
this value. Pull request courtesy Michael Gorven.
Mike Bayer [Mon, 2 Jan 2023 17:27:36 +0000 (12:27 -0500)]
add uber warning for 1.4
As we don't have any automatic deprecation warning for 2.0
unless SQLALCHEMY_WARN_20 is set, applications that are not
being monitored for deprecations have no way to guard against
2.0 being released on pypi unless they add a requirements
rule.
make sure we are putting out a major warning for people who
may have not noticed that SQLAlchemy 2.0 will break compatibility
with legacy use patterns.
Mike Bayer [Wed, 28 Dec 2022 17:04:07 +0000 (12:04 -0500)]
ensure whereclause, returning copied as tuples
Fixed issue in the internal SQL traversal for DML statements like
:class:`_dml.Update` and :class:`_dml.Delete` which would cause among other
potential issues, a specific issue using lambda statements with the ORM
update/delete feature.
Mike Bayer [Tue, 27 Dec 2022 17:29:38 +0000 (12:29 -0500)]
pass more contextual information to PyWrapper param create
Fixed issue in lambda SQL feature where the calculated type of a literal
value would not take into account the type coercion rules of the "compared
to type", leading to a lack of typing information for SQL expressions, such
as comparisons to :class:`.JSON` elements and similar.
Mike Bayer [Thu, 22 Dec 2022 23:14:31 +0000 (18:14 -0500)]
expand out Index if passed to "constraint"
Fixed bug where the PostgreSQL
:paramref:`_postgresql.OnConflictClause.constraint` parameter would accept
an :class:`.Index` object, however would not expand this index out into its
individual index expressions, instead rendering its name in an ON CONFLICT
ON CONSTRAINT clause, which is not accepted by PostgreSQL; the "constraint
name" form only accepts unique or exclude constraint names. The parameter
continues to accept the index but now expands it out into its component
expressions for the render.
Mike Bayer [Mon, 19 Dec 2022 20:15:35 +0000 (15:15 -0500)]
add joins_implicitly to column_valued()
Added parameter
:paramref:`.FunctionElement.column_valued.joins_implicitly`, which is
useful in preventing the "cartesian product" warning when making use of
table-valued or column-valued functions. This parameter was already
introduced for :meth:`.FunctionElement.table_valued` in :ticket:`7845`,
however it failed to be added for :meth:`.FunctionElement.column_valued`
as well.
Mike Bayer [Mon, 19 Dec 2022 13:34:51 +0000 (08:34 -0500)]
add exclusion for unusual chars in column names
Added new exclusion rule for third party dialects called
``unusual_column_name_characters``, which can be "closed" for third party
dialects that don't support column names with unusual characters such as
dots, slashes, or percent signs in them, even if the name is properly
quoted.
Mike Bayer [Fri, 16 Dec 2022 19:05:48 +0000 (14:05 -0500)]
dont call platform.architecture()
Fixed regression where the base compat module was calling upon
``platform.architecture()`` in order to detect some system properties,
which results in an over-broad system call against the system-level
``file`` call that is unavailable under some circumstances, including
within some secure environment configurations.
Mike Bayer [Thu, 15 Dec 2022 15:22:36 +0000 (10:22 -0500)]
implement literal_binds with expanding + bind_expression
Fixed bug where SQL compilation would fail (assertion fail in 2.0, NoneType
error in 1.4) when using an expression whose type included
:meth:`_types.TypeEngine.bind_expression`, in the context of an "expanding"
(i.e. "IN") parameter in conjunction with the ``literal_binds`` compiler
parameter.
Mike Bayer [Mon, 12 Dec 2022 23:05:07 +0000 (18:05 -0500)]
check index_list pragma for number of columns returned
Fixed regression caused by new support for reflection of partial indexes on
SQLite added in 1.4.45 for :ticket:`8804`, where the ``index_list`` pragma
command in very old versions of SQLite (possibly prior to 3.8.9) does not
return the current expected number of columns, leading to exceptions raised
when reflecting tables and indexes.
Mike Bayer [Mon, 12 Dec 2022 18:47:27 +0000 (13:47 -0500)]
catch all BaseException in pool and revert failed checkouts
Fixed a long-standing race condition in the connection pool which could
occur under eventlet/gevent monkeypatching schemes in conjunction with the
use of eventlet/gevent ``Timeout`` conditions, where a connection pool
checkout that's interrupted due to the timeout would fail to clean up the
failed state, causing the underlying connection record and sometimes the
database connection itself to "leak", leaving the pool in an invalid state
with unreachable entries. This issue was first identified and fixed in
SQLAlchemy 1.2 for :ticket:`4225`, however the failure modes detected in
that fix failed to accommodate for ``BaseException``, rather than
``Exception``, which prevented eventlet/gevent ``Timeout`` from being
caught. In addition, a block within initial pool connect has also been
identified and hardened with a ``BaseException`` -> "clean failed connect"
block to accommodate for the same condition in this location.
Big thanks to Github user @niklaus for their tenacious efforts in
identifying and describing this intricate issue.
Mike Bayer [Sun, 11 Dec 2022 16:27:52 +0000 (11:27 -0500)]
adjust for tox changes to passenv
Fixed issue in tox.ini file where changes in the tox 4.0 series to the
format of "passenv" caused tox to not function correctly, in particular
raising an error as of tox 4.0.6.
Mike Bayer [Fri, 9 Dec 2022 20:56:15 +0000 (15:56 -0500)]
look out for extras=None in freeze
Fixed issue where :meth:`_engine.Result.freeze` method would not work for
textual SQL using either :func:`_sql.text` or
:meth:`_engine.Connection.exec_driver_sql`.
Better syncronize async result docs with plain ones.
Removed non-functional method ``merge`` from :class:`_asyncio.AsyncResult`.
This method was non-functional and non-testes since the first introduction
of asyncio in SQLAlchemy.
Mike Bayer [Wed, 7 Dec 2022 20:54:59 +0000 (15:54 -0500)]
Oracle COLUMN_VALUE is a column name, not a keyword
Fixed issue in Oracle compiler where the syntax for
:meth:`.FunctionElement.column_valued` was incorrect, rendering the name
``COLUMN_VALUE`` without qualifying the source table correctly.
Mike Bayer [Mon, 5 Dec 2022 04:25:14 +0000 (23:25 -0500)]
adjustments for unreliable gc
sporadic (and at the moment persistent) test failures
related to aiosqlite seem to have in common that Python
gc stops working fully when we run a lot of tests with
aiosqlite. The failures are not limited to aiosqlite
as they are more involving places where we assume or
expect gc.collect() to get rid of things, and it doesn't.
Identify (based on reproducible case on the d3 CI runner)
the spots where this happens and add fixes.
test/orm/test_transaction.py test_gced_delete_on_rollback
has always been a very sensitive test with a lot of issues,
so here we move it to the test_memusage suite and limit
it only to when the memusage suite is running.
j00356287 [Mon, 5 Dec 2022 13:44:09 +0000 (08:44 -0500)]
doc change - Add new external dialect for openGauss
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
Added new external dialect for [openGauss](https://www.opengauss.org/en/).
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
wiseaidev [Mon, 5 Dec 2022 13:44:35 +0000 (08:44 -0500)]
Fixed an invalid syntax in an except statement
### Description
As the title suggests, I have fixed an invalid syntax in the docs for an `except` statement while reading the unusual.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Returned "GitHub Actions" support for py27 and py36 environments. (#8924)
GitHub recently upgraded the `ubuntu-latest` operating system label to
point to `ubuntu-22.04` instead of `ubuntu-22.04`. The `ubuntu-22.04` build
does not support Python 2.7 or 3.6.
In order to return support for CI Testing and release builds, the affected jobs
now utilize `include` and `exclude` commands in run matrixes to enable and
disable jobs under certain environments.
Jobs that requres Py27 and Py36 now run under an explicitly identified
`ubuntu-20.04` operating system. The majority of jobs require Py37 or higher,
which are all currently supported in the new `ubuntu-latest` operating system (
which points to `ubuntu-22.04`).
Although `ubuntu-20.04` should continue to receive support for several more
years, the `ubuntu-22.04` platform is likely to benefit from patch releases on
a faster schedule – so it is preferable to continue running all compatible
tests on `ubuntu-latest` rather than pinning everything to the earlier os
version.
Several jobs were also standardized to use the job "name" as a prefix, a
convention that most jobs in the workflows already adapted. This practice
greatly simplifies correlating failed tests to specific jobs.
Mike Bayer [Fri, 2 Dec 2022 22:00:10 +0000 (17:00 -0500)]
add spaces, leading underscore to oracle checks
Expand the test suite from #8708 which unfortunately did
not exercise the bound parameter codepaths completely.
Continued fixes for Oracle fix :ticket:`8708` released in 1.4.43 where
bound parameter names that start with underscores, which are disallowed by
Oracle, were still not being properly escaped in all circumstances.
Returned Github Actions support for py27 and py36.
GitHub recently upgraded the `ubuntu-latest` label from `ubuntu-20.04` to `ubuntu-22.04`.
The `ubuntu-22.04` image removed support for py27 and py36.
To return support, the affected jobs have been duplicated to `-legacy` versions.
The -legacy versions of jobs run py27 and py36 on a pinned `ubuntu-20.04` os.
The existing jobs continue to run py37+ on ubuntu-latest, as that platform may continue to benefit from patch releases on the python versions.
Federico Caselli [Sat, 19 Nov 2022 19:39:10 +0000 (20:39 +0100)]
Fix positional compiling bugs
Fixed a series of issues regarding positionally rendered bound parameters,
such as those used for SQLite, asyncpg, MySQL and others. Some compiled
forms would not maintain the order of parameters correctly, such as the
PostgreSQL ``regexp_replace()`` function as well as within the "nesting"
feature of the :class:`.CTE` construct first introduced in :ticket:`4123`.
Michael Gorven [Tue, 29 Nov 2022 23:36:19 +0000 (18:36 -0500)]
[sqlite] Reflect DEFERRABLE and INITIALLY options for foreign keys
Added support for the SQLite backend to reflect the "DEFERRABLE" and
"INITIALLY" keywords which may be present on a foreign key construct. Pull
request courtesy Michael Gorven.
Tobias Pfeiffer [Mon, 28 Nov 2022 12:52:31 +0000 (07:52 -0500)]
add partial index predicate to SQLiteDialect.get_indexes() result
Added support for reflection of expression-oriented WHERE criteria included
in indexes on the SQLite dialect, in a manner similar to that of the
PostgreSQL dialect. Pull request courtesy Tobias Pfeiffer.
Mike Bayer [Sat, 26 Nov 2022 16:03:45 +0000 (11:03 -0500)]
add new variation helper
I'm using a lot of @testing.combinations with either
a boolean True/False, or a series of string names, each indicating
some case to switch on. I want a descriptive name in the test
run (not True/False) and I don't want to compare strings.
So make a new helper around @combinations that provides an
object interface that has booleans inside of it, prints nicely
in the test output, raises an error if you name the case
incorrectly.
Mike Bayer [Fri, 25 Nov 2022 21:49:28 +0000 (16:49 -0500)]
improve column targeting issues with query_expression
Fixed issues in :func:`_orm.with_expression` where expressions that were
composed of columns within a subquery being SELECTed from, or when using
``.from_statement()``, would not render correct SQL **if** the expression
had a label name that matched the attribute which used
:func:`_orm.query_expression`, even when :func:`_orm.query_expression` had
no default expression. For the moment, if the :func:`_orm.query_expression`
**does** have a default expression, that label name is still used for that
default, and an additional label with the same name will be ignored.
Overall, this case is pretty thorny so further adjustments might be
warranted.
Michael Gorven [Thu, 24 Nov 2022 08:47:26 +0000 (03:47 -0500)]
Fix reflection of constraints in attached schemas
Backported a fix for SQLite reflection of unique constraints in attached
schemas, released in 2.0 as a small part of :ticket:`4379`. Previously,
unique constraints in attached schemas would be ignored by SQLite
reflection. Pull request courtesy Michael Gorven.
Mike Bayer [Wed, 23 Nov 2022 15:58:28 +0000 (10:58 -0500)]
add "merge" to viewonly cascades; propagate NO_RAISE when merging
Fixed bug where :meth:`_orm.Session.merge` would fail to preserve the
current loaded contents of relationship attributes that were indicated with
the :paramref:`_orm.relationship.viewonly` parameter, thus defeating
strategies that use :meth:`_orm.Session.merge` to pull fully loaded objects
from caches and other similar techniques. In a related change, fixed issue
where an object that contains a loaded relationship that was nonetheless
configured as ``lazy='raise'`` on the mapping would fail when passed to
:meth:`_orm.Session.merge`; checks for "raise" are now suspended within
the merge process assuming the :paramref:`_orm.Session.merge.load`
parameter remains at its default of ``True``.
Overall, this is a behavioral adjustment to a change introduced in the 1.4
series as of :ticket:`4994`, which took "merge" out of the set of cascades
applied by default to "viewonly" relationships. As "viewonly" relationships
aren't persisted under any circumstances, allowing their contents to
transfer during "merge" does not impact the persistence behavior of the
target object. This allows :meth:`_orm.Session.merge` to correctly suit one
of its use cases, that of adding objects to a :class:`.Session` that were
loaded elsewhere, often for the purposes of restoring from a cache.
Mike Bayer [Wed, 23 Nov 2022 22:44:30 +0000 (17:44 -0500)]
flake8 has very courteously referred us to another pyqca project and closed all comments, thank you! I'll get on that right away
pyqca/flake8-import-order does not seem to have a release or a commit
in the past two years, so while I have created an issue and PR [1] [2],
for now vendor our fork so we can get on with things.
Mike Bayer [Thu, 17 Nov 2022 01:11:18 +0000 (20:11 -0500)]
accommodate NULL format_type()
Made an adjustment to how the PostgreSQL dialect considers column types
when it reflects columns from a table, to accommodate for alternative
backends which may return NULL from the PG ``format_type()`` function.
Eitan Mosenkis [Mon, 14 Nov 2022 21:11:15 +0000 (23:11 +0200)]
Explicitly state what happens if `order_by` is called more than once. (#8791)
* Explicitly state what happens if `order_by` is called more than once.
The existing docs cover how to clear existing `order_by` clauses but don't actually describe the behavior of calling `order_by` multiple times with different clauses.
Mike Bayer [Mon, 14 Nov 2022 13:54:56 +0000 (08:54 -0500)]
add informative exception context for literal render
An informative re-raise is now thrown in the case where any "literal
bindparam" render operation fails, indicating the value itself and
the datatype in use, to assist in debugging when literal params
are being rendered in a statement.
Adjusted the test suite which tests the Mypy plugin to accommodate for
changes in Mypy 0.990 regarding how it handles message output, which affect
how sys.path is interpreted when determining if notes and errors should be
printed for particular files. The change broke the test suite as the files
within the test directory itself no longer produced messaging when run
under the mypy API.
Mike Bayer [Thu, 10 Nov 2022 22:01:58 +0000 (17:01 -0500)]
ensure anon_map is passed for most annotated traversals
We can cache the annotated cache key for Table, but
for selectables it's not safe, as it fails to pass the
anon_map along and creates many redudant structures in
observed test scenario. It is likely safe for a
Column that's mapped to a Table also, however this is
not implemented here. Will have to see if that part
needs adjusting.
Fixed critical memory issue identified in cache key generation, where for
very large and complex ORM statements that make use of lots of ORM aliases
with subqueries, cache key generation could produce excessively large keys
that were orders of magnitude bigger than the statement itself. Much thanks
to Rollo Konig Brock for their very patient, long term help in finally
identifying this issue.
Also within TypeEngine objects, when we generate elements
for instance variables, skip the None elements at least.
this also saves on tuple complexity.
Mike Bayer [Mon, 7 Nov 2022 23:40:03 +0000 (18:40 -0500)]
establish consistency for RETURNING column labels
For the PostgreSQL and SQL Server dialects only, adjusted the compiler so
that when rendering column expressions in the RETURNING clause, the "non
anon" label that's used in SELECT statements is suggested for SQL
expression elements that generate a label; the primary example is a SQL
function that may be emitting as part of the column's type, where the label
name should match the column's name by default. This restores a not-well
defined behavior that had changed in version 1.4.21 due to :ticket:`6718`,
:ticket:`6710`. The Oracle dialect has a different RETURNING implementation
and was not affected by this issue. Version 2.0 features an across the
board change for its widely expanded support of RETURNING on other
backends.
Fixed issue in the Oracle dialect where an INSERT statement that used
``insert(some_table).values(...).returning(some_table)`` against a full
:class:`.Table` object at once would fail to execute, raising an exception.
Mike Bayer [Fri, 11 Nov 2022 14:46:06 +0000 (09:46 -0500)]
repair --disable-asyncio parameter
Fixed issue where the ``--disable-asyncio`` parameter to the test suite
would fail to not actually run greenlet tests and would also not prevent
the suite from using a "wrapping" greenlet for the whole suite. This
parameter now ensures that no greenlet or asyncio use will occur within the
entire run when set.
Mike Bayer [Fri, 4 Nov 2022 16:48:43 +0000 (12:48 -0400)]
resolve synonyms in dictionary form of Session.get()
Improved "dictionary mode" for :meth:`_orm.Session.get` so that synonym
names which refer to primary key attribute names may be indicated in the
named dictionary.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
