Mike Bayer [Fri, 27 May 2022 20:07:01 +0000 (16:07 -0400)]
remove "deannotate" from column_property expression
Fixed issue where using a :func:`_orm.column_property` construct containing
a subquery against an already-mapped column attribute would not correctly
apply ORM-compilation behaviors to the subquery, including that the "IN"
expression added for a single-table inherits expression would fail to be
included.
This fix involves a few tweaks in the ORM adaptation logic,
including a missing "parententity" adaptation on the mapper
side. The specific mechanics here have a lot of moving parts
so we will continue to add tests to assert these cases. In
particular a more complete test for issue #2316 is added
that was relying upon the deannotate happening here.
Mike Bayer [Sun, 29 May 2022 16:07:46 +0000 (12:07 -0400)]
move bindparam quote application from compiler to default
in 296c84313ab29bf9599634f3 for #5653 we generalized Oracle's
parameter escaping feature into the compiler, so that it could also
work for PostgreSQL. The compiler used quoted names within parameter
dictionaries, which then led to the complexity that all functions
which interpreted keys from the compiled_params dict had to
also quote the param names to use the dictionary. This
extra complexity was not added to the ORM peristence.py however,
which led to the versioning id feature being broken as well as
other areas where persistence.py relies on naming schemes present
in context.compiled_params. It also was not added to the
"processors" lookup which led to #8053, that added this escaping
to that part of the compiler.
To both solve the whole problem as well as simplify the compiler
quite a bit, move the actual application of the escaped names
to be as late as possible, when default.py builds the final list
of parameters. This is more similar to how it worked previously
where OracleExecutionContext would be late-applying these
escaped names. This re-establishes context.compiled_params as
deterministically named regardless of dialect in use and moves
out the complexity of the quoted param names to be only at the
cursor.execute stage.
Fixed bug, likely a regression from 1.3, where usage of column names that
require bound parameter escaping, more concretely when using Oracle with
column names that require quoting such as those that start with an
underscore, or in less common cases with some PostgreSQL drivers when using
column names that contain percent signs, would cause the ORM versioning
feature to not work correctly if the versioning column itself had such a
name, as the ORM assumes certain bound parameter naming conventions that
were being interfered with via the quotes. This issue is related to
:ticket:`8053` and essentially revises the approach towards fixing this,
revising the original issue :ticket:`5653` that created the initial
implementation for generalized bound-parameter name quoting.
Mike Bayer [Wed, 25 May 2022 12:47:29 +0000 (08:47 -0400)]
apply bindparam escape name to processors dictionary
Fixed SQL compiler issue where the "bind processing" function for a bound
parameter would not be correctly applied to a bound value if the bound
parameter's name were "escaped". Concretely, this applies, among other
cases, to Oracle when a :class:`.Column` has a name that itself requires
quoting, such that the quoting-required name is then used for the bound
parameters generated within DML statements, and the datatype in use
requires bind processing, such as the :class:`.Enum` datatype.
Andrew Brookins [Sun, 22 May 2022 20:24:15 +0000 (16:24 -0400)]
Add a note on using server_onupdate=FetchedValue()
Add a note on using `server_onupdate=FetchedValue()` when using SQL expressions with `onupdate`.
My team encountered an issue with using a SQL expression with `onupdate`.
Despite the dialect (PG) supporting `RETURNING`, we needed to mark the column with
`server_onupdate=FetchedValue()` in order to get the column used with `onupdate`
to appear in the `RETURNING` clause of `UPDATE` statements.
This was not clear from the documentation, so I want to make it crystal clear for other
folks defining similar columns.
valievkarim [Wed, 18 May 2022 20:24:41 +0000 (16:24 -0400)]
Include new MySQL error code 4031 for MySQL disconnect check
Added disconnect code for MySQL error 4031, introduced in MySQL >= 8.0.24,
indicating connection idle timeout exceeded. In particular this repairs an
issue where pre-ping could not reconnect on a timed-out connection. Pull
request courtesy valievkarim.
Mike Bayer [Fri, 13 May 2022 20:08:34 +0000 (16:08 -0400)]
render col name in on conflict set clause, not given key
Fixed bug where the PostgreSQL :meth:`_postgresql.Insert.on_conflict`
method and the SQLite :meth:`_sqlite.Insert.on_conflict` method would both
fail to correctly accommodate a column with a separate ".key" when
specifying the column using its key name in the dictionary passed to
``set_``, as well as if the :attr:`_sqlite.Insert.excluded` or
:attr:`_postgresql.Insert.excluded` collection were used as the dictionary
directly.
Mike Bayer [Fri, 13 May 2022 19:43:53 +0000 (15:43 -0400)]
raise for same param name in expanding + non expanding
An informative error is raised if two individual :class:`.BindParameter`
objects share the same name, yet one is used within an "expanding" context
(typically an IN expression) and the other is not; mixing the same name in
these two different styles of usage is not supported and typically the
``expanding=True`` parameter should be set on the parameters that are to
receive list values outside of IN expressions (where ``expanding`` is set
by default).
Mike Bayer [Sat, 14 May 2022 14:25:53 +0000 (10:25 -0400)]
adjust log stacklevel for py3.11.0b1; enable greenlet
Fixed issue where support for logging "stacklevel" implemented in
:ticket:`7612` required adjustment to work with recently released Python
3.11.0b1, also repairs the unit tests which tested this feature.
Install greenlet from a py311 compat patch.
re: the stacklevel thing, this is going to be very inconvenient
if we have to keep hardcoding numbers everywhere for every
new python version
Mike Bayer [Mon, 9 May 2022 14:27:51 +0000 (10:27 -0400)]
dont use the label convention for memoized entities
Fixed issue where ORM results would apply incorrect key names to the
returned :class:`.Row` objects in the case where the set of columns to be
selected were changed, such as when using
:meth:`.Select.with_only_columns`.
khashashin [Tue, 3 May 2022 20:00:59 +0000 (22:00 +0200)]
docs(types) Fix missing import from sqlalchemy (#7978)
* docs(types) Fix missing import from sqlalchemy
The sample code is missing the import of Enum from sqlalchemy, which might confuse the reader, since we are using another enum type from Python itself here. So it makes sense to clarify that here.
Mike Bayer [Tue, 3 May 2022 12:58:27 +0000 (08:58 -0400)]
bypass pyodbc default server version / set charset
Further adjustments to the MySQL PyODBC dialect to allow for complete
connectivity, which was previously still not working despite fixes in
:ticket:`7871`.
Mike Bayer [Sun, 1 May 2022 16:28:36 +0000 (12:28 -0400)]
use bindparam_type in BinaryElementImpl._post_coercion if available
Fixed an issue where using :func:`.bindparam` with no explicit data or type
given could be coerced into the incorrect type when used in expressions
such as when using :meth:`.ARRAY.comparator.any` and
:meth:`.ARRAY.comparator.all`.
Mike Bayer [Tue, 26 Apr 2022 19:02:37 +0000 (15:02 -0400)]
repair fetch_setting call in mysql pyodbc dialect
Fixed a regression in the untested MySQL PyODBC dialect caused by the fix
for :ticket:`7518` in version 1.4.32 where an argument was being propagated
incorrectly upon first connect, leading to a ``TypeError``.
in 6f02d5edd88fe2475629438b0730181a2b00c5fe some cleanup
to ForeignKey repaired the use case of ForeignKey objects
referring to table name alone, by adding more robust
column resolution logic. This change also fixes an issue
where the "referred column" naming convention key uses the
resolved referred column earlier than usual when a
ForeignKey is setting up its constraint.
Fixed bug where :class:`.ForeignKeyConstraint` naming conventions using the
``referred_column_0`` naming convention key would not work if the foreign
key constraint were set up as a :class:`.ForeignKey` object rather than an
explicit :class:`.ForeignKeyConstraint` object. As this change makes use of
a backport of some fixes from version 2.0, an additional little-known
feature that has likely been broken for many years is also fixed which is
that a :class:`.ForeignKey` object may refer to a referred table by name of
the table alone without using a column name, if the name of the referent
column is the same as that of the referred column.
The ``referred_column_0`` naming convention key was not previously not
tested with the :class:`.ForeignKey` object, only
:class:`.ForeignKeyConstraint`, and this bug reveals that the feature has
never worked correctly unless :class:`.ForeignKeyConstraint` is used for
all FK constraints. This bug traces back to the original introduction of
the feature introduced for :ticket:`3989`.
Mike Bayer [Fri, 22 Apr 2022 14:57:00 +0000 (10:57 -0400)]
properly type array element in any() / all()
Fixed bug in :class:`.ARRAY` datatype in combination with :class:`.Enum` on
PostgreSQL where using the ``.any()`` method to render SQL ANY(), given
members of the Python enumeration as arguments, would produce a type
adaptation failure on all drivers.
Mike Bayer [Fri, 22 Apr 2022 14:27:38 +0000 (10:27 -0400)]
fix memory leak in resultproxy.c
the error raised for non-existent attribute didn't free
the "name" string, causing a leak.
Fixed a memory leak in the C extensions which could occur when calling upon
named members of :class:`.Row` when the member does not exist under Python
3; in particular this could occur during numpy transformations when it
attempts to call members such as ``.__array__``, but the issue was
surrounding any ``AttributeError`` thrown by the :class:`.Row` object. This
issue does not apply to version 2.0 which has already transitioned to
Cython. Thanks much to Sebastian Berg for identifying the problem.
Mike Bayer [Thu, 21 Apr 2022 17:27:16 +0000 (13:27 -0400)]
warn for result.columns() method
A warning is emitted when calling upon the :meth:`.Result.columns` method
with only one index, in particular ORM related cases, indicating that the
current behavior of :meth:`.Result.columns` is broken in this case and
will be changed in 2.0. To receive a collection of scalar values,
use the :meth:`.Result.scalars` method.
Alex Grönholm [Mon, 18 Apr 2022 17:07:19 +0000 (13:07 -0400)]
Implement UUID.python_type
Implemented :attr:`_postgresql.UUID.python_type` attribute for the
:class:`_postgresql.UUID` type object. The attribute will return either
``str`` or ``uuid.UUID`` based on the :paramref:`_postgresql.UUID.as_uuid`
parameter setting. Previously, this attribute was unimplemented. Pull
request courtesy Alex Grönholm.
Mike Bayer [Thu, 14 Apr 2022 16:01:16 +0000 (12:01 -0400)]
Ensure ORMInsert sets up bind state
Fixed regression where the change in #7861, released in version 1.4.33,
that brought the :class:`.Insert` construct to be partially recognized as
an ORM-enabled statement did not properly transfer the correct mapper /
mapped table state to the :class:`.Session`, causing the
:meth:`.Session.get_bind` method to fail for a :class:`.Session` that was
bound to engines and/or connections using the :paramref:`.Session.binds`
parameter.
Mike Bayer [Thu, 14 Apr 2022 13:59:11 +0000 (09:59 -0400)]
update Numeric/Float docstrings
These docs were very out of date re: cdecimal. Additionally,
as pointed out in #5252, the Numeric documentation is misleading;
SQLAlchemy's Numeric hierarchy resembles more of the Oracle
approach where precision and scale solely determine the kind of
datatype being worked with. Float is essentially Numeric with
different defaults.
Mike Bayer [Mon, 4 Apr 2022 23:01:54 +0000 (19:01 -0400)]
read from cls.__dict__ so init_subclass works
Modified the :class:`.DeclarativeMeta` metaclass to pass ``cls.__dict__``
into the declarative scanning process to look for attributes, rather than
the separate dictionary passed to the type's ``__init__()`` method. This
allows user-defined base classes that add attributes within an
``__init_subclass__()`` to work as expected, as ``__init_subclass__()`` can
only affect the ``cls.__dict__`` itself and not the other dictionary. This
is technically a regression from 1.3 where ``__dict__`` was being used.
Additionally makes the reference between ClassManager and
the declarative configuration object a weak reference, so that it
can be discarded after mappers are set up.
Mike Bayer [Tue, 12 Apr 2022 02:03:28 +0000 (22:03 -0400)]
repair ancient and incorrect comment
it referred towards _columntoproperty refering to
lists of MapperProperty. this comment goes all the
way to pre 0.1 being released. it's likely been
wrong for nearly all that time.
Mike Bayer [Sat, 9 Apr 2022 13:50:18 +0000 (09:50 -0400)]
add sane_rowcount to SimpleUpdateDeleteTest
For third party dialects, repaired a missing requirement for the
``SimpleUpdateDeleteTest`` suite test which was not checking for a working
"rowcount" function on the target dialect.
Mike Bayer [Fri, 8 Apr 2022 13:28:47 +0000 (09:28 -0400)]
explicitly refer to Apple M1 for greenlet issue
As developers are now buying lots of Apple M1
machines, and AFAWK greenlet is still not able
to provide a pre-built wheel, we are going to get a lot
of devs trying to use asyncio on their Apple M1s, in
greater proportions compared to devs running containers
etc. on other various less popular CPU architectures.
Add a new FAQ section for installation, add new
red dragon to the very top of asyncio docs,
add new verbiage, all of which includes
"Apple M1" in bold text, to minimize the chance of anyone
missing this.
Mike Bayer [Wed, 6 Apr 2022 13:41:11 +0000 (09:41 -0400)]
maintain complete cloned_set for BindParameter
Fixed regression caused by :ticket:`7823` which impacted the caching
system, such that bound parameters that had been "cloned" within ORM
operations, such as polymorphic loading, would in some cases not acquire
their correct execution-time value leading to incorrect bind values being
rendered.
Mike Bayer [Sun, 3 Apr 2022 15:28:57 +0000 (11:28 -0400)]
TableValuedAlias generation fixes
Fixed bug in newly implemented
:paramref:`.FunctionElement.table_valued.joins_implicitly` feature where
the parameter would not automatically propagate from the original
:class:`.TableValuedAlias` object to the secondary object produced when
calling upon :meth:`.TableValuedAlias.render_derived` or
:meth:`.TableValuedAlias.alias`.
Additionally repaired these issues in :class:`.TableValuedAlias`:
* repaired a potential memory issue which could occur when
repeatedly calling :meth:`.TableValuedAlias.render_derived` against
successive copies of the same object (for .alias(), we currently
have to still continue chaining from the previous element. not sure
if this can be improved but this is standard behavior for .alias()
elsewhere)
* repaired issue where the individual element types would be lost when
calling upon :meth:`.TableValuedAlias.render_derived` or
:meth:`.TableValuedAlias.alias`.
Mike Bayer [Sun, 3 Apr 2022 14:39:19 +0000 (10:39 -0400)]
clarify alternative mapping example
this second example is not part of the doctest steps,
clarify that it's not part of code examples to be present
in execution steps. Add an extra registry +
declarative base on top
so that even if someone does run it, the Base will have
been reset and the examples will continue to work
(noting that column order in statements may change, but
probably nothing else).
Mike Bayer [Thu, 31 Mar 2022 20:53:43 +0000 (16:53 -0400)]
allow executemany values for ON CONFLICT DO NOTHING
Scaled back a fix made for :ticket:`6581` where "executemany values" mode
for psycopg2 were disabled for all "ON CONFLICT" styles of INSERT, to
not apply to the "ON CONFLICT DO NOTHING" clause, which does not include
any parameters and is safe for "executemany values" mode. "ON CONFLICT
DO UPDATE" is still blocked from "executemany values" as there may
be additional parameters in the DO UPDATE clause that cannot be batched
(which is the original issue fixed by :ticket:`6581`).
Mike Bayer [Thu, 31 Mar 2022 18:56:52 +0000 (14:56 -0400)]
add template methods for ORMInsert
Fixed regression caused by :ticket:`7861` where invoking an
:class:`.Insert` construct which contained ORM entities via
:meth:`_orm.Session.execute` would fail.
Mike Bayer [Thu, 31 Mar 2022 13:08:11 +0000 (09:08 -0400)]
add close=False parameter to engine.dispose()
Added new parameter :paramref:`.Engine.dispose.close`, defaulting to True.
When False, the engine disposal does not touch the connections in the old
pool at all, simply dropping the pool and replacing it. This use case is so
that when the original pool is transferred from a parent process, the
parent process may continue to use those connections.
Federico Caselli [Wed, 30 Mar 2022 20:50:18 +0000 (22:50 +0200)]
Update bindparam cache key
The ``literal_execute`` parameter now takes part of the cache
generation of a bindparam, since it changes the sql string generated
by the compiler.
Previously the correct bind values were used, but the ``literal_execute``
would be ignored on subsequent executions of the same query.
Mike Bayer [Tue, 29 Mar 2022 13:48:24 +0000 (09:48 -0400)]
use annotated entity when adding secondary
Fixed regression in "dynamic" loader strategy where the
:meth:`_orm.Query.filter_by` method would not be given an appropriate
entity to filter from, in the case where a "secondary" table were present
in the relationship being queried and the mapping were against something
complex such as a "with polymorphic".
Mike Bayer [Mon, 28 Mar 2022 22:39:19 +0000 (18:39 -0400)]
apply loader criteria more specifically when refresh is true
Fixed bug in :func:`_orm.with_loader_criteria` function where loader
criteria would not be applied to a joined eager load that were invoked
within the scope of a refresh operation for the parent object.
Mike Bayer [Sat, 26 Mar 2022 20:20:34 +0000 (16:20 -0400)]
column_descriptions or equiv for DML, core select
Added new attributes :attr:`.ValuesBase.returning_column_descriptions` and
:attr:`.ValuesBase.entity_description` to allow for inspection of ORM
attributes and entities that are installed as part of an :class:`.Insert`,
:class:`.Update`, or :class:`.Delete` construct. The
:attr:`.Select.column_descriptions` accessor is also now implemented for
Core-only selectables.
Mike Bayer [Mon, 28 Mar 2022 17:46:24 +0000 (13:46 -0400)]
fix quotes regexp for SQLite CHECK constraints
Fixed bug where the name of CHECK constraints under SQLite would not be
reflected if the name were created using quotes, as is the case when the
name uses mixed case or special characters.
Mike Bayer [Mon, 28 Mar 2022 20:07:51 +0000 (16:07 -0400)]
pin click pre-8.1 for black targets
both packages seem to have a bunch of updates,
but as this branch is still formatted for python 2,
keep black at its old version and keep click before
8.1 which seems to be where symbols that are used by
the older black versions were removed
Mike Bayer [Mon, 28 Mar 2022 15:23:39 +0000 (11:23 -0400)]
block asyncmy 0.2.4
asyncmy made a backwards incompatible change to the
TIME datatype in 0.2.4 which hopefully will be reverted
as this change does not belong in a minor point release
Change-Id: Id8bf932d70f52caba06094308a5b87387e69b94b
References: https://github.com/long2ice/asyncmy/issues/37
(cherry picked from commit bbd6420b39edb24481f171b8d0081149dd2e5ba7)
Mike Bayer [Sat, 26 Mar 2022 14:09:10 +0000 (10:09 -0400)]
support BLANK_SCHEMA, RETAIN_SCHEMA FK schema on copy
Added support so that the :paramref:`.Table.tometadata.referred_schema_fn`
callable passed to :meth:`.Table.to_metadata` may return the value
:data:`.BLANK_SCHEMA` to indicate that the referenced foreign key should be
reset to None. The :data.`RETAIN_SCHEMA` symbol may also be returned from
this function to indicate "no change", which will behave the same as
``None`` currently does which also indicates no change.
Mike Bayer [Thu, 24 Mar 2022 17:58:20 +0000 (13:58 -0400)]
more autocommit messaging
Further clarified connection-level logging to indicate the BEGIN, ROLLBACK
and COMMIT log messages do not actually indicate a real transaction when
the AUTOCOMMIT isolation level is in use; messaging has been extended to
include the BEGIN message itself, and the messaging has also been fixed to
accommodate when the :class:`.Engine` level
:paramref:`.create_engine.isolation_level` parameter was used directly.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
