Mike Bayer [Mon, 12 Nov 2018 23:27:34 +0000 (18:27 -0500)]
Allow join() to pick the best candidate from multiple froms/entities
Refactored :meth:`.Query.join` to further clarify the individual components
of structuring the join. This refactor adds the ability for
:meth:`.Query.join` to determine the most appropriate "left" side of the
join when there is more than one element in the FROM list or the query is
against multiple entities. In particular this targets the regression we
saw in :ticket:`4363` but is also of general use. The codepaths within
:meth:`.Query.join` are now easier to follow and the error cases are
decided more specifically at an earlier point in the operation.
Mike Bayer [Wed, 14 Nov 2018 16:02:40 +0000 (11:02 -0500)]
Add secondary selectable to FROM clauses for correlated exists
In continuing with a similar theme as that of very recent :ticket:`4349`,
repaired issue with :meth:`.RelationshipProperty.Comparator.any` and
:meth:`.RelationshipProperty.Comparator.has` where the "secondary"
selectable needs to be explicitly part of the FROM clause in the
EXISTS subquery to suit the case where this "secondary" is a :class:`.Join`
object.
Mike Bayer [Sun, 11 Nov 2018 02:36:18 +0000 (21:36 -0500)]
Add new parameters for IDENTITY start/increment in mssql
Deprecated the use of :class:`.Sequence` with SQL Server in order to affect
the "start" and "increment" of the IDENTITY value, in favor of new
parameters ``mssql_identity_start`` and ``mssql_identity_increment`` which
set these parameters directly. :class:`.Sequence` will be used to generate
real ``CREATE SEQUENCE`` DDL with SQL Server in a future release.
Mike Bayer [Wed, 14 Nov 2018 16:31:22 +0000 (11:31 -0500)]
Deannotate "parententity" in primaryjoin/secondaryjoin
Fixed bug where the ORM annotations could be incorrect for the
primaryjoin/secondaryjoin a relationship if one used the pattern
``ForeignKey(SomeClass.id)`` in the declarative mappings. This pattern
would leak undesired annotations into the join conditions which can break
aliasing operations done within :class:`.Query` that are not supposed to
impact elements in that join condition. These annotations are now removed
up front if present.
Also add a test suite for has/any into test_query which will
form the basis for new tests to be added in :ticket:`4366`.
Mike Bayer [Mon, 12 Nov 2018 20:09:37 +0000 (15:09 -0500)]
Insert primary entity in dynamic "secondary"
Fixed regression caused by :ticket:`4349` where adding the "secondary"
table to the FROM clause for a dynamic loader would affect the ability of
the :class:`.Query` to make a subsequent join to another entity. The fix
adds the primary entity as the first element of the FROM list since
:meth:`.Query.join` wants to jump from that. Version 1.3 will have
a more comprehensive solution to this problem as well (:ticket:`4365`).
Mike Bayer [Sun, 11 Nov 2018 02:10:51 +0000 (21:10 -0500)]
Add Sequence to StrSQLCompiler
Added :class:`.Sequence` to the "string SQL" system that will render a
meaningful string expression (``"<next sequence value: my_sequence>"``)
when stringifying without a dialect a statement that includes a "sequence
nextvalue" expression, rather than raising a compilation error.
Mike Bayer [Wed, 7 Nov 2018 16:21:39 +0000 (11:21 -0500)]
Modernize deferred callable for many-to-one comparison
Improved the behavior of a relationship-bound many-to-one object expression
such that the retrieval of column values on the related object are now
resilient against the object being detached from its parent
:class:`.Session`, even if the attribute has been expired. New features
within the :class:`.InstanceState` are used to memoize the last known value
of a particular column attribute before its expired, so that the expression
can still evaluate when the object is detached and expired at the same
time. Error conditions are also improved using modern attribute state
features to produce more specific messages as needed.
To support the value being mutated while also being resilient towards
expiration, a new feature to InstanceState is added ._last_known_values
which holds onto the expired value when an individual key is expired.
Only takes effect specific to keys and InstanceState objects that
received a special instruction so this does not add to overall
memory/latency.
Mike Bayer [Mon, 5 Nov 2018 19:52:35 +0000 (14:52 -0500)]
Add new "all columns" naming convention tokens
Added new naming convention tokens ``column_0N_name``, ``column_0_N_name``,
etc., which will render the names / keys / labels for all columns referenced
by a particular constraint in a sequence. In order to accommodate for the
length of such a naming convention, the SQL compiler's auto-truncation
feature now applies itself to constraint names as well, which creates a
shortened, deterministically generated name for the constraint that will
apply to a target backend without going over the character limit of that
backend.
Additional notes:
1. the SQLite dialect had a format_index method that was apparently not
used, removed.
2. the naming convention logic has been applying the foreign key
remote column spec to the naming convention, and not the actual
column name. In the case where the referenced Table object uses
.key inside the columns and these are what ForeignKey() references,
the naming convention was doing the wrong thing. The patch here
fixes this, however this isn't noted in the migration notes.
Mike Bayer [Sat, 10 Nov 2018 04:18:55 +0000 (23:18 -0500)]
Use case insensitive matching on lower_case_table_names=1,2
Fixed regression caused by :ticket:`4344` released in 1.2.13, where the fix
for MySQL 8.0's case sensitivity problem with referenced column names when
reflecting foreign key referents is worked around using the
``information_schema.columns`` view. The workaround was failing on OSX /
``lower_case_table_names=2`` which produces non-matching casing for the
``information_schema.columns`` vs. that of ``SHOW CREATE TABLE``, so in
case-insensitive SQL modes case-insensitive matching is now used.
Denis Kataev [Mon, 12 Mar 2018 15:40:34 +0000 (11:40 -0400)]
Implement SQLite ON CONFLICT for constraints
Implemented the SQLite ``ON CONFLICT`` clause as understood at the DDL
level, e.g. for primary key, unique, and CHECK constraints as well as
specified on a :class:`.Column` to satisfy inline primary key and NOT NULL.
Pull request courtesy Denis Kataev.
Mike Bayer [Tue, 6 Nov 2018 20:13:03 +0000 (15:13 -0500)]
Improve documentation re: Session.binds and partitioning strategies
Update documentation to include background on arbitrary superclass
usage, add full cross-linking between all related methods and parameters.
De-emphasize "twophase" and document that it is not well-supported
in drivers.
Mike Bayer [Fri, 2 Nov 2018 20:40:59 +0000 (16:40 -0400)]
Deannotate ORM columns in ColumnEntity
Fixed a minor performance issue which could in some cases add unnecessary
overhead to result fetching, involving the use of ORM columns and entities
that include those same columns at the same time within a query. The issue
has to do with hash / eq overhead when referring to the column in different
ways.
Iwo Herka [Fri, 2 Nov 2018 18:06:25 +0000 (19:06 +0100)]
Tweak code-style and readability in `events/base.py`
This includes a few low-key, syntax-level tweaks and:
1. Rewrite of the if-statment in `Events._accept_with`.
2. Property name change, i.e. from `dispatcher.dispatch_cls` to
`dispatcher.dispatch`. In this case postfix `_cls` is confusing as
the property is not a class, but an instance of one.
Mike Bayer [Thu, 1 Nov 2018 15:11:03 +0000 (11:11 -0400)]
Add new NO_RAISE attribute flag and specify for m2o history load
Added new behavior to the lazy load that takes place when the "old" value of
a many-to-one is retrieved, such that exceptions which would be raised due
to either ``lazy="raise"`` or a detached session error are skipped.
Mike Bayer [Thu, 1 Nov 2018 17:15:14 +0000 (13:15 -0400)]
Implement __delete__
A long-standing oversight in the ORM, the ``__delete__`` method for a many-
to-one relationship was non-functional, e.g. for an operation such as ``del
a.b``. This is now implemented and is equivalent to setting the attribute
to ``None``.
Mike Bayer [Fri, 2 Nov 2018 01:53:18 +0000 (21:53 -0400)]
Use attr keys when testing bulk update params for primary key
Fixed bug in :meth:`.Session.bulk_update_mappings` where alternate mapped
attribute names would result in the primary key column of the UPDATE
statement being included in the SET clause, as well as the WHERE clause;
while usually harmless, for SQL Server this can raise an error due to the
IDENTITY column. This is a continuation of the same bug that was fixed in
:ticket:`.3849`, where testing was insufficient to catch this additional
flaw.
Tom Manderson [Tue, 30 Oct 2018 17:05:43 +0000 (13:05 -0400)]
Move pk on single-inh subclass check below conflict resolution check
The column conflict resolution technique discussed at
:ref:`declarative_column_conflicts` is now functional for a :class:`.Column`
that is also a primary key column. Previously, a check for primary key
columns declared on a single-inheritance subclass would occur before the
column copy were allowed to pass.
Mike Bayer [Tue, 23 Oct 2018 23:38:46 +0000 (19:38 -0400)]
Create object- and column-oriented versions of AssociationProxyInstance
The :class:`.AssociationProxy` now has standard column comparison operations
such as :meth:`.ColumnOperators.like` and
:meth:`.ColumnOperators.startswith` available when the target attribute is a
plain column - the EXISTS expression that joins to the target table is
rendered as usual, but the column expression is then use within the WHERE
criteria of the EXISTS. Note that this alters the behavior of the
``.contains()`` method on the association proxy to make use of
:meth:`.ColumnOperators.contains` when used on a column-based attribute.
Mike Bayer [Wed, 24 Oct 2018 13:09:32 +0000 (09:09 -0400)]
Pin pytest *before* 3.9.1 totally
Unfortunately they have released
3.9.2 before fixing issue 4181, which means we will have to
re-pin a third time after they eventually release with
a fix
Mike Bayer [Sun, 21 Oct 2018 02:20:06 +0000 (22:20 -0400)]
Use the same "current_timestamp" function for both sides of round trip
this test was using sysdate() and current_timestamp() together
in conjunction with a truncation to DAY, however for four hours
on saturday night (see commit time :) ) these two values will
have a different value if one side is EDT and the other is UTC.
tox does not transmit environment variables including TZ by
default, so even if the server is set up for EDT, running tox
will not set TZ and at least Oracle client seems to use this
value, producing UTC for session time but the database on CI
was configured for EDT, producing EDT for sysdate.
Mike Bayer [Fri, 19 Oct 2018 21:10:42 +0000 (17:10 -0400)]
Check more specifically for hybrid attr and not mapped property
Fixed regression caused by :ticket:`4326` in version 1.2.12 where using
:class:`.declared_attr` with a mixin in conjunction with
:func:`.orm.synonym` would fail to map the synonym properly to an inherited
subclass.
Mike Bayer [Wed, 17 Oct 2018 14:42:50 +0000 (10:42 -0400)]
Add prop.secondary to FROM for dynamic loader
Fixed bug where "dynamic" loader needs to explicitly set the "secondary"
table in the FROM clause of the query, to suit the case where the secondary
is a join object that is otherwise not pulled into the query from its
columns alone.
Mike Bayer [Mon, 15 Oct 2018 05:44:13 +0000 (01:44 -0400)]
Don't use "is" to compare exceptions
psycopg2 is introducing a fine grained exception model where
most exceptions will be specialized subclasses, so don't use
"is" to compare an expression type
Mike Bayer [Sun, 14 Oct 2018 19:55:46 +0000 (15:55 -0400)]
Don't call rollback on DBAPI connection that's "closed"
Use the existence of ConnectionRecord.connection to estimate
that this connection is likely closed, and if so, don't
try to call "rollback" on it. This rollback is normally harmless
but is causing segfaults in mysqlclient due to
https://github.com/PyMySQL/mysqlclient-python/issues/270.
Jayson Reis [Mon, 1 Oct 2018 16:58:46 +0000 (12:58 -0400)]
selectinload omit join
The "selectin" loader strategy now omits the JOIN in the case of a
simple one-to-many load, where it instead relies upon the foreign key
columns of the related table in order to match up to primary keys in
the parent table. This optimization can be disabled by setting
the :paramref:`.relationship.omit_join` flag to False.
Many thanks to Jayson Reis for the efforts on this.
As part of this change, horizontal shard no longer relies upon
the _mapper_zero() method to get the query-bound mapper, instead
using the more generalized _bind_mapper() (which will use mapper_zero
if no explicit FROM is present). A short check for the particular
recursive condition is added to BundleEntity and it no longer assigns
itself as the "namespace" to its ColumnEntity objects which creates
a reference cycle.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #4340
Change-Id: I649587e1c07b684ecd63f7d10054cd165891baf4
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/7
Mike Bayer [Wed, 7 Feb 2018 00:30:55 +0000 (19:30 -0500)]
Route bulk update/delete exec through new Query._execute_crud method
Added support for bulk :meth:`.Query.update` and :meth:`.Query.delete`
to the :class:`.ShardedQuery` class within the horiziontal sharding
extension. This also adds an additional expansion hook to the
bulk update/delete methods :meth:`.Query._execute_crud`.
Mike Bayer [Wed, 3 Oct 2018 14:40:38 +0000 (10:40 -0400)]
Support tuples of heterogeneous types for empty expanding IN
Pass a list of all the types for the left side of an
IN expression to the visit_empty_set_expr() method, so that
the "empty expanding IN" can produce clauses for each element.
Mike Bayer [Tue, 2 Oct 2018 21:49:44 +0000 (17:49 -0400)]
Perform additional retrieval of correct column names
Added a workaround for a MySQL bug #88718 introduced in the 8.0 series,
where the reflection of a foreign key constraint is not reporting the
correct case sensitivity for the referred column, leading to errors during
use of the reflected constraint such as when using the automap extension.
The workaround emits an additional query to the information_schema tables in
order to retrieve the correct case sensitive name.
Joe Urciuoli [Wed, 19 Sep 2018 17:40:23 +0000 (13:40 -0400)]
Fix dependency_for final argument
Fixed issue where part of the utility language helper internals was passing
the wrong kind of argument to the Python ``__import__`` builtin as the list
of modules to be imported. The issue produced no symptoms within the core
library but could cause issues with external applications that redefine the
``__import__`` builtin or otherwise instrument it. Pull request courtesy Joe
Urciuoli.
Per the submitter: "The fourth argument provided to `__import__` (which
`import_` feeds in to) is supposed to be a a list of strings, but this code is
passing a single string. This was causing the sqlalchemy `import_` function to
break the string (for example 'interfaces') into an array of single characters
['i', 'n', ...], which causes the actual `__import__` to not find the module
`sqlalchemy.orm.i` (since it's trying to import `sqlalchemy.orm.i` and
`sqlalchemy.orm.n` .. etc)"
No issue could be reproduced locally as it seems you can put anything non-
empty/None into that last argument, even a list like ``['X']``, and all the
sub-modules seem to appear. Omit it, and then the sub-modules aren't present.
Perhaps it just runs the module or not if this attribute is present.
The long-standing behavior of the association proxy collection maintaining
only a weak reference to the parent object is reverted; the proxy will now
maintain a strong reference to the parent for as long as the proxy
collection itself is also in memory, eliminating the "stale association
proxy" error. This change is being made on an experimental basis to see if
any use cases arise where it causes side effects.
Mike Bayer [Mon, 1 Oct 2018 20:23:33 +0000 (16:23 -0400)]
Add reflection support for Postgresql partitioned tables
Added rudimental support for reflection of Postgresql
partitioned tables, e.g. that relkind='p' is added to reflection
queries that return table information.
Mike Bayer [Tue, 27 Feb 2018 18:15:10 +0000 (13:15 -0500)]
Break association proxy into a descriptor + per-class accessor
Reworked :class:`.AssociationProxy` to store state that's specific to a
parent class in a separate object, so that a single
:class:`.AssocationProxy` can serve for multiple parent classes, as is
intrinsic to inheritance, without any ambiguity in the state returned by it.
A new method :meth:`.AssociationProxy.for_class` is added to allow
inspection of class-specific state.
Fixed additional warnings generated by Python 3.7 due to changes in the
organization of the Python ``collections`` and ``collections.abc`` packages.
Previous ``collections`` warnings were fixed in version 1.2.11. Pull request
courtesy xtreak.
See I2d1c0ef97c8ecac7af152cc56263422a40faa6bb for the original collections.abc
fixes.
Removed the collection converter used by the :class:`.MappedCollection`
class. This converter was used only to assert that the incoming dictionary
keys matched that of their corresponding objects, and only during a bulk set
operation. The converter can interfere with a custom validator or
:meth:`.AttributeEvents.bulk_replace` listener that wants to convert
incoming values further. The ``TypeError`` which would be raised by this
converter when an incoming key didn't match the value is removed; incoming
values during a bulk assignment will be keyed to their value-generated key,
and not the key that's explicitly present in the dictionary.
Overall, @converter is superseded by the
:meth:`.AttributeEvents.bulk_replace` event handler added as part of
:ticket:`3896`.
Mike Bayer [Tue, 25 Sep 2018 14:38:40 +0000 (10:38 -0400)]
Copy create_constraint flag for Enum
Fixed bug where the :paramref:`.Enum.create_constraint` flag on the
:class:`.Enum` datatype would not be propagated to copies of the type, which
affects use cases such as declarative mixins and abstract bases.
Mike Bayer [Wed, 13 Jun 2018 21:48:51 +0000 (17:48 -0400)]
Add use_nchar_for_unicode flag; don't use nchar types for generic unicode
The Oracle dialect will no longer use the NCHAR/NCLOB datatypes to
represent generic unicode strings or clob fields in conjunction with
:class:`.Unicode` and :class:`.UnicodeText` unless the flag
``use_nchar_for_unicode=True`` is passed to :func:`.create_engine`.
Additionally, string types under Oracle now coerce to unicode under Python
2 in all cases, however unlike in previous iterations, we use SQLAlchemy's
native unicode handlers which are very performant (when C extensions are
enabled; when they are not, we use cx_Oracle's handlers).
Samuel Chou [Wed, 19 Sep 2018 17:30:24 +0000 (13:30 -0400)]
Allow dialects to customize group by clause compilation
Refactored :class:`.SQLCompiler` to expose a
:meth:`.SQLCompiler.group_by_clause` method similar to the
:meth:`.SQLCompiler.order_by_clause` and :meth:`.SQLCompiler.limit_clause`
methods, which can be overridden by dialects to customize how GROUP BY
renders. Pull request courtesy Samuel Chou.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
