Frazer McLean [Sat, 11 Jun 2016 19:47:33 +0000 (21:47 +0200)]
Implement comments for tables, columns
Added support for SQL comments on :class:`.Table` and :class:`.Column`
objects, via the new :paramref:`.Table.comment` and
:paramref:`.Column.comment` arguments. The comments are included
as part of DDL on table creation, either inline or via an appropriate
ALTER statement, and are also reflected back within table reflection,
as well as via the :class:`.Inspector`. Supported backends currently
include MySQL, Postgresql, and Oracle.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #1546
Change-Id: Ib90683850805a2b4ee198e420dc294f32f15d35d
Mike Bayer [Wed, 25 Jan 2017 16:51:04 +0000 (11:51 -0500)]
Add bulk_replace event, integrate with @validates
Added new attribute event :meth:`.AttributeEvents.bulk_replace`.
This event is triggered when a collection is assigned to a
relationship, before the incoming collection is compared with the
existing one. This early event allows for conversion of incoming
non-ORM objects as well. The event is integrated with the
``@validates`` decorator.
The ``@validates`` decorator now allows the decorated method to receive
objects from a "bulk collection set" operation that have not yet
been compared to the existing collection. This allows incoming values
to be converted to compatible ORM objects as is already allowed
from an "append" event. Note that this means that the
``@validates`` method is called for **all** values during a collection
assignment, rather than just the ones that are new.
Mike Bayer [Mon, 16 Jan 2017 18:34:55 +0000 (13:34 -0500)]
Annotate parentmapper in primaryjoin / secondaryjoin
This patch applies the "parentmapper" annotation to the columns
in the primaryjoin/secondaryjoin, but more dramatically,
also removes all the "deannotate" steps that were historically
applied to the relationship primaryjoin/secondaryjoin.
These deannotation steps were left over from the initial
implementations of annotations where the behaviors were not
as reliable.
By ensuring these annotations are present,
the evaluator no longer needs to do a name-based lookup
when it sees a column that has no "parentmapper",
because it can be assured this is not a mapped column.
This fixes the issue where the expression were based on
a relationship primaryjoin but the name of a column
in the join condition didn't match the attribute name.
Mike Bayer [Tue, 14 Feb 2017 16:39:44 +0000 (11:39 -0500)]
Don't mutate old collection on bulk replace
For a bulk replace, assume the old collection is no longer
useful to the attribute system and only send the removal events,
not actually mutated the collection.
this changes behavior significantly and also means that dispose_collection
now receives the old collection intact.
Mike Bayer [Wed, 15 Mar 2017 14:15:12 +0000 (10:15 -0400)]
Consult compiled paramstyle on execute_compiled
Fixed bug where in the unusual case of passing a
:class:`.Compiled` object directly to :meth:`.Connection.execute`,
the dialect with which the :class:`.Compiled` object were generated
was not consulted for the paramstyle of the string statement, instead
assuming it would match the dialect-level paramstyle, causing
mismatches to occur.
Mike Bayer [Mon, 16 Jan 2017 18:06:44 +0000 (13:06 -0500)]
Consult _select_from_entity in _adjust_for_single_inheritance
Fixed bug in single-table inheritance where the select_from()
argument would not be taken into account when limiting rows
to a subclass. Previously, only expressions in the
columns requested would be taken into account.
Mike Bayer [Tue, 14 Mar 2017 16:00:56 +0000 (12:00 -0400)]
Add "empty in" strategies; default to "static"
The longstanding behavior of the :meth:`.Operators.in_` and
:meth:`.Operators.not_in_` operators emitting a warning when
the right-hand condition is an empty sequence has been revised;
a new flag :paramref:`.create_engine.empty_in_strategy` allows an
empty "IN" expression to generate a simple boolean expression, or
to invoke the previous behavior of dis-equating the expression to
itself, with or without a warning. The default behavior is now
to emit the simple boolean expression, allowing an empty IN to
be evaulated without any performance penalty.
Mike Bayer [Thu, 9 Mar 2017 16:36:19 +0000 (11:36 -0500)]
Improve serializer behavior
Fix an issue where the Annotated system needs to have a
__reduce__ method, also see why we can't default to HIGHEST_PROTOCOL.
This latter part might not be a good idea until 1.2 for compatibility
reasons.
Mike Bayer [Sat, 11 Mar 2017 15:52:43 +0000 (10:52 -0500)]
Emit after_rollback() event before snapshot removal
The state of the :class:`.Session` is now present when the
:meth:`.SessionEvents.after_rollback` event is emitted, that is, the
attribute state of objects prior to their being expired. This is now
consistent with the behavior of the
:meth:`.SessionEvents.after_commit` event which also emits before the
attribute state of objects is expired.
Mike Bayer [Mon, 13 Mar 2017 15:20:53 +0000 (11:20 -0400)]
Normalize Oracle reflected FK constraint name
Oracle reflection now "normalizes" the name given to a foreign key
constraint, that is, returns it as all lower case for a case
insensitive name. This was already the behavior for indexes
and primary key constraints as well as all table and column names.
This will allow Alembic autogenerate scripts to compare and render
foreign key constraint names correctly when initially specified
as case insensitive.
Mike Bayer [Mon, 13 Mar 2017 16:27:51 +0000 (12:27 -0400)]
Repair _execute_scalar for WITH_UNICODE mode
cx_Oracle 5.3 seems to code this flag ON now, so
remove the warning and ensure WITH_UNICODE handling works.
Additionally, the XE setup on jenkins is having more
problems here, in particular low-connections mode is
causing cx_Oracle to fail more frequently now. Turning
off low-connections fixes those but then we get the
TNS errors, so adding an emergency "retry" flag that
is not yet a feature available to users. Real world
applications are not dropping/creating thousands of
tables the way our test suite is.
Mike Bayer [Tue, 7 Mar 2017 17:53:00 +0000 (12:53 -0500)]
Allow SchemaType and Variant to work together
Added support for the :class:`.Variant` and the :class:`.SchemaType`
objects to be compatible with each other. That is, a variant
can be created against a type like :class:`.Enum`, and the instructions
to create constraints and/or database-specific type objects will
propagate correctly as per the variant's dialect mapping.
Also added testing for some potential double-event scenarios
on TypeDecorator but it seems usually this doesn't occur.
Mike Bayer [Mon, 6 Mar 2017 17:26:01 +0000 (12:26 -0500)]
Don't cache savepoint identifiers
Fixed bug in compiler where the string identifier of a savepoint would
be cached in the identifier quoting dictionary; as these identifiers
are arbitrary, a small memory leak could occur if a single
:class:`.Connection` had an unbounded number of savepoints used,
as well as if the savepoint clause constructs were used directly
with an unbounded umber of savepoint names. The memory leak does
**not** impact the vast majority of cases as normally the
:class:`.Connection`, which renders savepoint names with a simple
counter starting at "1", is used on a per-transaction or
per-fixed-number-of-transactions basis before being discarded.
The savepoint name in virtually all cases does not require quoting
at all, however to support potential third party use cases
the "check for quotes needed" logic is retained, at a small
performance cost. Uncondtionally quoting the name is another
option, but this would turn the name into a case sensitive name
which runs the risk of poor interactions with existing deployments
that may be looking at these names in other contexts.
Mike Bayer [Tue, 28 Feb 2017 19:03:47 +0000 (14:03 -0500)]
Only use schema_translate_map on SchemaItem subclasses
Fixed bug in new "schema translate" feature where the translated schema
name would be invoked in terms of an alias name when rendered along
with a column expression; occurred only when the source translate
name was "None". The "schema translate" feature now only takes
effect for :class:`.SchemaItem` and :class:`.SchemaType` subclasses,
that is, objects that correspond to a DDL-creatable structure in
a database.
Mike Bayer [Thu, 23 Feb 2017 19:40:00 +0000 (14:40 -0500)]
Performance within instances()
Continuing from Ie43beecf37945b2bb7fff0aaa597a597293daa18,
also observed is the overhead of PathRegsitry memoized token
functions, as these paths are not cached in the case of
long joinedloader chains. The memoizations here were made
with short paths in mind, and have been replaced with
an inlined straight create of these paths up front, producing
callcounts very similar to 0.8. Combined with the previous
optimizations, 1.1 now runs the "joined eager load of one row"
worst case test in about 40% fewer calls than 0.8 and 60%
fewer than 1.1.5.
Mike Bayer [Thu, 16 Feb 2017 16:31:26 +0000 (11:31 -0500)]
Memoize AliasedClass, ClauseAdapter objects in joined eager load
Addressed some long unattended performance concerns within the
joined eager loader query construction system. The use of ad-hoc
:class:`.AliasedClass` objects per query, which produces lots of column
lookup overhead each time, has been replaced with a cached approach
that makes use of a small pool of :class:`.AliasedClass` objects
that are reused between invocations of joined eager loading.
Callcount reduction of SQL query generation for worst-case joined
loader scenarios (lots of joins, lots of columns) is reduced by
approximately 270%.
Mike Bayer [Mon, 13 Feb 2017 19:20:38 +0000 (14:20 -0500)]
Add new DDL autocommit expressions for Postgresql
Added regular expressions for the "IMPORT FOREIGN SCHEMA",
"REFRESH MATERIALIZED VIEW" Postgresql statements so that they
autocommit when invoked via a connection or engine without
an explicit transaction. Pull requests courtesy Frazer McLean
and Paweł Stiasny.
Mike Bayer [Thu, 9 Feb 2017 02:05:16 +0000 (21:05 -0500)]
Check for columns not part of mapping, correct mapping for eager_defaults
Fixed two closely related bugs involving the mapper eager_defaults
flag in conjunction with single-table inheritance; one where the
eager defaults logic would inadvertently try to access a column
that's part of the mapper's "exclude_properties" list (used by
Declarative with single table inheritance) during the eager defaults
fetch, and the other where the full load of the row in order to
fetch the defaults would fail to use the correct inheriting mapper.
Mike Bayer [Thu, 9 Feb 2017 02:42:34 +0000 (21:42 -0500)]
Don't post-SELECT columns w/o a server default/onupdate for eager_defaults
Fixed a major inefficiency in the "eager_defaults" feature whereby
an unnecessary SELECT would be emitted for column values where the
ORM had explicitly inserted NULL, corresponding to attributes that
were unset on the object but did not have any server default
specified, as well as expired attributes on update that nevertheless
had no server onupdate set up. As these columns are not part of the
RETURNING that eager_defaults tries to use, they should not be
post-SELECTed either.
Mike Bayer [Wed, 1 Feb 2017 21:25:31 +0000 (16:25 -0500)]
Accept FetchedValue, text() for column "default" value
Fixed bug whereby the :meth:`.DDLEvents.column_reflect` event would not
allow a non-textual expression to be passed as the value of the
"default" for the new column, such as a :class:`.FetchedValue`
object to indicate a generic triggered default or a
:func:`.sql.expression.text` construct. Clarified the documentation
in this regard as well.
Mike Bayer [Thu, 26 Jan 2017 21:11:49 +0000 (16:11 -0500)]
Union the exclude_properties of the inheriting mapper in declarative
Fixed bug where the "automatic exclude" feature of declarative that
ensures a column local to a single table inheritance subclass does
not appear as an attribute on other derivations of the base would
not take effect for multiple levels of subclassing from the base.
Mike Bayer [Mon, 30 Jan 2017 17:10:16 +0000 (12:10 -0500)]
Copy whereclause / using in ExcludeConstraint
Fixed bug in Postgresql :class:`.ExcludeConstraint` where the
"whereclause" and "using" parameters would not be copied during an
operation like :meth:`.Table.tometadata`.
Mike Bayer [Sat, 28 Jan 2017 14:43:40 +0000 (09:43 -0500)]
- cte() has moved to HasCTE(), since SelectBase is referred to
a lot just add inherited members so links to things like SelectBase.cte
continue to function
Mike Bayer [Thu, 26 Jan 2017 21:54:02 +0000 (16:54 -0500)]
Don't check isolation level prior to SQL Server 2005
Added a version check to the "get_isolation_level" feature, which is
invoked upon first connect, so that it skips for SQL Server version
2000, as the necessary system view is not available prior to SQL Server
2005.
Mike Bayer [Thu, 26 Jan 2017 21:01:20 +0000 (16:01 -0500)]
- document that "column" and "where" are arbitrary SQL expressions
for ExcludeConstraint, if string is used then quoting must
be applied manually. fixes #3899
Mike Bayer [Thu, 19 Jan 2017 19:12:19 +0000 (14:12 -0500)]
Improve server-side Sequence documentation
Include the metadata argument for the Sequence
and explain the rationale. Correct inconsistencies
between Core / ORM examples and update language
regarding client side vs. server side Sequence
directive.
Mike Bayer [Thu, 19 Jan 2017 16:41:09 +0000 (11:41 -0500)]
Dont set _set_select_from() for alias object
Fixed bug first introduced in 0.9.7 as a result of :ticket:`3106`
which would cause an incorrect query in some forms of multi-level
subqueryload against aliased entities, with an unnecessary extra
FROM entity in the innermost subquery.
Lele Long [Sat, 3 Dec 2016 18:10:07 +0000 (13:10 -0500)]
Parse (but don't record) COMMENT portion of MySQL table key
The MySQL dialect now will not warn when a reflected column has a
"COMMENT" keyword on it, but note however the comment is not yet
reflected; this is on the roadmap for a future release. Pull request
courtesy Lele Long.
Joseph Schorr [Tue, 17 Jan 2017 15:02:17 +0000 (10:02 -0500)]
Add support for prefixes on CREATE INDEX statements in MySQL
Added a new parameter ``mysql_prefix`` supported by the :class:`.Index`
construct, allows specification of MySQL-specific prefixes such as
"FULLTEXT". Pull request courtesy Joseph Schorr.
Valery Yundin [Fri, 16 Dec 2016 14:22:08 +0000 (09:22 -0500)]
Better hide engine password
Avoid putting engine password in the exception message in
`MetaData.reflect` (since exception messages often appear in logs).
Use the same redacted `__repr__` implementation in
`TLEngine` as in its base class `Engine`
Mike Bayer [Mon, 16 Jan 2017 17:10:08 +0000 (12:10 -0500)]
Add "existing" populators for subqueryload
Fixed bug in subquery loading where an object encountered as an
"existing" row, e.g. already loaded from a different path in the
same query, would not invoke subquery loaders for unloaded attributes
that specified this loading. This issue is in the same area
as that of :ticket:`3431`, :ticket:`3811` which involved
similar issues with joined loading.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
