Eli Collins [Thu, 13 Jun 2019 14:37:16 +0000 (10:37 -0400)]
PostgreSQL now reflects per-column sort order on indexes.
Added support for column sorting flags when reflecting indexes for
PostgreSQL, including ASC, DESC, NULLSFIRST, NULLSLAST. Also adds this
facility to the reflection system in general which can be applied to other
dialects in future releases. Pull request courtesy Eli Collins.
Mike Bayer [Sat, 15 Jun 2019 22:06:50 +0000 (18:06 -0400)]
Turn off the is_literal flag when proxying literal_column() to Label
Fixed a series of quoting issues which all stemmed from the concept of the
:func:`.literal_column` construct, which when being "proxied" through a
subquery to be referred towards by a label that matches its text, the label
would not have quoting rules applied to it, even if the string in the
:class:`.Label` were set up as a :class:`.quoted_name` construct. Not
applying quoting to the text of the :class:`.Label` is a bug because this
text is strictly a SQL identifier name and not a SQL expression, and the
string should not have quotes embedded into it already unlike the
:func:`.literal_column` which it may be applied towards. The existing
behavior of a non-labeled :func:`.literal_column` being propagated as is on
the outside of a subquery is maintained in order to help with manual
quoting schemes, although it's not clear if valid SQL can be generated for
such a construct in any case.
Mike Bayer [Sat, 15 Jun 2019 02:44:59 +0000 (22:44 -0400)]
Consult is_attrbute flag to determine descriptor; enable for assoc proxy
Fixed bug where the :attr:`.Mapper.all_orm_descriptors` accessor would
return an entry for the :class:`.Mapper` itself under the declarative
``__mapper___`` key, when this is not a descriptor. The ``.is_attribute``
flag that's present on all :class:`.InspectionAttr` objects is now
consulted, which has also been modified to be ``True`` for an association
proxy, as it was erroneously set to False for this object.
Mike Bayer [Thu, 13 Jun 2019 20:35:12 +0000 (16:35 -0400)]
Generate Oracle ROWNUM scheme using named subqueries
The LIMIT / OFFSET scheme used in Oracle now makes use of named subqueries
rather than unnamed subqueries when it transparently rewrites a SELECT
statement to one that uses a subquery that includes ROWNUM. The change is
part of a larger change where unnamed subqueries are no longer directly
supported by Core, as well as to modernize the internal use of the select()
construct within the Oracle dialect.
Mike Bayer [Thu, 13 Jun 2019 16:45:05 +0000 (12:45 -0400)]
Reverse Alias nesting concept
The Alias object no longer has "element" and "original", it now
has "wrapped" and "element" (the name .original is also left
as a descriptor for legacy access by third party dialects).
These two data members refer to the
dual roles Alias needs to play, where in the Python sense it needs
to refer to the thing it was applied against directly, whereas in the
SQL sense it needs to refer to the ultimate "non-alias" thing it
refers towards. Both are necessary to maintain. However, the change
here has each Alias object access the non-Alias object immediately
so that the "unwrapping" is simpler and does not need any special
logic.
In the SQL sense, Alias objects don't nest, the only potential
was that of the CTE, however there is no such thing as
a nested CTE, see link below.
This change is an interim change along the way to breaking Alias
into more classes and breaking away Select objects from being
FromClause objects.
Mike Bayer [Wed, 12 Jun 2019 17:15:59 +0000 (13:15 -0400)]
Run PK/FK sync for multi-level inheritance w/ no intermediary update
Also fix DetectKeySwitch for intermediary class relationship
Fixed a series of related bugs regarding joined table inheritance more than
two levels deep, in conjunction with modification to primary key values,
where those primary key columns are also linked together in a foreign key
relationship as is typical for joined table inheritance. The intermediary
table in a three-level inheritance hierachy will now get its UPDATE if
only the primary key value has changed and passive_updates=False (e.g.
foreign key constraints not being enforced), whereas before it would be
skipped; similarly, with passive_updates=True (e.g. ON UPDATE CASCADE in
effect), the third-level table will not receive an UPDATE statement as was
the case earlier which would fail since CASCADE already modified it. In a
related issue, a relationship linked to a three-level inheritance hierarchy
on the primary key of an intermediary table of a joined-inheritance
hierarchy will also correctly have its foreign key column updated when the
parent object's primary key is modified, even if that parent object is a
subclass of the linked parent class, whereas before these classes would
not be counted.
Mike Bayer [Fri, 7 Jun 2019 15:19:23 +0000 (11:19 -0400)]
Don't discard inactive transaction until it is explicitly rolled back
The :class:`.Connection` object will now not clear a rolled-back
transaction until the outermost transaction is explicitly rolled back.
This is essentially the same behavior that the ORM :class:`.Session` has
had for a long time, where an explicit call to ``.rollback()`` on all
enclosing transactions is required for the transaction to logically clear,
even though the DBAPI-level transaction has already been rolled back.
The new behavior helps with situations such as the "ORM rollback test suite"
pattern where the test suite rolls the transaction back within the ORM
scope, but the test harness which seeks to control the scope of the
transaction externally does not expect a new transaction to start
implicitly.
Mike Bayer [Sun, 9 Jun 2019 14:59:23 +0000 (10:59 -0400)]
psycopg2 NOTICE fixup
- don't call relatively expensive isEnabledFor(), just call _log_notices
- don't reset the list if it's empty
- fix the test to use a custom function to definitely create a notice, confirmed
that PG seems to no longer create the "implicit sequence" notices
- assert that the reset of the notices works too
- update the docs to illustrate for folks who haven't worked with logging before
Mike Bayer [Fri, 7 Jun 2019 20:04:22 +0000 (16:04 -0400)]
Add "full" to the getargspec warning skip
Even though getfullargspec() is no longer deprecated in python3.8
as of b1, we aren't using it anymore so block any future warnings
from interfering with py.test or similar.
Mike Bayer [Fri, 7 Jun 2019 18:50:22 +0000 (14:50 -0400)]
Rework Session transaction FAQs
In preparation for #4712, add an errors.rst code to the Session's
exception about waiting to be rolled back and rework the FAQ entry
to be much more succinct. When this FAQ was first written, I found
it hard to describe why flush worked this way but as the use case is
clearer now, and #4712 actually showed it being confusing when it doesn't
work this way, we can make a simpler and more definitive statement
about this behavior. Additionally, language about "subtransactions"
is minimized as I might be removing or de-emphasizing this concept in
2.0 (though maybe not as it does seem to work well).
Mike Bayer [Mon, 3 Jun 2019 16:48:14 +0000 (12:48 -0400)]
Add "usecase" changelog tag
"usecase" indicates the library now supports something a user
was trying to do. It's not quite a "feature" since it's something
that seems like it should have worked, it's not a "bug" because
no mistake was made, it's just something that wasn't considered before.
The advantage of "usecase" is that it inherently suggests a different
style of prioritization vs. something that is preventing the library
from working as designed.
This change also adds docs/build/conf.py under the pep8 formatting
test coverage.
Mike Bayer [Fri, 31 May 2019 20:47:19 +0000 (16:47 -0400)]
Apply adaptation for most recent aliased=True first
Fixed regression in :meth:`.Query.join` where the ``aliased=True`` flag
would not properly apply clause adaptation to filter criteria, if a
previous join were made to the same entity. This is because the adapters
were placed in the wrong order. The order has been reversed so that the
adapter for the most recent ``aliased=True`` call takes precedence as was
the case in 1.2 and earlier. This broke the "elementtree" examples among
other things.
Mike Bayer [Fri, 31 May 2019 00:42:35 +0000 (20:42 -0400)]
PostgreSQL enum with no elements returns NULL for the "label", skip this
Fixed bug where PostgreSQL dialect could not correctly reflect an ENUM
datatype that has no members, returning a list with ``None`` for the
``get_enums()`` call and raising a TypeError when reflecting a column which
has such a datatype. The inspection now returns an empty list.
Mike Bayer [Wed, 29 May 2019 21:27:19 +0000 (17:27 -0400)]
Use fully vendored getfullargspec
Replaced the Python compatbility routines for ``getfullargspec()`` with a
fully vendored version from Python 3.3. Originally, Python was emitting
deprecation warnings for this function in Python 3.8 alphas. While this
change was reverted, it was observed that Python 3 implementations for
``getfullargspec()`` are an order of magnitude slower as of the 3.4 series
where it was rewritten against ``Signature``. While Python plans to
improve upon this situation, SQLAlchemy projects for now are using a simple
replacement to avoid any future issues.
Dmytro Starosud [Wed, 29 May 2019 13:47:13 +0000 (16:47 +0300)]
Rework AliasedClass __getattr__ to use top-level getattr()
Reworked the attribute mechanics used by :class:`.AliasedClass` to no
longer rely upon calling ``__getattribute__`` on the MRO of the wrapped
class, and to instead resolve the attribute normally on the wrapped class
using getattr(), and then unwrap/adapt that. This allows a greater range
of attribute styles on the mapped class including special ``__getattr__()``
schemes; but it also makes the code simpler and more resilient in general.
Fixes: #4694 Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: I28901e2472d3c21e881fe5cafa3b1d3af704fad8
Mike Bayer [Tue, 28 May 2019 13:46:20 +0000 (09:46 -0400)]
Reformat license name
While we have the OSI classifier for "MIT License", it looks
like for the "license" field, this is normally just the word
"MIT" and not "MIT License". While the pypa docs suggest we
only need it as the OSI classifier, keep it also in "license"
in order to appease common tooling.
Mike Bayer [Sun, 19 May 2019 00:35:01 +0000 (20:35 -0400)]
Use roles for ORM alias() conversion
as SELECT statements will have subquery() and not alias(),
start getting ready for the places where the ORM coerces SELECTs
into subqueries and be ready to warn about it
Mike Bayer [Tue, 28 May 2019 01:15:47 +0000 (21:15 -0400)]
Adjust test_concurrency failure modes
The test added for #4686 can raise for "B" missing which
is normal and should not fail the test. Also ensure mappers are
cleared to prevent subsequent tests elsewhere from being
affected.
Hannes Hansen [Thu, 23 May 2019 20:27:21 +0000 (16:27 -0400)]
MYSQL: added support for drop check/constraint
Added support for DROP CHECK constraint which is required by MySQL 8.0.16
to drop a CHECK constraint; MariaDB supports plain DROP CONSTRAINT. The
logic distinguishes between the two syntaxes by checking the server version
string for MariaDB presence. Alembic migrations has already worked
around this issue by implementing its own DROP for MySQL / MariaDB CHECK
constraints, however this change implements it straight in Core so that its
available for general use. Pull request courtesy Hannes Hansen.
Mike Bayer [Sat, 25 May 2019 22:04:58 +0000 (18:04 -0400)]
Hold implicitly created collections in a pending area
Accessing a collection-oriented attribute on a newly created object no
longer mutates ``__dict__``, but still returns an empty collection as has
always been the case. This allows collection-oriented attributes to work
consistently in comparison to scalar attributes which return ``None``, but
also don't mutate ``__dict__``. In order to accommodate for the collection
being mutated, the same empty collection is returned each time once
initially created, and when it is mutated (e.g. an item appended, added,
etc.) it is then moved into ``__dict__``. This removes the last of
mutating side-effects on read-only attribute access within the ORM.
Mike Bayer [Thu, 23 May 2019 22:01:07 +0000 (18:01 -0400)]
Unify NO_VALUE and NEVER_SET
There's no real difference between these two constants
except they are used in different places and therefore allow
various codepaths to work largely by accident. These
codepaths should be explicit. Assign NO_VALUE and NEVER_SET
to the same constant and work towards having just one constant
for "we have no value to return right now".
Mike Bayer [Thu, 23 May 2019 15:35:51 +0000 (11:35 -0400)]
Include active_history when propagating attribute listeners
Fixed issue where the :paramref:`.AttributeEvents.active_history` flag
would not be set for an event listener that propgated to a subclass via the
:paramref:`.AttributeEvents.propagate` flag. This bug has been present
for the full span of the :class:`.AttributeEvents` system.
Mike Bayer [Sun, 19 May 2019 16:38:14 +0000 (12:38 -0400)]
Add QueryableAttribute._impl_uses_objects accessor for AssociationProxy
Fixed regression where new association proxy system was still not proxying
hybrid attributes when they made use of the ``@hybrid_property.expression``
decorator to return an alternate SQL expression, or when the hybrid
returned an arbitrary :class:`.PropComparator`, at the expression level.
This involved futher generalization of the heuristics used to detect the
type of object being proxied at the level of :class:`.QueryableAttribute`,
to better detect if the descriptor ultimately serves mapped classes or
column expressions.
Mike Bayer [Tue, 30 Apr 2019 03:26:36 +0000 (23:26 -0400)]
Implement new ClauseElement role and coercion system
A major refactoring of all the functions handle all detection of
Core argument types as well as perform coercions into a new class hierarchy
based on "roles", each of which identify a syntactical location within a
SQL statement. In contrast to the ClauseElement hierarchy that identifies
"what" each object is syntactically, the SQLRole hierarchy identifies
the "where does it go" of each object syntactically. From this we define
a consistent type checking and coercion system that establishes well
defined behviors.
This is a breakout of the patch that is reorganizing select()
constructs to no longer be in the FromClause hierarchy.
Also includes a rename of as_scalar() into scalar_subquery(); deprecates
automatic coercion to scalar_subquery().
Mike Bayer [Thu, 16 May 2019 15:26:04 +0000 (11:26 -0400)]
Mutex the declarative scan/map process against configure_mappers()
Applied the mapper "configure mutex" against the declarative class mapping
process, to guard against the race which can occur if mappers are used
while dynamic module import schemes are still in the process of configuring
mappers for related classes. This does not guard against all possible race
conditions, such as if the concurrent import has not yet encountered the
dependent classes as of yet, however it guards against as much as possible
within the SQLAlchemy declarative process.
Mike Bayer [Mon, 13 May 2019 18:56:12 +0000 (14:56 -0400)]
Continue to assume None for un-accessed attribute on persistent
object during m2o fetch
Fixed regression in new relationship m2o comparison logic first introduced
at :ref:`change_4359` when comparing to an attribute that is persisted as
NULL and is in an un-fetched state in the mapped instance. Since the
attribute has no explicit default, it needs to default to NULL when
accessed in a persistent setting.
Mike Bayer [Sat, 11 May 2019 02:36:40 +0000 (22:36 -0400)]
Correct fix and tests for #4661
For #4661 we need to still warn if we are only deleting one row,
even if sane multi rowcount is false. Tests were failing for
pyodbc since the warning was removed for the single-row case.
the UPDATE logic raises if a single row doesn't match even
if sane multi rowcount is false, so this is now more consistent
with that. Add tests for the UPDATE case also. It is possible
there are already tests for this but as the DELETE case wasn't
well covered it's not clear.
Matthew Wilkes [Thu, 9 May 2019 22:04:35 +0000 (18:04 -0400)]
Move initialize do_rollback() outside of the dialect
Moved the "rollback" which occurs during dialect initialization so that it
occurs after additional dialect-specific initialize steps, in particular
those of the psycopg2 dialect which would inadvertently leave transactional
state on the first new connection, which could interfere with some
psycopg2-specific APIs which require that no transaction is started. Pull
request courtesy Matthew Wilkes.
Mike Bayer [Tue, 7 May 2019 15:38:48 +0000 (11:38 -0400)]
Turn FlushError for identity already exists into a warning.
The condition where a pending object being flushed with an identity that
already exists in the identity map has been adjusted to emit a warning,
rather than throw a :class:`.FlushError`. The rationale is so that the
flush will proceed and raise a :class:`.IntegrityError` instead, in the
same way as if the existing object were not present in the identity map
already. This helps with schemes that are uinsg the
:class:`.IntegrityError` as a means of catching whether or not a row
already exists in the table.
mollardthomas [Fri, 3 May 2019 15:31:57 +0000 (11:31 -0400)]
Add support for filtered indexes for mssql dialect
Added support for SQL Server filtered indexes, via the ``mssql_where``
parameter which works similarly to that of the ``postgresql_where`` index
function in the PostgreSQL dialect.
Adrien Berchet [Fri, 3 May 2019 16:02:17 +0000 (12:02 -0400)]
Do not register the GenericFunction in sql.functions._registry
Fixed that the :class:`.GenericFunction` class was inadvertently
registering itself as one of the named functions. Pull request courtesy
Adrien Berchet.
Mike Bayer [Fri, 3 May 2019 22:07:06 +0000 (18:07 -0400)]
Don't warn on multi delete rowcount if supports_sane_multi is False
Fixed an issue where the "number of rows matched" warning would emit even if
the dialect reported "supports_sane_multi_rowcount=False", as is the case
for psycogp2 with ``use_batch_mode=True`` and others.
Mike Bayer [Mon, 29 Apr 2019 21:31:12 +0000 (17:31 -0400)]
Make the GenericFunction registry fully case insensitive
Registered function names based on :class:`.GenericFunction` are now
retrieved in a case-insensitive fashion in all cases, removing the
deprecation logic from 1.3 which temporarily allowed multiple
:class:`.GenericFunction` objects to exist with differing cases. A
:class:`.GenericFunction` that replaces another on the same name whether or
not it's case sensitive emits a warning before replacing the object.
Add case insensitivity feature to GenericFunction.
The :class:`.GenericFunction` namespace is being migrated so that function
names are looked up in a case-insensitive manner, as SQL functions do not
collide on case sensitive differences nor is this something which would
occur with user-defined functions or stored procedures. Lookups for
functions declared with :class:`.GenericFunction` now use a case
insensitive scheme, however a deprecation case is supported which allows
two or more :class:`.GenericFunction` objects with the same name of
different cases to exist, which will cause case sensitive lookups to occur
for that particular name, while emitting a warning at function registration
time. Thanks to Adrien Berchet for a lot of work on this complicated
feature.
Mike Bayer [Wed, 17 Apr 2019 17:37:39 +0000 (13:37 -0400)]
Reimplement .compare() in terms of a visitor
Reworked the :meth:`.ClauseElement.compare` methods in terms of a new
visitor-based approach, and additionally added test coverage ensuring that
all :class:`.ClauseElement` subclasses can be accurately compared
against each other in terms of structure. Structural comparison
capability is used to a small degree within the ORM currently, however
it also may form the basis for new caching features.
Mike Bayer [Sun, 28 Apr 2019 16:40:31 +0000 (12:40 -0400)]
Warn on merge of already-pending object
A warning is now emitted for the case where a transient object is being
merged into the session with :meth:`.Session.merge` when that object is
already transient in the :class:`.Session`. This warns for the case where
the object would normally be double-inserted.
Mike Bayer [Thu, 25 Apr 2019 15:46:31 +0000 (10:46 -0500)]
Add ORM documentation for as_comparison()
In #3831 we added the ability for SQL functions to be used in
primaryjoin conditions as the source of comparison, however we
didn't add documentation from the main relationship docs so
nobody could find it unless they read the migration notes.
Since the two use cases that have come up for this are
materialized path with string functions, and geometry functions,
add the example based on the use case requested in
https://github.com/geoalchemy/geoalchemy2/issues/220#issuecomment-486709055
This example hasn't been tested yet and is subject to
revision.
Mike Bayer [Tue, 23 Apr 2019 22:33:08 +0000 (17:33 -0500)]
Add additional parsing to extract the "MariaDB" keyword from version string
Enhanced MySQL/MariaDB version string parsing to accommodate for exotic
MariaDB version strings where the "MariaDB" word is embedded among other
alphanumeric characters such as "MariaDBV1". This detection is critical in
order to correctly accomodate for API features that have split between MySQL
and MariaDB such as the "transaction_isolation" system variable.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
