Sanjana [Thu, 28 Feb 2019 15:16:13 +0000 (10:16 -0500)]
Add port comparison in __eq__() and __ne__() method to URL
Comparing two objects of :class:`.URL` using ``__eq__()`` did not take port
number into consideration, two objects differing only by port number were
considered equal. Port comparison is now added in ``__eq__()`` method of
:class:`.URL`, objects differing by port number are now not equal.
Additionally, ``__ne__()`` was not implemented for :class:`.URL` which
caused unexpected result when ``!=`` was used in Python2, since there are no
implied relationships among the comparison operators in Python2.
Mike Bayer [Thu, 28 Feb 2019 15:59:41 +0000 (10:59 -0500)]
Add resolve_fks=False option for reflection
Added new parameters :paramref:`.Table.resolve_fks` and
:paramref:`.MetaData.reflect.resolve_fks` which when set to False will
disable the automatic reflection of related tables encountered in
:class:`.ForeignKey` objects, which can both reduce SQL overhead for omitted
tables as well as avoid tables that can't be reflected for database-specific
reasons. Two :class:`.Table` objects present in the same :class:`.MetaData`
collection can still refer to each other even if the reflection of the two
tables occurred separately.
Toni Karppi [Thu, 21 Feb 2019 19:22:48 +0000 (14:22 -0500)]
Add test to as_interface when no cls or methods
Fixes #4511
### Description
The `as_interface` function requires that an argument be provided for at least one of the parameters `cls` or `methods`. There was no test that tested the case when neither of these arguments were provided.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Wed, 20 Feb 2019 20:00:09 +0000 (15:00 -0500)]
Prevent __init__ from being called for Alias, subclasses
The :class:`.Alias` class and related subclasses :class:`.CTE`,
:class:`.Lateral` and :class:`.TableSample` have been reworked so that it is
not possible for a user to construct the objects directly. These constructs
require that the standalone construction function or selectable-bound method
be used to instantiate new objects.
Nate Clark [Wed, 20 Feb 2019 17:58:18 +0000 (12:58 -0500)]
Include newlines in StatementError formatting
Revised the formatting for :class:`.StatementError` when stringified. Each
error detail is broken up over multiple newlines instead of spaced out on a
single line. Additionally, the SQL representation now stringifies the SQL
statement rather than using ``repr()``, so that newlines are rendered as is.
Pull request courtesy Nate Clark.
sanjana [Wed, 20 Feb 2019 04:07:12 +0000 (23:07 -0500)]
Add support for key-word based get()
The :meth:`.Query.get` method can now accept a dictionary of attribute keys
and values as a means of indicating the primary key value to load; is
particularly useful for composite primary keys. Pull request courtesy
Sanjana S.
Mike Bayer [Wed, 20 Feb 2019 00:46:17 +0000 (19:46 -0500)]
Ensure _simple_lazy_clause bind names are fixed before cloning
Fixed a regression in 1.2 due to the introduction of baked queries for
relationship lazy loaders, where a race condition is created during the
generation of the "lazy clause" which occurs within a memoized attribute. If
two threads initialize the memoized attribute concurrently, the baked query
could be generated with bind parameter keys that are then replaced with new
keys by the next run, leading to a lazy load query that specifies the
related criteria as ``None``. The fix establishes that the parameter names
are fixed before the new clause and parameter objects are generated, so that
the names are the same every time.
Mike Bayer [Tue, 19 Feb 2019 17:13:04 +0000 (12:13 -0500)]
Add NCHAR to Oracle dialect
Added support for reflection of the :class:`.NCHAR` datatype to the Oracle
dialect, and added :class:`.NCHAR` to the list of types exported by the
Oracle dialect.
sanjana [Wed, 13 Feb 2019 15:17:46 +0000 (10:17 -0500)]
Adding setter to should_evaluate_none property
Fixed issue where the :class:`.JSON` type had a read-only
:attr:`.JSON.should_evaluate_none` attribute, which would cause failures
when making use of the :meth:`.TypeEngine.evaluates_none` method in
conjunction with this type. Pull request courtesy Sanjana S.
Mike Bayer [Wed, 13 Feb 2019 16:26:54 +0000 (11:26 -0500)]
Set IDENTITY_INSERT for insert.values({column: expr})
Fixed bug where the SQL Server "IDENTITY_INSERT" logic that allows an INSERT
to proceed with an explicit value on an IDENTITY column was not detecting
the case where :meth:`.Insert.values` were used with a dictionary that
contained a :class:`.Column` as key and a SQL expression as a value.
Mike Bayer [Mon, 11 Feb 2019 22:00:47 +0000 (17:00 -0500)]
Allow SQL expression for ORM primary keys
A SQL expression can now be assigned to a primary key attribute for an ORM
flush in the same manner as ordinary attributes as described in
:ref:`flush_embedded_sql_expressions` where the expression will be evaulated
and then returned to the ORM using RETURNING, or in the case of pysqlite,
works using the cursor.lastrowid attribute.Requires either a database that
supports RETURNING (e.g. Postgresql, Oracle, SQL Server) or pysqlite.
Mike Bayer [Mon, 11 Feb 2019 14:07:03 +0000 (09:07 -0500)]
Add complete coverage and fix lower() for MySQL 88718 workaround
Fixed a second regression caused by :ticket:`4344` (the first was
:ticket:`4361`), which works around MySQL issue 88718, where the lower
casing function used was not correct for Python 2 with OSX/Windows casing
conventions, which would then raise ``TypeError``. Full coverage has been
added to this logic so that every codepath is exercised in a mock style for
all three casing conventions on all versions of Python. MySQL 8.0 has
meanwhile fixed issue 88718 so the workaround is only applies to a
particular span of MySQL 8.0 versions.
Mike Bayer [Sat, 9 Feb 2019 06:46:06 +0000 (01:46 -0500)]
Don't run pending_to_persistent for non-new objects
Fixed fairly simple but critical issue where the
:meth:`.SessionEvents.pending_to_persistent` event would be invoked for
objects not just when they move from pending to persistent, but when they
were also already persistent and just being updated, thus causing the event
to be invoked for all objects on every update.
Ivan Levkivskyi [Fri, 8 Feb 2019 19:10:19 +0000 (19:10 +0000)]
Add type stubs link to the docs
Moved by Mike to be a sidebar in the Declarative docs, as
the plugin appears to be specific to Declarative. These docs will
eventually be moved out of the "extension" section and merged
into the primary narrative documentation.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: Idba4c13ffc920b1140b5e51d206ab02f19015e8e
Mike Bayer [Mon, 4 Feb 2019 20:50:29 +0000 (15:50 -0500)]
Remove all remaining text() coercions and ensure identifiers are safe
Fully removed the behavior of strings passed directly as components of a
:func:`.select` or :class:`.Query` object being coerced to :func:`.text`
constructs automatically; the warning that has been emitted is now an
ArgumentError or in the case of order_by() / group_by() a CompileError.
This has emitted a warning since version 1.0 however its presence continues
to create concerns for the potential of mis-use of this behavior.
Note that public CVEs have been posted for order_by() / group_by() which
are resolved by this commit: CVE-2019-7164 CVE-2019-7548
Added "SQL phrase validation" to key DDL phrases that are accepted as plain
strings, including :paramref:`.ForeignKeyConstraint.on_delete`,
:paramref:`.ForeignKeyConstraint.on_update`,
:paramref:`.ExcludeConstraint.using`,
:paramref:`.ForeignKeyConstraint.initially`, for areas where a series of SQL
keywords only are expected.Any non-space characters that suggest the phrase
would need to be quoted will raise a :class:`.CompileError`. This change
is related to the series of changes committed as part of :ticket:`4481`.
Fixed issue where using an uppercase name for an index type (e.g. GIST,
BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to
be quoted, rather than rendering it as is. The new behavior converts these
types to lowercase and ensures they contain only valid SQL characters.
Quoting is applied to :class:`.Function` names, those which are usually but
not necessarily generated from the :attr:`.sql.func` construct, at compile
time if they contain illegal characters, such as spaces or punctuation. The
names are as before treated as case insensitive however, meaning if the
names contain uppercase or mixed case characters, that alone does not
trigger quoting. The case insensitivity is currently maintained for
backwards compatibility.
Mike Bayer [Sun, 27 Jan 2019 00:49:44 +0000 (19:49 -0500)]
Improve support for with_polymorphic in mapper options
Improved the behavior of :func:`.orm.with_polymorphic` in conjunction with
loader options, in particular wildcard operations as well as
:func:`.orm.load_only`. The polymorphic object will be more accurately
targeted so that column-level options on the entity will correctly take
effect.The issue is a continuation of the same kinds of things fixed in
:ticket:`4468`.
The path logic when using chained mapper options is improved
to be more accurate in terms of the entities being linked
in the path; when using :func:`.with_polymorphic`, mapper
options against this entity need to specify attributes
in terms of the with_polymorphic() object and not against the
base mappings. New error conditions are raised which were previously
more than likely silenty failures.
Mike Bayer [Sun, 3 Feb 2019 18:29:06 +0000 (13:29 -0500)]
Add bulk_replace to AssociationSet, AssociationDict
Implemented a more comprehensive assignment operation (e.g. "bulk replace")
when using association proxy with sets or dictionaries. Fixes the problem
of redundant proxy objects being created to replace the old ones, which
leads to excessive events and SQL and in the case of unique constraints
will cause the flush to fail.
Parth Shandilya [Fri, 1 Feb 2019 04:24:29 +0000 (23:24 -0500)]
Remove Nose support
The test system has removed support for Nose, which is unmaintained for
several years and is producing warnings under Python 3. The test suite is
currently standardized on Pytest. Pull request courtesy Parth Shandilya.
Render parenthesis around sqlite expression defaults
Fixed bug in SQLite DDL where using an expression as a server side default
required that it be contained within parenthesis to be accepted by the
sqlite parser. Pull request courtesy Bartlomiej Biernacki.
Mike Bayer [Wed, 30 Jan 2019 05:53:10 +0000 (23:53 -0600)]
Add informative failure modes to _DeferredMapperConfig
Added some helper exceptions that invoke when a mapping based on
:class:`.AbstractConcreteBase`, :class:`.DeferredReflection`, or
:class:`.AutoMap` is used before the mapping is ready to be used, which
contain descriptive information on the class, rather than falling through
into other failure modes that are less informative.
Mike Bayer [Sat, 26 Jan 2019 19:53:45 +0000 (14:53 -0500)]
Ensure of_type subclass taken into account with wildcards
Fixed a regression in 1.2 where a wildcard/load_only loader option would
not work correctly against a loader path where of_type() were used to limit
to a particular subclass. The fix only works for of_type() of a simple
subclass so far, not a with_polymorphic entity which will be addressed in a
separate issue; it is unlikely this latter case was working previously.
Since we ensure that the entity is broken out into its superclasses
when a wilcard is encountered, we can limit the entity path to the
specific entity given in this case.
Within this issue some additional issues with with_polymorphic()
loaders were found which will be addressed in #4469.
Mike Bayer [Wed, 9 Jan 2019 07:01:16 +0000 (02:01 -0500)]
Improve error messages in the area of loader options
Improved error messages emitted by the ORM in the area of loader option
traversal. This includes early detection of mis-matched loader strategies
along with a clearer explanation why these strategies don't match.
Lele Gaifax [Mon, 14 Jan 2019 16:26:33 +0000 (11:26 -0500)]
Fix many spell glitches
This affects mostly docstrings, except in orm/events.py::dispose_collection()
where one parameter gets renamed: given that the method is
empty, it seemed reasonable to me to fix that too.
Mike Bayer [Fri, 25 Jan 2019 16:36:54 +0000 (11:36 -0500)]
Fix mssql quote schema warning
The deprecations review didn't include tests of identifier_preparer.quote.force
for backends, so MSSQL slipped through. We have to fully reimplement
the deprecation warning here so that it passes tests which are now
enabled for all backends.
Daniel Lister [Thu, 24 Jan 2019 21:35:16 +0000 (16:35 -0500)]
Add getters for all execution_options
Added accessors for execution options to Core and ORM, via
:meth:`.Query.get_execution_options`,
:meth:`.Connection.get_execution_options`,
:meth:`.Engine.get_execution_options`, and
:meth:`.Executable.get_execution_options`. PR courtesy Daniel Lister.
Mike Bayer [Sun, 30 Dec 2018 01:54:29 +0000 (20:54 -0500)]
Implement relationship to AliasedClass; deprecate non primary mappers
Implemented a new feature whereby the :class:`.AliasedClass` construct can
now be used as the target of a :func:`.relationship`. This allows the
concept of "non primary mappers" to no longer be necessary, as the
:class:`.AliasedClass` is much easier to configure and automatically inherits
all the relationships of the mapped class, as well as preserves the
ability for loader options to work normally.
- introduce new name for mapped_table, "persist_selectable". this is
the selectable that selects against the local mapper and its superclasses,
but does not include columns local only to subclasses.
- relationship gains "entity" which is the mapper or aliasedinsp.
- clarfiy name "entity" vs. "query_entity" in loader strategies.
Mike Bayer [Thu, 24 Jan 2019 21:56:44 +0000 (16:56 -0500)]
Use pg_get_constraintdef for CHECK constraint reflection
Revised the query used when reflecting CHECK constraints to make use of the
``pg_get_constraintdef`` function, as the ``consrc`` column is being
deprecated in PG 12. Thanks to John A Stevenson for the tip.
Mike Bayer [Fri, 21 Dec 2018 03:05:36 +0000 (22:05 -0500)]
Add deprecation warnings to all deprecated APIs
A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.
See the notes added to the changelog and migration notes for further
details.
Added new event hooks :meth:`.QueryEvents.before_compile_update` and
:meth:`.QueryEvents.before_compile_delete` which complement
:meth:`.QueryEvents.before_compile` in the case of the :meth:`.Query.update`
and :meth:`.Query.delete` methods.
Mike Bayer [Sat, 19 Jan 2019 02:30:21 +0000 (21:30 -0500)]
Don't use cx_Oracle.NATIVE_INT in output type handlers
Fixed regression in integer precision logic due to the refactor of the
cx_Oracle dialect in 1.2. We now no longer apply the cx_Oracle.NATIVE_INT
type to result columns sending integer values (detected as positive
precision with scale ==0) which encounters integer overflow issues with
values that go beyond the 32 bit boundary. Instead, the output variable
is left untyped so that cx_Oracle can choose the best option.
Mike Bayer [Fri, 18 Jan 2019 04:38:40 +0000 (23:38 -0500)]
Cleanup with query aliasing
Try to simplify some of the "adapter" stuff in query:
1. identify that join(.., aliased=True) doesn't work if the
right side has no mapper. The adaption of the right side is
done via the mapper with aliased(), so that doesn't effect
a selectable only. raise an error, so we can simplify
the code.
2. build fewer adapter objects. these are confusing to follow
and we should try to figure out exactly what purpose which
one serves where and make that clear.
Mike Bayer [Fri, 18 Jan 2019 06:02:07 +0000 (01:02 -0500)]
Feature mysqlclient
MySQL-Python hasn't been released in about five years and there
is no reason that it should be listed as a viable DBAPI;
all MySQL / MariaDB users should be using mysqlclient or PyMySQL today.
Mike Bayer [Fri, 18 Jan 2019 01:08:10 +0000 (20:08 -0500)]
Adapt single inh criteria more specifically
Fixed issue where when using single-table inheritance in conjunction with a
joined inheritance hierarchy that uses "with polymorphic" loading, the
"single table criteria" for that single-table entity could get confused for
that of other entities from the same hierarchy used in the same query.The
adaption of the "single table criteria" is made more specific to the target
entity to avoid it accidentally getting adapted to other tables in the
query.
Mike Bayer [Thu, 17 Jan 2019 18:16:04 +0000 (13:16 -0500)]
Repair use of deprecated text() typemap, bindparams parameters
These will emit a deprecation warning once
If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b is merged, modernize these
ahead of time as this should likely be backported to 1.2 as well.
Mike Bayer [Sun, 13 Jan 2019 23:15:52 +0000 (18:15 -0500)]
Render N'' for SQL Server unicode literals
The ``literal_processor`` for the :class:`.Unicode` and
:class:`.UnicodeText` datatypes now render an ``N`` character in front of
the literal string expression as required by SQL Server for Unicode string
values rendered in SQL expressions.
Note that this adds full unicode characters to the standard test suite,
which means we also need to bump MySQL provisioning up to utf8mb4.
Modern installs do not seem to be reproducing the 1271 issue locally,
if it reproduces in CI it would be better for us to skip those ORM-centric
tests for MySQL.
Also remove unused _StringType from SQL Server dialect
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
