sanjana [Wed, 13 Feb 2019 15:17:46 +0000 (10:17 -0500)]
Adding setter to should_evaluate_none property
Fixed issue where the :class:`.JSON` type had a read-only
:attr:`.JSON.should_evaluate_none` attribute, which would cause failures
when making use of the :meth:`.TypeEngine.evaluates_none` method in
conjunction with this type. Pull request courtesy Sanjana S.
Mike Bayer [Wed, 13 Feb 2019 16:26:54 +0000 (11:26 -0500)]
Set IDENTITY_INSERT for insert.values({column: expr})
Fixed bug where the SQL Server "IDENTITY_INSERT" logic that allows an INSERT
to proceed with an explicit value on an IDENTITY column was not detecting
the case where :meth:`.Insert.values` were used with a dictionary that
contained a :class:`.Column` as key and a SQL expression as a value.
Mike Bayer [Mon, 11 Feb 2019 22:00:47 +0000 (17:00 -0500)]
Allow SQL expression for ORM primary keys
A SQL expression can now be assigned to a primary key attribute for an ORM
flush in the same manner as ordinary attributes as described in
:ref:`flush_embedded_sql_expressions` where the expression will be evaulated
and then returned to the ORM using RETURNING, or in the case of pysqlite,
works using the cursor.lastrowid attribute.Requires either a database that
supports RETURNING (e.g. Postgresql, Oracle, SQL Server) or pysqlite.
Mike Bayer [Mon, 11 Feb 2019 14:07:03 +0000 (09:07 -0500)]
Add complete coverage and fix lower() for MySQL 88718 workaround
Fixed a second regression caused by :ticket:`4344` (the first was
:ticket:`4361`), which works around MySQL issue 88718, where the lower
casing function used was not correct for Python 2 with OSX/Windows casing
conventions, which would then raise ``TypeError``. Full coverage has been
added to this logic so that every codepath is exercised in a mock style for
all three casing conventions on all versions of Python. MySQL 8.0 has
meanwhile fixed issue 88718 so the workaround is only applies to a
particular span of MySQL 8.0 versions.
Mike Bayer [Sat, 9 Feb 2019 06:46:06 +0000 (01:46 -0500)]
Don't run pending_to_persistent for non-new objects
Fixed fairly simple but critical issue where the
:meth:`.SessionEvents.pending_to_persistent` event would be invoked for
objects not just when they move from pending to persistent, but when they
were also already persistent and just being updated, thus causing the event
to be invoked for all objects on every update.
Ivan Levkivskyi [Fri, 8 Feb 2019 19:10:19 +0000 (19:10 +0000)]
Add type stubs link to the docs
Moved by Mike to be a sidebar in the Declarative docs, as
the plugin appears to be specific to Declarative. These docs will
eventually be moved out of the "extension" section and merged
into the primary narrative documentation.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: Idba4c13ffc920b1140b5e51d206ab02f19015e8e
Mike Bayer [Mon, 4 Feb 2019 20:50:29 +0000 (15:50 -0500)]
Remove all remaining text() coercions and ensure identifiers are safe
Fully removed the behavior of strings passed directly as components of a
:func:`.select` or :class:`.Query` object being coerced to :func:`.text`
constructs automatically; the warning that has been emitted is now an
ArgumentError or in the case of order_by() / group_by() a CompileError.
This has emitted a warning since version 1.0 however its presence continues
to create concerns for the potential of mis-use of this behavior.
Note that public CVEs have been posted for order_by() / group_by() which
are resolved by this commit: CVE-2019-7164 CVE-2019-7548
Added "SQL phrase validation" to key DDL phrases that are accepted as plain
strings, including :paramref:`.ForeignKeyConstraint.on_delete`,
:paramref:`.ForeignKeyConstraint.on_update`,
:paramref:`.ExcludeConstraint.using`,
:paramref:`.ForeignKeyConstraint.initially`, for areas where a series of SQL
keywords only are expected.Any non-space characters that suggest the phrase
would need to be quoted will raise a :class:`.CompileError`. This change
is related to the series of changes committed as part of :ticket:`4481`.
Fixed issue where using an uppercase name for an index type (e.g. GIST,
BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to
be quoted, rather than rendering it as is. The new behavior converts these
types to lowercase and ensures they contain only valid SQL characters.
Quoting is applied to :class:`.Function` names, those which are usually but
not necessarily generated from the :attr:`.sql.func` construct, at compile
time if they contain illegal characters, such as spaces or punctuation. The
names are as before treated as case insensitive however, meaning if the
names contain uppercase or mixed case characters, that alone does not
trigger quoting. The case insensitivity is currently maintained for
backwards compatibility.
Mike Bayer [Sun, 27 Jan 2019 00:49:44 +0000 (19:49 -0500)]
Improve support for with_polymorphic in mapper options
Improved the behavior of :func:`.orm.with_polymorphic` in conjunction with
loader options, in particular wildcard operations as well as
:func:`.orm.load_only`. The polymorphic object will be more accurately
targeted so that column-level options on the entity will correctly take
effect.The issue is a continuation of the same kinds of things fixed in
:ticket:`4468`.
The path logic when using chained mapper options is improved
to be more accurate in terms of the entities being linked
in the path; when using :func:`.with_polymorphic`, mapper
options against this entity need to specify attributes
in terms of the with_polymorphic() object and not against the
base mappings. New error conditions are raised which were previously
more than likely silenty failures.
Mike Bayer [Sun, 3 Feb 2019 18:29:06 +0000 (13:29 -0500)]
Add bulk_replace to AssociationSet, AssociationDict
Implemented a more comprehensive assignment operation (e.g. "bulk replace")
when using association proxy with sets or dictionaries. Fixes the problem
of redundant proxy objects being created to replace the old ones, which
leads to excessive events and SQL and in the case of unique constraints
will cause the flush to fail.
Parth Shandilya [Fri, 1 Feb 2019 04:24:29 +0000 (23:24 -0500)]
Remove Nose support
The test system has removed support for Nose, which is unmaintained for
several years and is producing warnings under Python 3. The test suite is
currently standardized on Pytest. Pull request courtesy Parth Shandilya.
Render parenthesis around sqlite expression defaults
Fixed bug in SQLite DDL where using an expression as a server side default
required that it be contained within parenthesis to be accepted by the
sqlite parser. Pull request courtesy Bartlomiej Biernacki.
Mike Bayer [Wed, 30 Jan 2019 05:53:10 +0000 (23:53 -0600)]
Add informative failure modes to _DeferredMapperConfig
Added some helper exceptions that invoke when a mapping based on
:class:`.AbstractConcreteBase`, :class:`.DeferredReflection`, or
:class:`.AutoMap` is used before the mapping is ready to be used, which
contain descriptive information on the class, rather than falling through
into other failure modes that are less informative.
Mike Bayer [Sat, 26 Jan 2019 19:53:45 +0000 (14:53 -0500)]
Ensure of_type subclass taken into account with wildcards
Fixed a regression in 1.2 where a wildcard/load_only loader option would
not work correctly against a loader path where of_type() were used to limit
to a particular subclass. The fix only works for of_type() of a simple
subclass so far, not a with_polymorphic entity which will be addressed in a
separate issue; it is unlikely this latter case was working previously.
Since we ensure that the entity is broken out into its superclasses
when a wilcard is encountered, we can limit the entity path to the
specific entity given in this case.
Within this issue some additional issues with with_polymorphic()
loaders were found which will be addressed in #4469.
Mike Bayer [Wed, 9 Jan 2019 07:01:16 +0000 (02:01 -0500)]
Improve error messages in the area of loader options
Improved error messages emitted by the ORM in the area of loader option
traversal. This includes early detection of mis-matched loader strategies
along with a clearer explanation why these strategies don't match.
Lele Gaifax [Mon, 14 Jan 2019 16:26:33 +0000 (11:26 -0500)]
Fix many spell glitches
This affects mostly docstrings, except in orm/events.py::dispose_collection()
where one parameter gets renamed: given that the method is
empty, it seemed reasonable to me to fix that too.
Mike Bayer [Fri, 25 Jan 2019 16:36:54 +0000 (11:36 -0500)]
Fix mssql quote schema warning
The deprecations review didn't include tests of identifier_preparer.quote.force
for backends, so MSSQL slipped through. We have to fully reimplement
the deprecation warning here so that it passes tests which are now
enabled for all backends.
Daniel Lister [Thu, 24 Jan 2019 21:35:16 +0000 (16:35 -0500)]
Add getters for all execution_options
Added accessors for execution options to Core and ORM, via
:meth:`.Query.get_execution_options`,
:meth:`.Connection.get_execution_options`,
:meth:`.Engine.get_execution_options`, and
:meth:`.Executable.get_execution_options`. PR courtesy Daniel Lister.
Mike Bayer [Sun, 30 Dec 2018 01:54:29 +0000 (20:54 -0500)]
Implement relationship to AliasedClass; deprecate non primary mappers
Implemented a new feature whereby the :class:`.AliasedClass` construct can
now be used as the target of a :func:`.relationship`. This allows the
concept of "non primary mappers" to no longer be necessary, as the
:class:`.AliasedClass` is much easier to configure and automatically inherits
all the relationships of the mapped class, as well as preserves the
ability for loader options to work normally.
- introduce new name for mapped_table, "persist_selectable". this is
the selectable that selects against the local mapper and its superclasses,
but does not include columns local only to subclasses.
- relationship gains "entity" which is the mapper or aliasedinsp.
- clarfiy name "entity" vs. "query_entity" in loader strategies.
Mike Bayer [Thu, 24 Jan 2019 21:56:44 +0000 (16:56 -0500)]
Use pg_get_constraintdef for CHECK constraint reflection
Revised the query used when reflecting CHECK constraints to make use of the
``pg_get_constraintdef`` function, as the ``consrc`` column is being
deprecated in PG 12. Thanks to John A Stevenson for the tip.
Mike Bayer [Fri, 21 Dec 2018 03:05:36 +0000 (22:05 -0500)]
Add deprecation warnings to all deprecated APIs
A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.
See the notes added to the changelog and migration notes for further
details.
Added new event hooks :meth:`.QueryEvents.before_compile_update` and
:meth:`.QueryEvents.before_compile_delete` which complement
:meth:`.QueryEvents.before_compile` in the case of the :meth:`.Query.update`
and :meth:`.Query.delete` methods.
Mike Bayer [Sat, 19 Jan 2019 02:30:21 +0000 (21:30 -0500)]
Don't use cx_Oracle.NATIVE_INT in output type handlers
Fixed regression in integer precision logic due to the refactor of the
cx_Oracle dialect in 1.2. We now no longer apply the cx_Oracle.NATIVE_INT
type to result columns sending integer values (detected as positive
precision with scale ==0) which encounters integer overflow issues with
values that go beyond the 32 bit boundary. Instead, the output variable
is left untyped so that cx_Oracle can choose the best option.
Mike Bayer [Fri, 18 Jan 2019 04:38:40 +0000 (23:38 -0500)]
Cleanup with query aliasing
Try to simplify some of the "adapter" stuff in query:
1. identify that join(.., aliased=True) doesn't work if the
right side has no mapper. The adaption of the right side is
done via the mapper with aliased(), so that doesn't effect
a selectable only. raise an error, so we can simplify
the code.
2. build fewer adapter objects. these are confusing to follow
and we should try to figure out exactly what purpose which
one serves where and make that clear.
Mike Bayer [Fri, 18 Jan 2019 06:02:07 +0000 (01:02 -0500)]
Feature mysqlclient
MySQL-Python hasn't been released in about five years and there
is no reason that it should be listed as a viable DBAPI;
all MySQL / MariaDB users should be using mysqlclient or PyMySQL today.
Mike Bayer [Fri, 18 Jan 2019 01:08:10 +0000 (20:08 -0500)]
Adapt single inh criteria more specifically
Fixed issue where when using single-table inheritance in conjunction with a
joined inheritance hierarchy that uses "with polymorphic" loading, the
"single table criteria" for that single-table entity could get confused for
that of other entities from the same hierarchy used in the same query.The
adaption of the "single table criteria" is made more specific to the target
entity to avoid it accidentally getting adapted to other tables in the
query.
Mike Bayer [Thu, 17 Jan 2019 18:16:04 +0000 (13:16 -0500)]
Repair use of deprecated text() typemap, bindparams parameters
These will emit a deprecation warning once
If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b is merged, modernize these
ahead of time as this should likely be backported to 1.2 as well.
Mike Bayer [Sun, 13 Jan 2019 23:15:52 +0000 (18:15 -0500)]
Render N'' for SQL Server unicode literals
The ``literal_processor`` for the :class:`.Unicode` and
:class:`.UnicodeText` datatypes now render an ``N`` character in front of
the literal string expression as required by SQL Server for Unicode string
values rendered in SQL expressions.
Note that this adds full unicode characters to the standard test suite,
which means we also need to bump MySQL provisioning up to utf8mb4.
Modern installs do not seem to be reproducing the 1271 issue locally,
if it reproduces in CI it would be better for us to skip those ORM-centric
tests for MySQL.
Also remove unused _StringType from SQL Server dialect
Mike Bayer [Mon, 14 Jan 2019 23:53:58 +0000 (18:53 -0500)]
Relax "ambiguous" association proxy restrictions, support Proxy
Fixed issue in association proxy due to :ticket:`3423` which caused the use
of custom :class:`.PropComparator` objects with hybrid attribites, such as
the one demonstrated in the ``dictlike-polymorphic`` example to not
function within an association proxy. The strictness that was added in
:ticket:`3423` has been relaxed, and additional logic to accomodate for
an association proxy that links to a custom hybrid have been added.
Add standalone orm.close_all method and deprecate SessionMaker.close_all
Added a new function :func:`.close_all_sessions` which takes
over the task of the :meth:`.Session.close_all` method, which
is now deprecated as this is confusing as a classmethod.
Pull request courtesy Augustin Trancart.
Mike Bayer [Tue, 8 Jan 2019 22:46:55 +0000 (17:46 -0500)]
use ..deprecated directive w/ version in all cases
These changes should be ported from 1.3 back to 1.0 or
possibly 0.9 to the extent they are relevant in each
version. In 1.3 we hope to turn all deprecation documentation
into warnings.
Mike Bayer [Thu, 10 Jan 2019 17:03:40 +0000 (12:03 -0500)]
Leave bytestring exception messages as bytestrings
Fixed a regression introduced in version 1.2 where a refactor
of the :class:`.SQLAlchemyError` base exception class introduced an
inappropriate coercion of a plain string message into Unicode under
python 2k, which is not handled by the Python interpreter for characters
outside of the platform's encoding (typically ascii). The
:class:`.SQLAlchemyError` class now passes a bytestring through under
Py2K for ``__str__()`` as is the behavior of exception objects in general
under Py2K, does a safe coercion to unicode utf-8 with
backslash fallback for ``__unicode__()``. For Py3K the message is
typically unicode already, but if not is again safe-coerced with utf-8
with backslash fallback for the ``__str__()`` method.
Mike Bayer [Wed, 9 Jan 2019 16:42:02 +0000 (11:42 -0500)]
Skip expression-based index reflection for SQLite
Reflection of an index based on SQL expressions are now skipped with a
warning, in the same way as that of the Postgresql dialect, where we currently
do not support reflecting indexes that have SQL expressions within them.
Previously, an index with columns of None were produced which would break
tools like Alembic.
Mike Bayer [Wed, 9 Jan 2019 16:18:02 +0000 (11:18 -0500)]
Render correct DDL for unsetting table comments
Fixed issue where the DDL emitted for :class:`.DropTableComment`, which
will be used by an upcoming version of Alembic, was incorrect for the MySQL
and Oracle databases.
Mike Bayer [Sun, 6 Jan 2019 06:19:47 +0000 (01:19 -0500)]
Post black reformatting
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Mike Bayer [Sun, 6 Jan 2019 06:14:26 +0000 (01:14 -0500)]
Run black -l 79 against all source files
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Mike Bayer [Sun, 6 Jan 2019 03:42:08 +0000 (22:42 -0500)]
Assorted pre-Black fixes
Fixes to the test suite, a few errant imports, and setup.py:
- mysql and postgresql have unused 'json' imports; remove
- postgresql is exporting the 'json' symbol, remove
- make sure setup.py can find __version__ using " or '
- retry logic in provision create database for postgresql fixed
- refactor test_magazine to use cls.tables rather than globals
- remove unused class in test_scoping
- add a comment to test_deprecations that this test suite itself
is deprecated
- don't use mapper() and orm_mapper() in test_unitofwork, just
use mapper()
- remove dupe test_scalar_set_None test in test_attributes
- Python 2.7 and above includes unittest.SkipTest, remove pre-2.7
fallback
- use imported SkipTest in profiling
- declarative test_reflection tests with "reflectable_autoincrement"
already don't run on oracle or firebird; remove conditional logic
for these, which also removes an "id" symbol
- clean up test in test_functions, remove print statement
- remove dupe test_literal_processor_coercion_native_int_out_of_range
in test/sql/test_types.py
- fix psycopg2_hstore ref
Mike Bayer [Fri, 4 Jan 2019 03:28:09 +0000 (22:28 -0500)]
Prep for flake8 refactoring
a few code changes ahead of time to handle some __all__
issues better. also include new flake8 rules, since the
existing flake8 doesn't pass in any case.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
