Mike Bayer [Sun, 6 Jan 2019 06:19:47 +0000 (01:19 -0500)]
Post black reformatting
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Mike Bayer [Sun, 6 Jan 2019 06:14:26 +0000 (01:14 -0500)]
Run black -l 79 against all source files
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Mike Bayer [Sun, 6 Jan 2019 03:42:08 +0000 (22:42 -0500)]
Assorted pre-Black fixes
Fixes to the test suite, a few errant imports, and setup.py:
- mysql and postgresql have unused 'json' imports; remove
- postgresql is exporting the 'json' symbol, remove
- make sure setup.py can find __version__ using " or '
- retry logic in provision create database for postgresql fixed
- refactor test_magazine to use cls.tables rather than globals
- remove unused class in test_scoping
- add a comment to test_deprecations that this test suite itself
is deprecated
- don't use mapper() and orm_mapper() in test_unitofwork, just
use mapper()
- remove dupe test_scalar_set_None test in test_attributes
- Python 2.7 and above includes unittest.SkipTest, remove pre-2.7
fallback
- use imported SkipTest in profiling
- declarative test_reflection tests with "reflectable_autoincrement"
already don't run on oracle or firebird; remove conditional logic
for these, which also removes an "id" symbol
- clean up test in test_functions, remove print statement
- remove dupe test_literal_processor_coercion_native_int_out_of_range
in test/sql/test_types.py
- fix psycopg2_hstore ref
Mike Bayer [Fri, 4 Jan 2019 03:28:09 +0000 (22:28 -0500)]
Prep for flake8 refactoring
a few code changes ahead of time to handle some __all__
issues better. also include new flake8 rules, since the
existing flake8 doesn't pass in any case.
Mike Bayer [Fri, 28 Dec 2018 14:33:07 +0000 (09:33 -0500)]
Add missing disambiguation to non-primary mapper example
The mapping to B over a join defines an alternate primary key
based on all the primary key columns in the join unless we
re-define it explicitly. Similarly, people expect that
``.id`` looks the same. make sure these line up with the
old mapping.
Mike Bayer [Sun, 23 Dec 2018 00:16:50 +0000 (19:16 -0500)]
Check collection less than two items remaining before firing scalar backref remove
Fixed long-standing issue where duplicate collection members would cause a
backref to delete the association between the member and its parent object
when one of the duplicates were removed, as occurs as a side effect of
swapping two objects in one statement.
Mike Bayer [Fri, 28 Dec 2018 03:10:45 +0000 (22:10 -0500)]
Call __del() before remove()
The "remove" event for collections is now called before the item is removed
in the case of the ``collection.remove()`` method, as is consistent with the
behavior for most other forms of collection item removal (such as
``__delitem__``, replacement under ``__setitem__``). The ``pop()`` methods
are now the only exception as the target item is not available until after
the pop operation proceeds.
This allows ``remove()`` to be consistent in its behavior with all
the other collection operations, allows the "before_delete" hook
to be local to "pop()" operations only, and removes some method overhead.
We are also looking here to gain some more predictability in terms
of the fix for #1103.
Mike Bayer [Thu, 27 Dec 2018 17:39:15 +0000 (12:39 -0500)]
Implement SynonymProperty.get_history()
Implemented the ``.get_history()`` method, which also implies availability
of :attr:`.AttributeState.history`, for :func:`.synonym` attributes.
Previously, trying to access attribute history via a synonym would raise an
``AttributeError``.
Mike Bayer [Fri, 21 Dec 2018 22:35:12 +0000 (17:35 -0500)]
Maintain compiled_params / replacement_expressions within expanding IN
Fixed issue in "expanding IN" feature where using the same bound parameter
name more than once in a query would lead to a KeyError within the process
of rewriting the parameters in the query.
Mike Bayer [Fri, 21 Dec 2018 16:04:14 +0000 (11:04 -0500)]
Handle PostgreSQL enums in remote schemas
Fixed issue where a :class:`.postgresql.ENUM` or a custom domain present
in a remote schema would not be recognized within column reflection if
the name of the enum/domain or the name of the schema required quoting.
A new parsing scheme now fully parses out quoted or non-quoted tokens
including support for SQL-escaped quotes.
Fixed issue where multiple :class:`.postgresql.ENUM` objects referred to
by the same :class:`.MetaData` object would fail to be created if
multiple objects had the same name under different schema names. The
internal memoization the Postgresql dialect uses to track if it has
created a particular :class:`.postgresql.ENUM` in the database during
a DDL creation sequence now takes schema name into account.
Michael Howitz [Mon, 17 Dec 2018 16:04:44 +0000 (11:04 -0500)]
Omit include of no longer existing files
Warnings are generated during setup.py install such as:
```
Getting distribution for 'SQLAlchemy==1.2.15'.
warning: no files found matching '*.jpg' under directory 'doc'
warning: no files found matching '*.mako' under directory 'doc'
warning: no files found matching 'distribute_setup.py'
warning: no files found matching 'sa2to3.py'
warning: no files found matching 'ez_setup.py'
```
This change removes those expressions from MANIFEST.in.
The current example code does not pass `**kw` down to the `compiler.process` calls, thus the example does not work when invoked with, `literal_binds=True`.
Besides, the calls to `process` each argument twice are wasteful, and reusing the built-in `case` expression instead of hard-coding the SQL statements is slightly nicer overall, isn't it?
Agam Rafaeli [Mon, 17 Dec 2018 16:20:09 +0000 (11:20 -0500)]
Set a reading order to the compatibility import file
The order is as follows:
1. Imports, in the same format of "import X"
2. Members denoting versions of python
3. Members of imports that are shadowed for usage across the system
4. Global variables
5. If conditions of imports. (internally organized by this order)
6. Function definitions
Mike Bayer [Wed, 12 Dec 2018 17:51:20 +0000 (12:51 -0500)]
Document and support nested composites
Composites can behave in a "nested" fashion by defining the
class in that way. To make the constructor more convenient,
a callable can be passed to :func:`.composite` instead of the
class itself. This works now, so add a test to ensure this
pattern remains available.
Chris Wilson [Mon, 10 Dec 2018 21:37:51 +0000 (16:37 -0500)]
Add before_mapper_configured event
This event is intended to allow a specific mapper to be skipped during the
configure step, by returning a value of `.orm.interfaces.EXT_SKIP` which means
the mapper will be skipped within this configure run. The "new mappers" flag
will remain set in this case and the configure operation will occur again.
This event, and its return value, make it possible to query one base while a
different one still needs configuration, which cannot be completed at this
time.
Mike Bayer [Fri, 7 Dec 2018 18:08:29 +0000 (13:08 -0500)]
Take instance into account when determining AssociationProxyInstance
Fixed a regression in 1.3.0b1 caused by :ticket:`3423` where association
proxy objects that access an attribute that's only present on a polymorphic
subclass would raise an ``AttributeError`` even though the actual instance
being accessed was an instance of that subclass.
Mike Bayer [Fri, 7 Dec 2018 21:01:04 +0000 (16:01 -0500)]
Refer to existing of_type when resolving string attribute name
Fixed bug where chaining of mapper options using
:meth:`.RelationshipProperty.of_type` in conjunction with a chained option
that refers to an attribute name by string only would fail to locate the
attribute.
Mike Bayer [Fri, 7 Dec 2018 19:01:23 +0000 (14:01 -0500)]
Add versioned_update_old_row example
A modification to the "turn UPDATE into INSERT" recipe that
also UPDATEs the previous row. The example is using timestamps
to maintain a relationship between two objects.
Mike Bayer [Sat, 1 Dec 2018 19:28:57 +0000 (14:28 -0500)]
Move CRUDTest, InlineDefaultTest from test_compiler
test_compiler is mostly related to SELECT statements as well
as smaller SQL elements. While it still has some DDL related
tests, move out all the remaining insert/update tests into
the already present test_insert.py, test_update.py
Mike Bayer [Fri, 30 Nov 2018 19:36:20 +0000 (14:36 -0500)]
Apply path generation for superclasses to Load._set_path_strategy()
Extended the fix first made as part of :ticket:`3287`, where a loader option
made against a subclass using a wildcard would extend itself to include
application of the wildcard to attributes on the super classes as well, to a
"bound" loader option as well, e.g. in an expression like
``Load(SomeSubClass).load_only('foo')``. Columns that are part of the
parent class of ``SomeSubClass`` will also be excluded in the same way as if
the unbound option ``load_only('foo')`` were used.
jun923.gu [Fri, 30 Nov 2018 05:21:29 +0000 (00:21 -0500)]
Add the write_timeout option for mysql.
As MySQLdb support read_timeout and write_timeout option, and
sqlalchemy just support read_timeout option. So sqlalchemy need to add
write_timeout option.
Mike Bayer [Thu, 29 Nov 2018 15:13:03 +0000 (10:13 -0500)]
Allow optional *args with base AnsiFunction
Amended the :class:`.AnsiFunction` class, the base of common SQL
functions like ``CURRENT_TIMESTAMP``, to accept positional arguments
like a regular ad-hoc function. This to suit the case that many of
these functions on specific backends accept arguments such as
"fractional seconds" precision and such. If the function is created
with arguments, it renders the the parenthesis and the arguments. If
no arguents are present, the compiler generates the non-parenthesized form.
Mike Bayer [Thu, 22 Nov 2018 15:39:29 +0000 (10:39 -0500)]
Add __clause_element__ to ColumnProperty
Added a ``__clause_element__()`` method to :class:`.ColumnProperty` which
can allow the usage of a not-fully-declared column or deferred attribute in
a declarative mapped class slightly more friendly when it's used in a
constraint or other column-oriented scenario within the class declaration,
though this still can't work in open-ended expressions; prefer to call the
:attr:`.ColumnProperty.expression` attribute if receiving ``TypeError``.
Mike Bayer [Mon, 26 Nov 2018 05:59:01 +0000 (00:59 -0500)]
Warn for lower-case column attribute on declarative
A warning is emitted in the case that a :func:`.column` object is applied to
a declarative class, as it seems likely this intended to be a
:class:`.Column` object.
Mike Bayer [Fri, 16 Nov 2018 00:48:45 +0000 (19:48 -0500)]
Repair tutorial code example to be deterministic
A few queries featured an aggregate without a GROUP BY
or a complete ordering, causing the doctests to fail
with recent versions of SQLite. The queries are now made
to include a GROUP BY as well as ORDER BY both columns.
Mike Bayer [Thu, 15 Nov 2018 17:11:12 +0000 (12:11 -0500)]
Modernize cx_Oracle parameters
Updated the parameters that can be sent to the cx_Oracle DBAPI to both allow
for all current parameters as well as for future parameters not added yet.
In addition, removed unused parameters that were deprecated in version 1.2,
and additionally we are now defaulting "threaded" to False.
Mike Bayer [Mon, 12 Nov 2018 23:27:34 +0000 (18:27 -0500)]
Allow join() to pick the best candidate from multiple froms/entities
Refactored :meth:`.Query.join` to further clarify the individual components
of structuring the join. This refactor adds the ability for
:meth:`.Query.join` to determine the most appropriate "left" side of the
join when there is more than one element in the FROM list or the query is
against multiple entities. In particular this targets the regression we
saw in :ticket:`4363` but is also of general use. The codepaths within
:meth:`.Query.join` are now easier to follow and the error cases are
decided more specifically at an earlier point in the operation.
Mike Bayer [Wed, 14 Nov 2018 16:02:40 +0000 (11:02 -0500)]
Add secondary selectable to FROM clauses for correlated exists
In continuing with a similar theme as that of very recent :ticket:`4349`,
repaired issue with :meth:`.RelationshipProperty.Comparator.any` and
:meth:`.RelationshipProperty.Comparator.has` where the "secondary"
selectable needs to be explicitly part of the FROM clause in the
EXISTS subquery to suit the case where this "secondary" is a :class:`.Join`
object.
Mike Bayer [Sun, 11 Nov 2018 02:36:18 +0000 (21:36 -0500)]
Add new parameters for IDENTITY start/increment in mssql
Deprecated the use of :class:`.Sequence` with SQL Server in order to affect
the "start" and "increment" of the IDENTITY value, in favor of new
parameters ``mssql_identity_start`` and ``mssql_identity_increment`` which
set these parameters directly. :class:`.Sequence` will be used to generate
real ``CREATE SEQUENCE`` DDL with SQL Server in a future release.
Mike Bayer [Wed, 14 Nov 2018 16:31:22 +0000 (11:31 -0500)]
Deannotate "parententity" in primaryjoin/secondaryjoin
Fixed bug where the ORM annotations could be incorrect for the
primaryjoin/secondaryjoin a relationship if one used the pattern
``ForeignKey(SomeClass.id)`` in the declarative mappings. This pattern
would leak undesired annotations into the join conditions which can break
aliasing operations done within :class:`.Query` that are not supposed to
impact elements in that join condition. These annotations are now removed
up front if present.
Also add a test suite for has/any into test_query which will
form the basis for new tests to be added in :ticket:`4366`.
Mike Bayer [Mon, 12 Nov 2018 20:09:37 +0000 (15:09 -0500)]
Insert primary entity in dynamic "secondary"
Fixed regression caused by :ticket:`4349` where adding the "secondary"
table to the FROM clause for a dynamic loader would affect the ability of
the :class:`.Query` to make a subsequent join to another entity. The fix
adds the primary entity as the first element of the FROM list since
:meth:`.Query.join` wants to jump from that. Version 1.3 will have
a more comprehensive solution to this problem as well (:ticket:`4365`).
Mike Bayer [Sun, 11 Nov 2018 02:10:51 +0000 (21:10 -0500)]
Add Sequence to StrSQLCompiler
Added :class:`.Sequence` to the "string SQL" system that will render a
meaningful string expression (``"<next sequence value: my_sequence>"``)
when stringifying without a dialect a statement that includes a "sequence
nextvalue" expression, rather than raising a compilation error.
Mike Bayer [Wed, 7 Nov 2018 16:21:39 +0000 (11:21 -0500)]
Modernize deferred callable for many-to-one comparison
Improved the behavior of a relationship-bound many-to-one object expression
such that the retrieval of column values on the related object are now
resilient against the object being detached from its parent
:class:`.Session`, even if the attribute has been expired. New features
within the :class:`.InstanceState` are used to memoize the last known value
of a particular column attribute before its expired, so that the expression
can still evaluate when the object is detached and expired at the same
time. Error conditions are also improved using modern attribute state
features to produce more specific messages as needed.
To support the value being mutated while also being resilient towards
expiration, a new feature to InstanceState is added ._last_known_values
which holds onto the expired value when an individual key is expired.
Only takes effect specific to keys and InstanceState objects that
received a special instruction so this does not add to overall
memory/latency.
Mike Bayer [Mon, 5 Nov 2018 19:52:35 +0000 (14:52 -0500)]
Add new "all columns" naming convention tokens
Added new naming convention tokens ``column_0N_name``, ``column_0_N_name``,
etc., which will render the names / keys / labels for all columns referenced
by a particular constraint in a sequence. In order to accommodate for the
length of such a naming convention, the SQL compiler's auto-truncation
feature now applies itself to constraint names as well, which creates a
shortened, deterministically generated name for the constraint that will
apply to a target backend without going over the character limit of that
backend.
Additional notes:
1. the SQLite dialect had a format_index method that was apparently not
used, removed.
2. the naming convention logic has been applying the foreign key
remote column spec to the naming convention, and not the actual
column name. In the case where the referenced Table object uses
.key inside the columns and these are what ForeignKey() references,
the naming convention was doing the wrong thing. The patch here
fixes this, however this isn't noted in the migration notes.
Mike Bayer [Sat, 10 Nov 2018 04:18:55 +0000 (23:18 -0500)]
Use case insensitive matching on lower_case_table_names=1,2
Fixed regression caused by :ticket:`4344` released in 1.2.13, where the fix
for MySQL 8.0's case sensitivity problem with referenced column names when
reflecting foreign key referents is worked around using the
``information_schema.columns`` view. The workaround was failing on OSX /
``lower_case_table_names=2`` which produces non-matching casing for the
``information_schema.columns`` vs. that of ``SHOW CREATE TABLE``, so in
case-insensitive SQL modes case-insensitive matching is now used.
Denis Kataev [Mon, 12 Mar 2018 15:40:34 +0000 (11:40 -0400)]
Implement SQLite ON CONFLICT for constraints
Implemented the SQLite ``ON CONFLICT`` clause as understood at the DDL
level, e.g. for primary key, unique, and CHECK constraints as well as
specified on a :class:`.Column` to satisfy inline primary key and NOT NULL.
Pull request courtesy Denis Kataev.
Mike Bayer [Tue, 6 Nov 2018 20:13:03 +0000 (15:13 -0500)]
Improve documentation re: Session.binds and partitioning strategies
Update documentation to include background on arbitrary superclass
usage, add full cross-linking between all related methods and parameters.
De-emphasize "twophase" and document that it is not well-supported
in drivers.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
