Lele Long [Sat, 3 Dec 2016 18:10:07 +0000 (13:10 -0500)]
Parse (but don't record) COMMENT portion of MySQL table key
The MySQL dialect now will not warn when a reflected column has a
"COMMENT" keyword on it, but note however the comment is not yet
reflected; this is on the roadmap for a future release. Pull request
courtesy Lele Long.
Joseph Schorr [Tue, 17 Jan 2017 15:02:17 +0000 (10:02 -0500)]
Add support for prefixes on CREATE INDEX statements in MySQL
Added a new parameter ``mysql_prefix`` supported by the :class:`.Index`
construct, allows specification of MySQL-specific prefixes such as
"FULLTEXT". Pull request courtesy Joseph Schorr.
Valery Yundin [Fri, 16 Dec 2016 14:22:08 +0000 (09:22 -0500)]
Better hide engine password
Avoid putting engine password in the exception message in
`MetaData.reflect` (since exception messages often appear in logs).
Use the same redacted `__repr__` implementation in
`TLEngine` as in its base class `Engine`
Mike Bayer [Mon, 16 Jan 2017 17:10:08 +0000 (12:10 -0500)]
Add "existing" populators for subqueryload
Fixed bug in subquery loading where an object encountered as an
"existing" row, e.g. already loaded from a different path in the
same query, would not invoke subquery loaders for unloaded attributes
that specified this loading. This issue is in the same area
as that of :ticket:`3431`, :ticket:`3811` which involved
similar issues with joined loading.
Mike Bayer [Fri, 13 Jan 2017 17:43:24 +0000 (12:43 -0500)]
Use full column->type processing for ON CONFLICT SET clause
Fixed bug in new "ON CONFLICT DO UPDATE" feature where the "set"
values for the UPDATE clause would not be subject to type-level
processing, as normally takes effect to handle both user-defined
type level conversions as well as dialect-required conversions, such
as those required for JSON datatypes. Additionally, clarified that
the keys in the set_ dictionary should match the "key" of the column,
if distinct from the column name. A warning is emitted
for remaining column names that don't match column keys; for
compatibility reasons, these are emitted as they were previously.
Mike Bayer [Wed, 11 Jan 2017 15:12:12 +0000 (10:12 -0500)]
Support python3.6
Corrects some warnings and adds tox config. Adds DeprecationWarning
to the error category. Large sweep for string literals w/ backslashes
as this is common in docstrings
Set autoincrement to False; use sqlite_autoincrement in versioned_history
Ensure that the history table sets autoincrement=False, since these values
are copied in all cases; the flag will emit an error as of 1.1 if the
primary key is composite. Additionally, use the sqlite_autoincrement flag
so that SQLite uses unique primary key identifiers for new rows even if
some rows have been deleted.
Mike Bayer [Wed, 28 Dec 2016 21:32:53 +0000 (16:32 -0500)]
Pass **kw to bound params in multi values
Fixed bug where literal_binds compiler flag was not honored by the
:class:`.Insert` construct for the "multiple values" feature; the
subsequent values are now rendered as literals.
Mike Bayer [Mon, 9 Jan 2017 19:16:22 +0000 (14:16 -0500)]
Adapt from "localparent" in joinedloader
Fixed bug involving joined eager loading against multiple entities
when polymorphic inheritance is also in use which would throw
"'NoneType' object has no attribute 'isa'". The issue was introduced
by the fix for :ticket:`3611`.
Mike Bayer [Fri, 6 Jan 2017 22:02:32 +0000 (17:02 -0500)]
Tighten rules for order_by(Label) resolution
- Fixed bug originally introduced in 0.9 via :ticket:`1068` where
order_by(<some Label()>) would order by the label name based on name
alone, that is, even if the labeled expression were not at all the same
expression otherwise present, implicitly or explicitly, in the
selectable. The logic that orders by label now ensures that the
labeled expression is related to the one that resolves to that name
before ordering by the label name; additionally, the name has to
resolve to an actual label explicit in the expression elsewhere, not
just a column name. This logic is carefully kept separate from the
order by(textual name) feature that has a slightly different purpose.
plumSemPy [Wed, 4 Jan 2017 17:06:48 +0000 (12:06 -0500)]
Ensure session.no_autoflush uses finally
The :attr:`.Session.no_autoflush` context manager now ensures that
the autoflush flag is reset within a "finally" block, so that if
an exception is raised within the block, the state still resets
appropriately. Pull request courtesy Emin Arakelian.
Mike Bayer [Wed, 21 Dec 2016 15:16:31 +0000 (10:16 -0500)]
Don't select lastrowid for inline=True
- Fixed bug where SQL Server dialects would attempt to select the
last row identity for an INSERT from SELECT, failing in the case when
the SELECT has no rows. For such a statement,
the inline flag is set to True indicating no last primary key
should be fetched.
Mike Bayer [Tue, 20 Dec 2016 14:15:55 +0000 (09:15 -0500)]
Bump "table compression" flag to Oracle 10.1
- Fixed bug where the "COMPRESSION" keyword was used in the ALL_TABLES
query on Oracle 9.2; even though Oracle docs state table compression
was introduced in 9i, the actual column is not present until
10.1.
Mike Bayer [Mon, 19 Dec 2016 17:39:15 +0000 (12:39 -0500)]
Add real .entities to _BundleEntity
Fixed bug where the single-table inheritance query criteria would not
be inserted into the query in the case that the :class:`.Bundle`
construct were used as the selection criteria.
Mike Bayer [Wed, 23 Nov 2016 14:14:02 +0000 (09:14 -0500)]
Add _extend_on deduplicating set for metadata.reflect()
The "extend_existing" option of :class:`.Table` reflection would
cause indexes and constraints to be doubled up in the case that the parameter
were used with :meth:`.MetaData.reflect` (as the automap extension does)
due to tables being reflected both within the foreign key path as well
as directly. A new de-duplicating set is passed through within the
:meth:`.MetaData.reflect` sequence to prevent double reflection in this
way.
Mike Bayer [Tue, 22 Nov 2016 20:36:32 +0000 (15:36 -0500)]
Ensure Variant passes along impl right-hand type
Fixed issue in :class:`.Variant` where the "right hand coercion" logic,
inherited from :class:`.TypeDecorator`, would
coerce the right-hand side into the :class:`.Variant` itself, rather than
what the default type for the :class:`.Variant` would do. In the
case of :class:`.Variant`, we want the type to act mostly like the base
type so the default logic of :class:`.TypeDecorator` is now overridden
to fall back to the underlying wrapped type's logic. Is mostly relevant
for JSON at the moment.
This patch additionally adds documentation and basic tests to allow
for backend-agnostic comparison of JSON index elements to other objects.
A future version should attempt to improve upon this by providing
"astext", "asint" types of operators.
Mike Bayer [Fri, 18 Nov 2016 16:49:00 +0000 (11:49 -0500)]
Disable single-inheritance critera on the outside of UNION
Fixed bug related to :ticket:`3177`, where a UNION or other set operation
emitted by a :class:`.Query` would apply "single-inheritance" criteria
to the outside of the union (also referencing the wrong selectable),
even though this criteria is now expected to
be already present on the inside subqueries. The single-inheritance
criteria is now omitted once union() or another set operation is
called against :class:`.Query` in the same way as :meth:`.Query.from_self`.
Mike Bayer [Wed, 16 Nov 2016 14:57:36 +0000 (09:57 -0500)]
Port lower case quoted name fix to firebird
Ported the fix for Oracle quoted-lowercase names to Firebird, so that
a table name that is quoted as lower case can be reflected properly
including when the table name comes from the get_table_names()
inspection function.
Also genericize the test to the test suite for denormlized name
dialects.
Mike Bayer [Mon, 14 Nov 2016 21:09:13 +0000 (16:09 -0500)]
Add missing items to collection.__getstate__
the refactor in b606e47ddc54 / ticket:3457 failed
to adjust __getstate__ / __setstate__. need to memoize
a few more things including the class itself so that we
can navigate back to "attr".
Mike Bayer [Sat, 12 Nov 2016 17:34:01 +0000 (12:34 -0500)]
Count columns using PrimaryKeyConstraint.__len__ directly
PrimaryKeyConstraint is present on Table however
on table() and others it's a ColumnSet. The warning
here only needs len() and PrimaryKeyConstraint supports that
directly in the same way as ColumnSet.
Mike Bayer [Thu, 10 Nov 2016 22:08:06 +0000 (17:08 -0500)]
Quote URL tokens with semicolons for pyodbc, adodbapi
Fixed bug in pyodbc dialect (as well as in the mostly non-working
adodbapi dialect) whereby a semicolon present in the password
or username fields could be interpreted as a separator for another
token; the values are now quoted when semicolons are present.
Mike Bayer [Thu, 10 Nov 2016 19:24:48 +0000 (14:24 -0500)]
Use configured props for mapper.attrs, mapper.all_orm_descriptors
Fixed bug where the :attr:`.Mapper.attrs`,
:attr:`.Mapper.all_orm_descriptors` and other derived attributes would
fail to refresh when mapper properties or other ORM constructs were
added to the mapper/class after these accessors were first called.
(also trying different ways to get the changelog to merge cleanly)
with eng.connect() as conn:
result = conn.execute(table.select().limit(1)).fetchone()
if result is None:
for _ in range(1000):
conn.execute(
table.insert(),
[{'a': random.randint(1, 100000),
'b': ''.join(random.choice(string.ascii_letters) for _ in range(100))}
for _ in range(1000)]
)
with eng.connect() as conn:
for row in conn.execution_options(stream_results=True).execute(table.select()):
pass
now uses ~23 MB of memory instead of ~327 MB on CPython 3.5.2 and
PyMySQL 0.7.9.
psycopg2 implementation and execution options (stream_results,
server_side_cursors) are reused.
The method `engine.base.Connection.default_schema_name` is broken since 4b532e2 (this was when `engine.dialect.get_default_schema_name` was
replaced with `engine.dialect.default_schema_name`). The method is
unused and also is not functional for end-user use, so remove it
for now.
Mike Bayer [Tue, 8 Nov 2016 02:41:11 +0000 (21:41 -0500)]
Put include_table=True for DO UPDATE..WHERE, but not ON CONFLICT
Fixed regression caused by the fix in :ticket:`3807` (version 1.1.0)
where we ensured that the tablename was qualified in the WHERE clause
of the DO UPDATE portion of PostgreSQL's ON CONFLICT, however you
*cannot* put the table name in the WHERE clause in the actual ON
CONFLICT itself. This was an incorrect assumption, so that portion
of the change in :ticket:`3807` is rolled back.
Mike Bayer [Sun, 6 Nov 2016 17:46:28 +0000 (12:46 -0500)]
Move setup functionality into _register_attribute
Options like uselist and backref can be determined from
within _register_attribute based on parent_property
given; move this logic inside so that individual strategies
have less responsibility. Also don't require that
_register_attribute consider the "strategy" itself
at all; it would be better if we could no longer require
that Joined/Subquery/etc loaders call upon the "lazy" strategy
in order to initialize attribute instrumentation and
this could be done more generically.
Fixes long-standing bug where the "noload" relationship loading
strategy would cause backrefs and/or back_populates options to be
ignored. There is concern that some application that uses
"noload" might be surprised at a back-populating attribute
appearing suddenly, which may have side effects. However,
"noload" itself must be extremely seldom used since as a
strategy, it already disables loading, population of attributes
is the only behavior that is even supported, so that this issue has existed
for at least through 0.7 four years ago without ever being
reported indicates extremely low use of this option.
Mike Bayer [Fri, 28 Oct 2016 18:13:31 +0000 (14:13 -0400)]
Add check for blank string coming from MySQL's enum
MySQL's native ENUM type supports any non-valid value being sent, and
in response will return a blank string. A hardcoded rule to check for
"is returning the blank string" has been added to the MySQL
implementation for ENUM so that this blank string is returned to the
application rather than being rejected as a non-valid value. Note that
if your MySQL enum is linking values to objects, you still get the
blank string back.
Mike Bayer [Thu, 27 Oct 2016 13:51:50 +0000 (09:51 -0400)]
Restore object to the identity_map upon delete() unconditionally
Fixed regression caused by :ticket:`2677` whereby calling
:meth:`.Session.delete` on an object that was already flushed as
deleted in that session would fail to set up the object in the
identity map (or reject the object), causing flush errors as the
object were in a state not accommodated by the unit of work.
The pre-1.1 behavior in this case has been restored, which is that
the object is put back into the identity map so that the DELETE
statement will be attempted again, which emits a warning that the number
of expected rows was not matched (unless the row were restored outside
of the session).
Mike Bayer [Fri, 21 Oct 2016 13:34:32 +0000 (09:34 -0400)]
Ensure .mapper is set on _ColumnEntity
_ColumnEntity didn't seem to have .mapper present, which
due to the way _mapper_zero() worked didn't tend to come
across it. With :ticket:`3608` _mapper_zero() has
been simplified so make sure this is now present.
Also ensure that _select_from_entity is an entity and
not a mapped class, though this does not seem to matter
at the moment.
Mike Bayer [Thu, 20 Oct 2016 21:36:59 +0000 (17:36 -0400)]
Don't set pg autoincrement if type affinity is not Integer
Postgresql table reflection will ensure that the
:paramref:`.Column.autoincrement` flag is set to False when reflecting
a primary key column that is not of an :class:`.Integer` datatype,
even if the default is related to an integer-generating sequence.
This can happen if a column is created as SERIAL and the datatype
is changed. The autoincrement flag can only be True if the datatype
is of integer affinity in the 1.1 series.
This bug is related to a test failure in downstream sqlalchemy_migrate.
Mike Bayer [Thu, 20 Oct 2016 19:59:46 +0000 (15:59 -0400)]
Convert expression type for concat + Enum
Fixed bug involving new value translation and validation feature
in :class:`.Enum` whereby using the enum object in a string
concatenation would maintain the :class:`.Enum` type as the type
of the expression overall, producing missing lookups. A string
concatenation against an :class:`.Enum`-typed column now uses
:class:`.String` as the datatype of the expression itself.
Ensure TypeDecorator delegates _set_parent_with_dispatch as well as
_set_parent to itself as well as its impl, as the TypeDecorator
class itself may have an active SchemaType implementation as well.
Fixed regression which occurred as a side effect of :ticket:`2919`,
which in the less typical case of a user-defined
:class:`.TypeDecorator` that was also itself an instance of
:class:`.SchemaType` (rather than the implementation being such)
would cause the column attachment events to be skipped for the
type itself.
Mike Bayer [Wed, 19 Oct 2016 16:52:55 +0000 (12:52 -0400)]
Rewrite migration notes for [ticket:3514]
The change to "evaluates none" datatypes in the ORM was
not fully described in the migration notes, missing the
key behavioral change that a column which is missing a default
entirely will not receive a value for a missing JSON column now.
The issue here touched upon a revisit of the assumptions
in [ticket:3514], but overall the old behavior "worked" mostly
because the ORM wants to explicitly render NULL into an INSERT
for column values that are missing, which itself is a legacy
behavior which should be considered for possible removal in
a future major release. Given that "missing ORM value + no
column default set up == dont put it in the INSERT" would be
the most intuitive behavior, the move in [ticket:3514] represents
a step in this direction.
Mike Bayer [Mon, 17 Oct 2016 16:35:26 +0000 (12:35 -0400)]
Remove the use of the word "fundamental" to describe ColumnElements
The context is unclear if "fundamental" refers to the object's
role within a SELECT composition or within the class hierarchy
of ClauseElement classes.
Mike Bayer [Mon, 17 Oct 2016 15:07:14 +0000 (11:07 -0400)]
Add a qualifying character to multi INSERT..VALUES parameter names
Changed the naming convention used when generating bound parameters
for a multi-VALUES insert statement, so that the numbered parameter
names don't conflict with the anonymized parameters of a WHERE clause,
as is now common in a PostgreSQL ON CONFLICT construct.
Mike Bayer [Mon, 17 Oct 2016 04:22:38 +0000 (00:22 -0400)]
Add explicit copy() to Enum
The Boolean and Enum types both place SchemaType second in the
inheritance hierarchy. In the case of Enum, this works
out that the copy() method is called from the base TypeEngine
which fails to transfer _create_events. The test suite
doesn't seem to work with the inhertance hierarchy set up like
this as the event listeners don't work out, the _on_metadata_create
and _on_table_create hooks cause the production of an adapted type
which then adds event listeners that cause deque changed while
iteration. It's not clear why Enum /Boolean don't have this problem.
But in any case it seems like the class mechanics for these types
remains fragile and would benefit from yet another refactor someday.
Mike Bayer [Thu, 13 Oct 2016 16:27:18 +0000 (12:27 -0400)]
Memoize load_path in all cases, run quick populators for path change
Adds a new variant to the "isnew" state within entity loading
for isnew=False, but the load path is new. This is to address
the use case of an entity appearing in multiple places in
the row in a more generalized way than the fixes in [ticket:3431],
[ticket:3811] in that loading.py will be able to tell the
populator that this row is not "isnew" but is a "new" path
for the entity. For the moment, the new information is only
being applied to the use of "quick" populators so that
simple column loads can take place on top of a deferred loader
from elsewhere in the row.
As part of this change, state.load_path() will now always
be populated with the "path" that was in effect when this state
was originally loaded, which for multi-path loads of the
same entity is still non-deterministic. Ideally there'd be some
kind of "here's all the paths that loaded this state and how"
type of data structure though it's not clear if that could be
done while maintaining performance.
Mike Bayer [Fri, 14 Oct 2016 21:06:07 +0000 (17:06 -0400)]
Assemble "don't joinedload other side" rule using query._current_path
Discovered during testing for [ticket:3822], the rule added
for [ticket:1495] will fail if the source object has propagated
options set up, which add elements to query._current_path.
Matt Riedemann [Mon, 10 Oct 2016 15:43:07 +0000 (11:43 -0400)]
Update the Column.nullable docstring for the primary_key=True case
While reviewing a change that created a new table, the
primary_key column value was set to True but nullable was not
explicitly set, which led to some confusion over the default
behavior for the nullable column value when setting a primary_key.
Looking at the docs it's not clear, but the code shows that if
nullable is not specified, then nullable = not primary_key, so
nullable defaults to False when primary_key is True.
This patch adds a simple clarification to the docs so people
don't have to check the code.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
