with eng.connect() as conn:
result = conn.execute(table.select().limit(1)).fetchone()
if result is None:
for _ in range(1000):
conn.execute(
table.insert(),
[{'a': random.randint(1, 100000),
'b': ''.join(random.choice(string.ascii_letters) for _ in range(100))}
for _ in range(1000)]
)
with eng.connect() as conn:
for row in conn.execution_options(stream_results=True).execute(table.select()):
pass
now uses ~23 MB of memory instead of ~327 MB on CPython 3.5.2 and
PyMySQL 0.7.9.
psycopg2 implementation and execution options (stream_results,
server_side_cursors) are reused.
The method `engine.base.Connection.default_schema_name` is broken since 4b532e2 (this was when `engine.dialect.get_default_schema_name` was
replaced with `engine.dialect.default_schema_name`). The method is
unused and also is not functional for end-user use, so remove it
for now.
Mike Bayer [Tue, 8 Nov 2016 02:41:11 +0000 (21:41 -0500)]
Put include_table=True for DO UPDATE..WHERE, but not ON CONFLICT
Fixed regression caused by the fix in :ticket:`3807` (version 1.1.0)
where we ensured that the tablename was qualified in the WHERE clause
of the DO UPDATE portion of PostgreSQL's ON CONFLICT, however you
*cannot* put the table name in the WHERE clause in the actual ON
CONFLICT itself. This was an incorrect assumption, so that portion
of the change in :ticket:`3807` is rolled back.
Mike Bayer [Sun, 6 Nov 2016 17:46:28 +0000 (12:46 -0500)]
Move setup functionality into _register_attribute
Options like uselist and backref can be determined from
within _register_attribute based on parent_property
given; move this logic inside so that individual strategies
have less responsibility. Also don't require that
_register_attribute consider the "strategy" itself
at all; it would be better if we could no longer require
that Joined/Subquery/etc loaders call upon the "lazy" strategy
in order to initialize attribute instrumentation and
this could be done more generically.
Fixes long-standing bug where the "noload" relationship loading
strategy would cause backrefs and/or back_populates options to be
ignored. There is concern that some application that uses
"noload" might be surprised at a back-populating attribute
appearing suddenly, which may have side effects. However,
"noload" itself must be extremely seldom used since as a
strategy, it already disables loading, population of attributes
is the only behavior that is even supported, so that this issue has existed
for at least through 0.7 four years ago without ever being
reported indicates extremely low use of this option.
Mike Bayer [Fri, 28 Oct 2016 18:13:31 +0000 (14:13 -0400)]
Add check for blank string coming from MySQL's enum
MySQL's native ENUM type supports any non-valid value being sent, and
in response will return a blank string. A hardcoded rule to check for
"is returning the blank string" has been added to the MySQL
implementation for ENUM so that this blank string is returned to the
application rather than being rejected as a non-valid value. Note that
if your MySQL enum is linking values to objects, you still get the
blank string back.
Mike Bayer [Thu, 27 Oct 2016 13:51:50 +0000 (09:51 -0400)]
Restore object to the identity_map upon delete() unconditionally
Fixed regression caused by :ticket:`2677` whereby calling
:meth:`.Session.delete` on an object that was already flushed as
deleted in that session would fail to set up the object in the
identity map (or reject the object), causing flush errors as the
object were in a state not accommodated by the unit of work.
The pre-1.1 behavior in this case has been restored, which is that
the object is put back into the identity map so that the DELETE
statement will be attempted again, which emits a warning that the number
of expected rows was not matched (unless the row were restored outside
of the session).
Mike Bayer [Fri, 21 Oct 2016 13:34:32 +0000 (09:34 -0400)]
Ensure .mapper is set on _ColumnEntity
_ColumnEntity didn't seem to have .mapper present, which
due to the way _mapper_zero() worked didn't tend to come
across it. With :ticket:`3608` _mapper_zero() has
been simplified so make sure this is now present.
Also ensure that _select_from_entity is an entity and
not a mapped class, though this does not seem to matter
at the moment.
Mike Bayer [Thu, 20 Oct 2016 21:36:59 +0000 (17:36 -0400)]
Don't set pg autoincrement if type affinity is not Integer
Postgresql table reflection will ensure that the
:paramref:`.Column.autoincrement` flag is set to False when reflecting
a primary key column that is not of an :class:`.Integer` datatype,
even if the default is related to an integer-generating sequence.
This can happen if a column is created as SERIAL and the datatype
is changed. The autoincrement flag can only be True if the datatype
is of integer affinity in the 1.1 series.
This bug is related to a test failure in downstream sqlalchemy_migrate.
Mike Bayer [Thu, 20 Oct 2016 19:59:46 +0000 (15:59 -0400)]
Convert expression type for concat + Enum
Fixed bug involving new value translation and validation feature
in :class:`.Enum` whereby using the enum object in a string
concatenation would maintain the :class:`.Enum` type as the type
of the expression overall, producing missing lookups. A string
concatenation against an :class:`.Enum`-typed column now uses
:class:`.String` as the datatype of the expression itself.
Ensure TypeDecorator delegates _set_parent_with_dispatch as well as
_set_parent to itself as well as its impl, as the TypeDecorator
class itself may have an active SchemaType implementation as well.
Fixed regression which occurred as a side effect of :ticket:`2919`,
which in the less typical case of a user-defined
:class:`.TypeDecorator` that was also itself an instance of
:class:`.SchemaType` (rather than the implementation being such)
would cause the column attachment events to be skipped for the
type itself.
Mike Bayer [Wed, 19 Oct 2016 16:52:55 +0000 (12:52 -0400)]
Rewrite migration notes for [ticket:3514]
The change to "evaluates none" datatypes in the ORM was
not fully described in the migration notes, missing the
key behavioral change that a column which is missing a default
entirely will not receive a value for a missing JSON column now.
The issue here touched upon a revisit of the assumptions
in [ticket:3514], but overall the old behavior "worked" mostly
because the ORM wants to explicitly render NULL into an INSERT
for column values that are missing, which itself is a legacy
behavior which should be considered for possible removal in
a future major release. Given that "missing ORM value + no
column default set up == dont put it in the INSERT" would be
the most intuitive behavior, the move in [ticket:3514] represents
a step in this direction.
Mike Bayer [Mon, 17 Oct 2016 16:35:26 +0000 (12:35 -0400)]
Remove the use of the word "fundamental" to describe ColumnElements
The context is unclear if "fundamental" refers to the object's
role within a SELECT composition or within the class hierarchy
of ClauseElement classes.
Mike Bayer [Mon, 17 Oct 2016 15:07:14 +0000 (11:07 -0400)]
Add a qualifying character to multi INSERT..VALUES parameter names
Changed the naming convention used when generating bound parameters
for a multi-VALUES insert statement, so that the numbered parameter
names don't conflict with the anonymized parameters of a WHERE clause,
as is now common in a PostgreSQL ON CONFLICT construct.
Mike Bayer [Mon, 17 Oct 2016 04:22:38 +0000 (00:22 -0400)]
Add explicit copy() to Enum
The Boolean and Enum types both place SchemaType second in the
inheritance hierarchy. In the case of Enum, this works
out that the copy() method is called from the base TypeEngine
which fails to transfer _create_events. The test suite
doesn't seem to work with the inhertance hierarchy set up like
this as the event listeners don't work out, the _on_metadata_create
and _on_table_create hooks cause the production of an adapted type
which then adds event listeners that cause deque changed while
iteration. It's not clear why Enum /Boolean don't have this problem.
But in any case it seems like the class mechanics for these types
remains fragile and would benefit from yet another refactor someday.
Mike Bayer [Thu, 13 Oct 2016 16:27:18 +0000 (12:27 -0400)]
Memoize load_path in all cases, run quick populators for path change
Adds a new variant to the "isnew" state within entity loading
for isnew=False, but the load path is new. This is to address
the use case of an entity appearing in multiple places in
the row in a more generalized way than the fixes in [ticket:3431],
[ticket:3811] in that loading.py will be able to tell the
populator that this row is not "isnew" but is a "new" path
for the entity. For the moment, the new information is only
being applied to the use of "quick" populators so that
simple column loads can take place on top of a deferred loader
from elsewhere in the row.
As part of this change, state.load_path() will now always
be populated with the "path" that was in effect when this state
was originally loaded, which for multi-path loads of the
same entity is still non-deterministic. Ideally there'd be some
kind of "here's all the paths that loaded this state and how"
type of data structure though it's not clear if that could be
done while maintaining performance.
Mike Bayer [Fri, 14 Oct 2016 21:06:07 +0000 (17:06 -0400)]
Assemble "don't joinedload other side" rule using query._current_path
Discovered during testing for [ticket:3822], the rule added
for [ticket:1495] will fail if the source object has propagated
options set up, which add elements to query._current_path.
Matt Riedemann [Mon, 10 Oct 2016 15:43:07 +0000 (11:43 -0400)]
Update the Column.nullable docstring for the primary_key=True case
While reviewing a change that created a new table, the
primary_key column value was set to True but nullable was not
explicitly set, which led to some confusion over the default
behavior for the nullable column value when setting a primary_key.
Looking at the docs it's not clear, but the code shows that if
nullable is not specified, then nullable = not primary_key, so
nullable defaults to False when primary_key is True.
This patch adds a simple clarification to the docs so people
don't have to check the code.
Mike Bayer [Wed, 5 Oct 2016 14:57:30 +0000 (10:57 -0400)]
Check for __clause_element__() in ORM insert/update
ORM attributes can now be assigned any object that is has a
``__clause_element__()`` attribute, which will result in inline
SQL the way any :class:`.ClauseElement` class does. This covers other
mapped attributes not otherwise transformed by further expression
constructs.
As part of this, it was considered that we could add
__clause_element__() to ClauseElement, however this causes endless loops
in a "while" pattern and this pattern has been identified in third
party libraries. Add a test to ensure we never make that change.
Mike Bayer [Tue, 4 Oct 2016 20:46:08 +0000 (16:46 -0400)]
Check row for None with implicit returning PK to accommodate ON CONFLICT
An adjustment to ON CONFLICT such that the "inserted_primary_key"
logic is able to accommodate the case where there's no INSERT or
UPDATE and there's no net change. The value comes out as None
in this case, rather than failing on an exception.
Mike Bayer [Mon, 3 Oct 2016 19:55:04 +0000 (15:55 -0400)]
Enhance "raise" strategy to include "raise_on_sql" option
The "raise_on_sql" option differentiates from "raise" in that
firing a lazy loader is OK as long as it does a simple
get from identity map. Whereas "raise" is more useful
for the case that objects are to be detached.
As part of this, refactors the strategy initiation logic
a bit so that a LoaderStrategy itself knows what "key" was used
to create it, thus allowing variants of a single strategy
based on what the "lazy" argument is. To achieve this we
have to also get rid of _get_strategy_by_cls().
Everything here is internal with the one exception of an apparently
undocumented, but not underscored, "strategy_class" key
on relationship(). Though it's not clear what
"strategy_class" accomplishes; at this point the strategy
system is extensible using Property.strategy_for().
Mike Bayer [Mon, 3 Oct 2016 20:55:54 +0000 (16:55 -0400)]
Honor additional row coming in with value of None
The change in #3431 still checks that the instance() is
non-None, deferring to other loading schemes if it is.
These columns are dedicated towards the entity however, so if the value
is None, we should set it. If it conflicts, we are detecting that
in any case.
Mike Bayer [Tue, 4 Oct 2016 16:01:12 +0000 (12:01 -0400)]
- add a note to baked documentation indicating it is not
really of general use. This extension is there only
for those who really want it based on observed performance
characteristics.
Mike Bayer [Tue, 4 Oct 2016 15:32:20 +0000 (11:32 -0400)]
Use schema_name() for SQL Server default schema
Changed the query used to get "default schema name", from one that
queries the database principals table to using the
"schema_name()" function, as issues have been reported that the
former system was unavailable on the Azure Data Warehouse edition.
It is hoped that this will finally work across all SQL Server
versions and authentication styles.
Mike Bayer [Tue, 4 Oct 2016 15:17:26 +0000 (11:17 -0400)]
Use SQL Server SERVERPROPERTY for version info w/ pyodbc
Updated the server version info scheme for pyodbc to use SQL Server
SERVERPROPERTY(), rather than relying upon pyodbc.SQL_DBMS_VER, which
continues to be unreliable particularly with FreeTDS.
Mike Bayer [Mon, 3 Oct 2016 16:25:42 +0000 (12:25 -0400)]
Ensure strong ref to obj before calling persistent_to_deleted, others
Add checks in spots where state.obj() might be late-GC'ed before
we get a chance to call the event. There may be more cases
of these which we should address as they come up. The Session
should always be maintaining strong refs to objects that have
pending operations left on them, so for these cases we need
to ensure that ref remains long enough for the event to be called.
Mike Bayer [Fri, 30 Sep 2016 14:09:56 +0000 (10:09 -0400)]
Add "eager_parenthesis" late-compilation rule, use w/ PG JSON/HSTORE
Added compiler-level flags used by Postgresql to place additional
parenthesis than would normally be generated by precedence rules
around operations involving JSON, HSTORE indexing operators as well as
within their operands since it has been observed that Postgresql's
precedence rules for at least the HSTORE indexing operator is not
consistent between 9.4 and 9.5.
Mike Bayer [Wed, 28 Sep 2016 22:01:57 +0000 (18:01 -0400)]
Enable include_table for ON CONFLICT whereclauses
Fixed issue in new PG "on conflict" construct where columns including
those of the "excluded" namespace would not be table-qualified
in the WHERE clauses in the statement.
Mike Bayer [Sat, 24 Sep 2016 04:49:22 +0000 (00:49 -0400)]
- improve documentation for SessionTransaction re: parent
and nested attributes and what these mean
- improve linking for after_transaction_create() / after_transaction_end()
events
- add public .parent attribute to detect top-level transaction within
these events
Mike Bayer [Thu, 22 Sep 2016 15:08:09 +0000 (11:08 -0400)]
- clarify documentation on timezone flag, since Oracle has both
DATE / TIMESTAMP separately the timezone flag will not bump the
type to TIMESTAMP WITH TIMEZONE on that backend.
Mike Bayer [Wed, 21 Sep 2016 19:37:20 +0000 (15:37 -0400)]
Handle BaseException in all _handle_dbapi_error
Tests illustrate that exceptions like GreenletExit and
even KeyboardInterrupt can corrupt the state of a DBAPI
connection like that of pymysql and mysqlclient. Intercept
BaseException errors within the handle_error scheme and
invalidate just the connection alone in this case, but not
the whole pool.
The change is backwards-incompatible with a program that
currently intercepts ctrl-C within a database transaction
and wants to continue working on that transaction. Ensure
the event hook can be used to reverse this behavior.
Mike Bayer [Wed, 21 Sep 2016 21:55:39 +0000 (17:55 -0400)]
Ensure mapper.polymorphic_on is polymorphic_prop.columns[0]
Fixed bug where joined eager loading would fail for a polymorphically-
loaded mapper, where the polymorphic_on was set to an un-mapped
expression such as a CASE expression.
John Perkins [Fri, 5 Aug 2016 17:45:49 +0000 (13:45 -0400)]
Add the "triggering mapper" to the configure_mappers error message.
There are cases where the originating mapper name is not
present in the exception message, such as relationship initialization
against an unmapped class. Ensure the originating mapper is named
in the string output.
Mike Bayer [Tue, 20 Sep 2016 15:33:16 +0000 (11:33 -0400)]
Allow SQL expressions to be set on PK columns
Removes an unnecessary transfer of modified PK column
value to the params dictionary, so that if the modified PK column
is already present in value_params, this remains in effect. Also
propagate a new flag through to _emit_update_statements() that will
trip "return_defaults()" across the board if a PK col w/ SQL expression
change is present, and pull this PK value in _postfetch as well assuming
we're an UPDATE.
John Passaro [Mon, 19 Sep 2016 19:43:46 +0000 (15:43 -0400)]
Exclude eq and ne from associative operators
The "eq" and "ne" operators are no longer part of the list of
"associative" operators, while they remain considered to be
"commutative". This allows an expression like ``(x == y) == z``
to be maintained at the SQL level with parenthesis. Pull request
courtesy John Passaro.
Mike Bayer [Mon, 19 Sep 2016 20:22:08 +0000 (16:22 -0400)]
Support bindparam() with callable for primaryjoin
Fixes the comparison of bindparam() objects based on
the "callable" parameter being present which helps to correctly
detect use_get, and also checks for "callable" when detecting
parameters for value substitution and will not impact the
object if present.
Mike Bayer [Fri, 2 Sep 2016 19:10:32 +0000 (15:10 -0400)]
Additions to support HAAlchemy plugin
- add a connect=True key to connection record to support
pre-loading of _ConnectionRecord objects
- ensure _ConnectionRecord.close() leaves the record in a good
state for reopening
- add _ConnectionRecord.record_info for persistent storage
- add "in_use" accessor based on fairy_ref being present or not
- allow for the exclusions system and SuiteRequirements to be
usable without the full plugin_base setup.
- move some Python-env requirements to the importable
requirements.py module.
- allow starttime to be queried
- add additional events for engine plugins
- have "dialect" be a first-class parameter to the pool,
ensure the engine strategy supplies it up front
Mike Bayer [Thu, 15 Sep 2016 04:50:17 +0000 (00:50 -0400)]
Repair foreign_keys population for Join._refresh_for_new_column
Fixed bug where setting up a single-table inh subclass of a joined-table
subclass which included an extra column would corrupt the foreign keys
collection of the mapped table, thereby interfering with the
initialization of relationships.
Mike Bayer [Fri, 2 Sep 2016 15:27:58 +0000 (11:27 -0400)]
Repair clauselist comparison to account for clause ordering
Fixed bug where the "simple many-to-one" condition that allows lazy
loading to use get() from identity map would fail to be invoked if the
primaryjoin of the relationship had multiple clauses separated by AND
which were not in the same order as that of the primary key columns
being compared in each clause. This ordering
difference occurs for a composite foreign key where the table-bound
columns on the referencing side were not in the same order in the .c
collection as the primary key columns on the referenced side....which
in turn occurs a lot if one is using declarative mixins and/or
declared_attr to set up columns.
Mike Bayer [Fri, 2 Sep 2016 15:48:15 +0000 (11:48 -0400)]
Allow stringify compiler to render unnamed column
Stringify of expression with unnamed :class:`.Column` objects, as
occurs in lots of situations including ORM error reporting,
will now render the name in string context as "<name unknown>"
rather than raising a compile error.
Mike Bayer [Wed, 31 Aug 2016 18:34:54 +0000 (14:34 -0400)]
Check for supports_execution at ClauseElement base
Raise a more descriptive exception / message when ClauseElement
or non-SQLAlchemy objects that are not "executable" are erroneously
passed to ``.execute()``; a new exception ObjectNotExecutableError
is raised consistently in all cases.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
