Mike Bayer [Fri, 22 May 2020 17:14:58 +0000 (13:14 -0400)]
Don't emit pyodbc "no driver" warning for empty URL
Fixed an issue in the pyodbc connector such that a warning about pyodbc
"drivername" would be emitted when using a totally empty URL. Empty URLs
are normal when producing a non-connected dialect object or when using the
"creator" argument to create_engine(). The warning now only emits if the
driver name is missing but other parameters are still present.
Miguel Grinberg [Thu, 21 May 2020 09:54:47 +0000 (05:54 -0400)]
Fix query string escaping in engine URLs
Fixed issue in :class:`.URL` object where stringifying the object
would not URL encode special characters, preventing the URL from being
re-consumable as a real URL. Pull request courtesy Miguel Grinberg.
Mike Bayer [Sat, 16 May 2020 17:05:00 +0000 (13:05 -0400)]
Reword delete-orphan on many error message and document
For many years we have encountered users making use of the
"single_parent" flag in response to the error message for
"delete-orphan" expressing this as a means to cancel the current
error. However, the actual issue here is usually a misuse
of the delete-orphan cascade setting. Reword the error message to
be much more descriptive about what this means and add new
error link sections describing the situation in as much detail
as possible.
Dylan Modesitt [Wed, 6 May 2020 18:17:23 +0000 (14:17 -0400)]
Add 'schema' parameter to table
Added a "schema" parameter to the :func:`_expression.table` construct,
allowing ad-hoc table expressions to also include a schema name.
Pull request courtesy Dylan Modesitt.
Mike Bayer [Thu, 14 May 2020 14:51:29 +0000 (10:51 -0400)]
Actively unset reset agent in discard transaction
The assumptions in _discard_transaction from 916e1fea25afcd07fa1d1d2f72043b372cd02223 were too narrow,
assuming that if the given transaction were not our
"current" one, that this would not be the reset agent. however
as the legacy behvaior is that even a "nested" transaction gets
set as "self._transaction", this did not accommodate for the nested
transaction being thrown away. We will attempt to refine all of this
logic in #5327 for 1.4 /master assuming this is feasible for the
full suite of current use cases.
Mike Bayer [Wed, 13 May 2020 16:42:08 +0000 (12:42 -0400)]
Assert reset agent always set correctly and is active
Fixed fairly critical issue where the DBAPI connection could be returned to
the connection pool while still in an un-rolled-back state. The reset agent
responsible for rolling back the connection could be corrupted in the case
that the transaction was "closed" without being rolled back or committed,
which can occur in some scenarios when using ORM sessions and emitting
.close() in a certain pattern involving savepoints. The fix ensures that
the reset agent is always active.
Note for 1.3 the cherry-pick modifies the approach from master as
transaction handling has diverged.
Mike Bayer [Thu, 7 May 2020 17:56:38 +0000 (13:56 -0400)]
Use cx_oracle.LONG_STRING /LONG_BINARY for CLOB/BLOB
Changed the implementation of fetching CLOB and BLOB objects to use
cx_Oracle's native implementation which fetches CLOB/BLOB objects inline
with other result columns, rather than performing a separate fetch. As
always, this can be disabled by setting auto_convert_lobs to False.
As part of this change, the behavior of a CLOB that was given a blank
string on INSERT now returns None on SELECT, which is now consistent with
that of VARCHAR on Oracle.
Mike Bayer [Fri, 8 May 2020 15:25:14 +0000 (11:25 -0400)]
Expand CTE testing and begin to make this more prominent.
SQLite supports CTEs as do both the MariaDB and MySQL
databases now. Expand test support and begin adding
new tutorial sections to make this concept more prominent.
Also fix some formatting issues in the tutorial, and start
moving more towards {opensql} - the "SQL" button thing
is cool but the tutorials mainly need to illustrate how
the SQL looks for things. Screens are bigger now and
we likely should move away from the SQL button almost
entirely.
CTE for the tutorial needed zzzeeksphinx fix to recognize
"WITH" so bumping that to latest release number in git.
Mike Bayer [Thu, 7 May 2020 21:13:35 +0000 (17:13 -0400)]
Warn when sorted_tables is not actually sorting
A warning is emitted when making use of the :attr:`.MetaData.sorted_tables`
attribute as well as the :func:`_schema.sort_tables` function, and the
given tables cannot be correctly sorted due to a cyclic dependency between
foreign key constraints. In this case, the functions will no longer sort
the involved tables by foreign key, and a warning will be emitted. Other
tables that are not part of the cycle will still be returned in dependency
order. Previously, the sorted_table routines would return a collection that
would unconditionally omit all foreign keys when a cycle was detected, and
no warning was emitted.
Introduce :paramref:`_orm.relationship.sync_backref` flag in a relationship
to control if the synchronization events that mutate the in-Python
attributes are added.
This flag is implied in a ``viewonly=True`` relationship.
This supersedes the previous change #5149, that warned that ``viewonly=True``
relationship target of a back_populates or backref configuration would be
disallowed.
Mike Bayer [Wed, 6 May 2020 13:47:26 +0000 (09:47 -0400)]
Clarify create_engine encoding; update cx_Oracle
The create_engine()->encoding parameter is mostly irrelevant
under Python 3. make it clear this parameter is not generally
useful anymore and refer readers to the dialect documenation.
Update cx_Oracle documentation to feature many examples of
the encoding / nencoding parameters, remove extra detail that
is not generally useful and reorganize information into
more specific sections, de-emphasizing legacy / Python 2
specific sections.
Mike Bayer [Mon, 4 May 2020 19:48:48 +0000 (15:48 -0400)]
Consider aliased=True, from_joinpoint as legacy
For a 1.4 / 1.3 merge, rewrite the documentation for
Query.join() to indicate calling forms that are now considered
legacy, including the use of strings in join(), sending a
series of join paths in one call, and using the aliased=True
flag. update the elementtree examples as well to use aliased()
(they are much simpler to understand this way too) and update
other links.
Also improve docs for aliased() and some other ORM targets
such as PropComparator.
Mike Bayer [Sun, 3 May 2020 23:35:54 +0000 (19:35 -0400)]
Baked query needs to spoil fully on uncachable option
Fixed issue in the area of where loader options such as selectinload()
interact with the baked query system, such that the caching of a query is
not supposed to occur if the loader options themselves have elements such
as with_polymorphic() objects in them that currently are not
cache-compatible. The baked loader could sometimes not fully invalidate
itself in these some of these scenarios leading to missed eager loads.
Mike Bayer [Mon, 4 May 2020 00:27:24 +0000 (20:27 -0400)]
Don't apply sets or similar to objects in IdentitySet
Modified the internal "identity set" implementation, which is a set that
hashes objects on their id() rather than their hash values, to not actually
call the ``__hash__()`` method of the objects, which are typically
user-mapped objects. Some methods were calling this method as a side
effect of the implementation.
Mike Bayer [Sat, 2 May 2020 00:30:18 +0000 (20:30 -0400)]
Handle non-string, non column expression in index for deferred attach
Fixed issue where an :class:`.Index` that is deferred in being associated
with a table, such as as when it contains a :class:`.Column` that is not
associated with any :class:`.Table` yet, would fail to attach correctly if
it also contained a non table-oriented expession.
Mike Bayer [Mon, 27 Apr 2020 20:51:43 +0000 (16:51 -0400)]
Alias the onclause if ORM join is to same polymorphic selectable
Fixed bug where using :func:`.with_polymorphic` as the target of a join via
:meth:`.RelationshipComparator.of_type` on a mapper that already has a
subquery-based with_polymorphic setting that's equivalent to the one
requested would not correctly alias the ON clause in the join.
Fix a regression introduced by the reflection of computed column in
MSSQL when using SQL server versions before 2012, which does not support
the ``concat`` function and when using the legacy TDS version 4.2.
The dialect will try to detect the protocol version of first connect
and run in compatibility mode if it cannot detect it.
Mike Bayer [Mon, 20 Apr 2020 16:24:40 +0000 (12:24 -0400)]
Add ColumnProperty.Comparator.expressions
Added an accessor :attr:`.ColumnProperty.Comparator.expressions` which
provides access to the group of columns mapped under a multi-column
:class:`.ColumnProperty` attribute.
Added support for columns or type :class:`.ARRAY` of :class:`.Enum`,
:class:`.JSON` or :class:`_postgresql.JSONB` in PostgreSQL.
Previously a workaround was required in these use cases.
Raise an explicit :class:`.exc.CompileError` when adding a table with a
column of type :class:`.ARRAY` of :class:`.Enum` configured with
:paramref:`.Enum.native_enum` set to ``False`` when
:paramref:`.Enum.create_constraint` is not set to ``False``
Mike Bayer [Sun, 19 Apr 2020 20:52:54 +0000 (16:52 -0400)]
Raise informative error when non-object m2o comparison used
An informative error message is raised when an ORM many-to-one comparison
is attempted against an object that is not an actual mapped instance.
Comparisons such as those to scalar subqueries aren't supported;
generalized comparison with subqueries is better achieved using
:meth:`~.RelationshipProperty.Comparator.has`.
Mike Bayer [Tue, 14 Apr 2020 19:30:28 +0000 (15:30 -0400)]
Pass connection to TablesTest.insert_data()
towards the goal of reducing verbosity and repetition
in test fixtures as well as that we are moving to
connection only for execution, move the insert_data()
classmethod to accept a connection and adjust all
fixtures to use it.
Mike Bayer [Sun, 12 Apr 2020 19:18:02 +0000 (15:18 -0400)]
Enable zzzeeksphinx module prefixes
zzzeeksphinx 1.1.2 in git can now convert short
prefix names in a configured lookup to fully qualified module
names, so that
we can have succinct and portable pyrefs
that still resolve absolutely.
It also includes a formatter that will format all pyrefs
in a fully consistent way regardless of the package path,
by unconditionally removing all package tokens but always
leaving class names in place including for methods, which
means we no longer have to deal with tildes in pyrefs.
The most immediate goal of the absolute prefixes is
that we have lots of
"ambiguous" names that appear in muliple places, like select(),
ARRAY, ENUM etc. With the incoming future packages there
is going to be lots of name overlap so it is necessary
that all names eventually use absolute package paths
when Sphinx receives them.
In multiple stages, pyrefs will be converted using the
zzzeeksphinx tools/fix_xrefs.py tool so that doclinks can
be made absolute using symbolic prefixes.
For this review, the actual search and replace of symbols
is not performed, instead some general cleanup to prepare
the docs as well as a lookup file used by the tool
to do the conversion. this relatively small patch will
be backported
with appropriate changes to 1.3, 1.2, 1.1 and the tool
can then be run on each branch individually. We are shooting
for almost no warnings at all for master (still a handful
I can't figure out which don't seem to have any impact)
, very few for 1.3,
and for 1.2 / 1.1 we hope for a significant reduction
in warnings.
Overall for all versions pyrefs should
always point to the correct target, if they are in fact
hyperlinked. it's better for a ref to go nowhere and
be plain text than go to the wrong thing. Right now,
hundreds of API links are pointing to the wrong thing
as they are ambiguous names such as refresh(), insert(),
update(), select(), join(), JSON etc. and Sphinx sends these all
to essesntially random destinations among as many as five
or six possible choices per symbol. A shorthand system
that allows us to use absolute refs without having
to type out a full blown absoulte module is the only
way this is going to work, and we should ultimately
seek to abandon any use of prefix dot for lookups. Everything
should be on an underscore token so at the very least the module
spaces can be reorganized without having to search and replace
the entire documentation every time.
Mike Bayer [Mon, 13 Apr 2020 22:39:47 +0000 (18:39 -0400)]
Adjust changelog files for removed directories
unreleased_11 and unreleased_12 are gone so remove these
also there's no place for the 4065 file to go, this is an old
and very small change and it looks like the file was not published in any
case, so just remove it.
Mike Bayer [Fri, 10 Apr 2020 20:58:41 +0000 (16:58 -0400)]
Use define_tables for IsOrIsNotDistinctFromTest
In Ie8127ef29f1ec91e7afb88e1429538c27a321784 Mike
failed to notice that this test is build on TablesTest and
has just the one test function, which means the table is
built up by the fixture.
Mike Bayer [Fri, 10 Apr 2020 16:23:32 +0000 (12:23 -0400)]
Evaluate fixes for cx_Oracle 8 API changes
in [1] we have reported some behavioral changes in cx_Oracle
symbols. As it is unclear if these are intended changes in
cx_Oracle, test workarounds for these changes against
cx_Oracle master.
Mike Bayer [Fri, 10 Apr 2020 18:28:54 +0000 (14:28 -0400)]
Repair CTE a in b tests
In I27cac9bd265c86ff2a3381ff9f844f60ef991cfc we modernized
the default tests and converted the "a in b" CTE tests to combinations,
however apparently the existing tests were not testing all
combinations and had repeats instead. The combinations
decorator has made this much easier to spot, so use
the correct combinations that were originally intended.
Add github action workflow to run tests on master and on pr to master
Trying the pr to check if it works right away
<!-- Describe your changes in detail -->
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 7 Apr 2020 21:37:14 +0000 (17:37 -0400)]
Use dot-separated name resolution for relationship target
The string argument accepted as the first positional argument by the
:func:`.relationship` function when using the Declarative API is no longer
interpreted using the Python ``eval()`` function; instead, the name is dot
separated and the names are looked up directly in the name resolution
dictionary without treating the value as a Python expression. However,
passing a string argument to the other :func:`.relationship` parameters
that necessarily must accept Python expressions will still use ``eval()``;
the documentation has been clarified to ensure that there is no ambiguity
that this is in use.
Mike Bayer [Tue, 7 Apr 2020 23:21:01 +0000 (19:21 -0400)]
Gracefully skip isolation level if no row returned
Fixed issue in MySQL dialect when connecting to a psuedo-MySQL database
such as that provided by ProxySQL, the up front check for isolation level
when it returns no row will not prevent the dialect from continuing to
connect. A warning is emitted that the isolation level could not be
detected.
Gord Thompson [Thu, 13 Feb 2020 19:14:42 +0000 (12:14 -0700)]
Replace engine.execute w/ context manager (step1)
First (baby) step at replacing engine.execute
calls in test code with the new preferred way
of executing. MSSQL was targeted because it was
the easiest for me to test locally.
Federico Caselli [Tue, 24 Mar 2020 20:06:04 +0000 (21:06 +0100)]
Add length parameter in `Enum`
The `Enum` type now supports the parameter `Enum.length`
to specify the length of the VARCHAR column to create when using
non native enums by setting `Enum.native_enum` to `False`
Mike Bayer [Fri, 3 Apr 2020 20:00:22 +0000 (16:00 -0400)]
Key subqueryloaders on the property object, not string key
Fixed bug in :func:`.orm.selectinload` loading option where two or more
loaders that represent different relationships with the same string key
name as referenced from a single :func:`.orm.with_polymorphic` construct
with multiple subclass mappers would fail to invoke each subqueryload
separately, instead making use of a single string-based slot that would
prevent the other loaders from being invoked.
Mike Bayer [Mon, 30 Mar 2020 15:04:24 +0000 (11:04 -0400)]
Remove ORDER BY pk from subqueryload, selectinload
Modified the queries used by subqueryload and selectinload to no longer
ORDER BY the primary key of the parent entity; this ordering was there to
allow the rows as they come in to be copied into lists directly with a
minimal level of Python-side collation. However, these ORDER BY clauses
can negatively impact the performance of the query as in many scenarios
these columns are derived from a subquery or are otherwise not actual
primary key columns such that SQL planners cannot make use of indexes. The
Python-side collation uses the native itertools.group_by() to collate the
incoming rows, and has been modified to allow multiple
row-groups-per-parent to be assembled together using list.extend(), which
should still allow for relatively fast Python-side performance. There will
still be an ORDER BY present for a relationship that includes an explicit
order_by parameter, however this is the only ORDER BY that will be added to
the query for both kinds of loading.
Federico Caselli [Tue, 24 Mar 2020 21:55:46 +0000 (22:55 +0100)]
String compiler can now literal compile datetime objects
Add ability to literal compile a :class:`DateTime`, :class:`Date`
or :class:"Time" when using the string dialect for debugging purposes.
This change does not impact real dialect implementation that retain
their current behavior.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
