Mike Bayer [Thu, 7 May 2020 17:56:38 +0000 (13:56 -0400)]
Use cx_oracle.LONG_STRING /LONG_BINARY for CLOB/BLOB
Changed the implementation of fetching CLOB and BLOB objects to use
cx_Oracle's native implementation which fetches CLOB/BLOB objects inline
with other result columns, rather than performing a separate fetch. As
always, this can be disabled by setting auto_convert_lobs to False.
As part of this change, the behavior of a CLOB that was given a blank
string on INSERT now returns None on SELECT, which is now consistent with
that of VARCHAR on Oracle.
Mike Bayer [Fri, 8 May 2020 15:25:14 +0000 (11:25 -0400)]
Expand CTE testing and begin to make this more prominent.
SQLite supports CTEs as do both the MariaDB and MySQL
databases now. Expand test support and begin adding
new tutorial sections to make this concept more prominent.
Also fix some formatting issues in the tutorial, and start
moving more towards {opensql} - the "SQL" button thing
is cool but the tutorials mainly need to illustrate how
the SQL looks for things. Screens are bigger now and
we likely should move away from the SQL button almost
entirely.
CTE for the tutorial needed zzzeeksphinx fix to recognize
"WITH" so bumping that to latest release number in git.
Mike Bayer [Thu, 7 May 2020 21:13:35 +0000 (17:13 -0400)]
Warn when sorted_tables is not actually sorting
A warning is emitted when making use of the :attr:`.MetaData.sorted_tables`
attribute as well as the :func:`_schema.sort_tables` function, and the
given tables cannot be correctly sorted due to a cyclic dependency between
foreign key constraints. In this case, the functions will no longer sort
the involved tables by foreign key, and a warning will be emitted. Other
tables that are not part of the cycle will still be returned in dependency
order. Previously, the sorted_table routines would return a collection that
would unconditionally omit all foreign keys when a cycle was detected, and
no warning was emitted.
Introduce :paramref:`_orm.relationship.sync_backref` flag in a relationship
to control if the synchronization events that mutate the in-Python
attributes are added.
This flag is implied in a ``viewonly=True`` relationship.
This supersedes the previous change #5149, that warned that ``viewonly=True``
relationship target of a back_populates or backref configuration would be
disallowed.
Mike Bayer [Wed, 6 May 2020 13:47:26 +0000 (09:47 -0400)]
Clarify create_engine encoding; update cx_Oracle
The create_engine()->encoding parameter is mostly irrelevant
under Python 3. make it clear this parameter is not generally
useful anymore and refer readers to the dialect documenation.
Update cx_Oracle documentation to feature many examples of
the encoding / nencoding parameters, remove extra detail that
is not generally useful and reorganize information into
more specific sections, de-emphasizing legacy / Python 2
specific sections.
Mike Bayer [Mon, 4 May 2020 19:48:48 +0000 (15:48 -0400)]
Consider aliased=True, from_joinpoint as legacy
For a 1.4 / 1.3 merge, rewrite the documentation for
Query.join() to indicate calling forms that are now considered
legacy, including the use of strings in join(), sending a
series of join paths in one call, and using the aliased=True
flag. update the elementtree examples as well to use aliased()
(they are much simpler to understand this way too) and update
other links.
Also improve docs for aliased() and some other ORM targets
such as PropComparator.
Mike Bayer [Sun, 3 May 2020 23:35:54 +0000 (19:35 -0400)]
Baked query needs to spoil fully on uncachable option
Fixed issue in the area of where loader options such as selectinload()
interact with the baked query system, such that the caching of a query is
not supposed to occur if the loader options themselves have elements such
as with_polymorphic() objects in them that currently are not
cache-compatible. The baked loader could sometimes not fully invalidate
itself in these some of these scenarios leading to missed eager loads.
Mike Bayer [Mon, 4 May 2020 00:27:24 +0000 (20:27 -0400)]
Don't apply sets or similar to objects in IdentitySet
Modified the internal "identity set" implementation, which is a set that
hashes objects on their id() rather than their hash values, to not actually
call the ``__hash__()`` method of the objects, which are typically
user-mapped objects. Some methods were calling this method as a side
effect of the implementation.
Mike Bayer [Sat, 2 May 2020 00:30:18 +0000 (20:30 -0400)]
Handle non-string, non column expression in index for deferred attach
Fixed issue where an :class:`.Index` that is deferred in being associated
with a table, such as as when it contains a :class:`.Column` that is not
associated with any :class:`.Table` yet, would fail to attach correctly if
it also contained a non table-oriented expession.
Mike Bayer [Mon, 27 Apr 2020 20:51:43 +0000 (16:51 -0400)]
Alias the onclause if ORM join is to same polymorphic selectable
Fixed bug where using :func:`.with_polymorphic` as the target of a join via
:meth:`.RelationshipComparator.of_type` on a mapper that already has a
subquery-based with_polymorphic setting that's equivalent to the one
requested would not correctly alias the ON clause in the join.
Fix a regression introduced by the reflection of computed column in
MSSQL when using SQL server versions before 2012, which does not support
the ``concat`` function and when using the legacy TDS version 4.2.
The dialect will try to detect the protocol version of first connect
and run in compatibility mode if it cannot detect it.
Mike Bayer [Mon, 20 Apr 2020 16:24:40 +0000 (12:24 -0400)]
Add ColumnProperty.Comparator.expressions
Added an accessor :attr:`.ColumnProperty.Comparator.expressions` which
provides access to the group of columns mapped under a multi-column
:class:`.ColumnProperty` attribute.
Added support for columns or type :class:`.ARRAY` of :class:`.Enum`,
:class:`.JSON` or :class:`_postgresql.JSONB` in PostgreSQL.
Previously a workaround was required in these use cases.
Raise an explicit :class:`.exc.CompileError` when adding a table with a
column of type :class:`.ARRAY` of :class:`.Enum` configured with
:paramref:`.Enum.native_enum` set to ``False`` when
:paramref:`.Enum.create_constraint` is not set to ``False``
Mike Bayer [Sun, 19 Apr 2020 20:52:54 +0000 (16:52 -0400)]
Raise informative error when non-object m2o comparison used
An informative error message is raised when an ORM many-to-one comparison
is attempted against an object that is not an actual mapped instance.
Comparisons such as those to scalar subqueries aren't supported;
generalized comparison with subqueries is better achieved using
:meth:`~.RelationshipProperty.Comparator.has`.
Mike Bayer [Tue, 14 Apr 2020 19:30:28 +0000 (15:30 -0400)]
Pass connection to TablesTest.insert_data()
towards the goal of reducing verbosity and repetition
in test fixtures as well as that we are moving to
connection only for execution, move the insert_data()
classmethod to accept a connection and adjust all
fixtures to use it.
Mike Bayer [Sun, 12 Apr 2020 19:18:02 +0000 (15:18 -0400)]
Enable zzzeeksphinx module prefixes
zzzeeksphinx 1.1.2 in git can now convert short
prefix names in a configured lookup to fully qualified module
names, so that
we can have succinct and portable pyrefs
that still resolve absolutely.
It also includes a formatter that will format all pyrefs
in a fully consistent way regardless of the package path,
by unconditionally removing all package tokens but always
leaving class names in place including for methods, which
means we no longer have to deal with tildes in pyrefs.
The most immediate goal of the absolute prefixes is
that we have lots of
"ambiguous" names that appear in muliple places, like select(),
ARRAY, ENUM etc. With the incoming future packages there
is going to be lots of name overlap so it is necessary
that all names eventually use absolute package paths
when Sphinx receives them.
In multiple stages, pyrefs will be converted using the
zzzeeksphinx tools/fix_xrefs.py tool so that doclinks can
be made absolute using symbolic prefixes.
For this review, the actual search and replace of symbols
is not performed, instead some general cleanup to prepare
the docs as well as a lookup file used by the tool
to do the conversion. this relatively small patch will
be backported
with appropriate changes to 1.3, 1.2, 1.1 and the tool
can then be run on each branch individually. We are shooting
for almost no warnings at all for master (still a handful
I can't figure out which don't seem to have any impact)
, very few for 1.3,
and for 1.2 / 1.1 we hope for a significant reduction
in warnings.
Overall for all versions pyrefs should
always point to the correct target, if they are in fact
hyperlinked. it's better for a ref to go nowhere and
be plain text than go to the wrong thing. Right now,
hundreds of API links are pointing to the wrong thing
as they are ambiguous names such as refresh(), insert(),
update(), select(), join(), JSON etc. and Sphinx sends these all
to essesntially random destinations among as many as five
or six possible choices per symbol. A shorthand system
that allows us to use absolute refs without having
to type out a full blown absoulte module is the only
way this is going to work, and we should ultimately
seek to abandon any use of prefix dot for lookups. Everything
should be on an underscore token so at the very least the module
spaces can be reorganized without having to search and replace
the entire documentation every time.
Mike Bayer [Mon, 13 Apr 2020 22:39:47 +0000 (18:39 -0400)]
Adjust changelog files for removed directories
unreleased_11 and unreleased_12 are gone so remove these
also there's no place for the 4065 file to go, this is an old
and very small change and it looks like the file was not published in any
case, so just remove it.
Mike Bayer [Fri, 10 Apr 2020 20:58:41 +0000 (16:58 -0400)]
Use define_tables for IsOrIsNotDistinctFromTest
In Ie8127ef29f1ec91e7afb88e1429538c27a321784 Mike
failed to notice that this test is build on TablesTest and
has just the one test function, which means the table is
built up by the fixture.
Mike Bayer [Fri, 10 Apr 2020 16:23:32 +0000 (12:23 -0400)]
Evaluate fixes for cx_Oracle 8 API changes
in [1] we have reported some behavioral changes in cx_Oracle
symbols. As it is unclear if these are intended changes in
cx_Oracle, test workarounds for these changes against
cx_Oracle master.
Mike Bayer [Fri, 10 Apr 2020 18:28:54 +0000 (14:28 -0400)]
Repair CTE a in b tests
In I27cac9bd265c86ff2a3381ff9f844f60ef991cfc we modernized
the default tests and converted the "a in b" CTE tests to combinations,
however apparently the existing tests were not testing all
combinations and had repeats instead. The combinations
decorator has made this much easier to spot, so use
the correct combinations that were originally intended.
Add github action workflow to run tests on master and on pr to master
Trying the pr to check if it works right away
<!-- Describe your changes in detail -->
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 7 Apr 2020 21:37:14 +0000 (17:37 -0400)]
Use dot-separated name resolution for relationship target
The string argument accepted as the first positional argument by the
:func:`.relationship` function when using the Declarative API is no longer
interpreted using the Python ``eval()`` function; instead, the name is dot
separated and the names are looked up directly in the name resolution
dictionary without treating the value as a Python expression. However,
passing a string argument to the other :func:`.relationship` parameters
that necessarily must accept Python expressions will still use ``eval()``;
the documentation has been clarified to ensure that there is no ambiguity
that this is in use.
Mike Bayer [Tue, 7 Apr 2020 23:21:01 +0000 (19:21 -0400)]
Gracefully skip isolation level if no row returned
Fixed issue in MySQL dialect when connecting to a psuedo-MySQL database
such as that provided by ProxySQL, the up front check for isolation level
when it returns no row will not prevent the dialect from continuing to
connect. A warning is emitted that the isolation level could not be
detected.
Gord Thompson [Thu, 13 Feb 2020 19:14:42 +0000 (12:14 -0700)]
Replace engine.execute w/ context manager (step1)
First (baby) step at replacing engine.execute
calls in test code with the new preferred way
of executing. MSSQL was targeted because it was
the easiest for me to test locally.
Federico Caselli [Tue, 24 Mar 2020 20:06:04 +0000 (21:06 +0100)]
Add length parameter in `Enum`
The `Enum` type now supports the parameter `Enum.length`
to specify the length of the VARCHAR column to create when using
non native enums by setting `Enum.native_enum` to `False`
Mike Bayer [Fri, 3 Apr 2020 20:00:22 +0000 (16:00 -0400)]
Key subqueryloaders on the property object, not string key
Fixed bug in :func:`.orm.selectinload` loading option where two or more
loaders that represent different relationships with the same string key
name as referenced from a single :func:`.orm.with_polymorphic` construct
with multiple subclass mappers would fail to invoke each subqueryload
separately, instead making use of a single string-based slot that would
prevent the other loaders from being invoked.
Mike Bayer [Mon, 30 Mar 2020 15:04:24 +0000 (11:04 -0400)]
Remove ORDER BY pk from subqueryload, selectinload
Modified the queries used by subqueryload and selectinload to no longer
ORDER BY the primary key of the parent entity; this ordering was there to
allow the rows as they come in to be copied into lists directly with a
minimal level of Python-side collation. However, these ORDER BY clauses
can negatively impact the performance of the query as in many scenarios
these columns are derived from a subquery or are otherwise not actual
primary key columns such that SQL planners cannot make use of indexes. The
Python-side collation uses the native itertools.group_by() to collate the
incoming rows, and has been modified to allow multiple
row-groups-per-parent to be assembled together using list.extend(), which
should still allow for relatively fast Python-side performance. There will
still be an ORDER BY present for a relationship that includes an explicit
order_by parameter, however this is the only ORDER BY that will be added to
the query for both kinds of loading.
Federico Caselli [Tue, 24 Mar 2020 21:55:46 +0000 (22:55 +0100)]
String compiler can now literal compile datetime objects
Add ability to literal compile a :class:`DateTime`, :class:`Date`
or :class:"Time" when using the string dialect for debugging purposes.
This change does not impact real dialect implementation that retain
their current behavior.
Mike Bayer [Tue, 24 Mar 2020 16:51:53 +0000 (12:51 -0400)]
Reword changelog for issue #5207
SQLAlchemy can remain using setuptools even when pep-517 is
the standard installation process. We are omitting the file
for now because it leads to new pip installation issues that need
to be worked out in the greater pip community before SQLAlchemy
opts into this system.
Gord Thompson [Thu, 12 Mar 2020 18:54:37 +0000 (12:54 -0600)]
Implement autocommit isolation level for cx_oracle
As part of this change Oracle also gets the concept of a
default isolation level, however since Oracle does not provide a
fixed method of knowing what the isolation level would be without a
server side transaction actually in progress, for now we hardcode
just to "READ COMMITTED".
Enhanced the test suite for isolation level testing in the dialect
test suite and added features to requirements so that the supported
isolation levels can be reported generically for dialects.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #5200
Change-Id: I2c4d49da9ff80ccc228c21e196ec9a961de53478
(cherry picked from commit 1afca5823efe22c0f8cbc01455bafc573254ab55)
Mike Bayer [Sat, 21 Mar 2020 21:26:24 +0000 (17:26 -0400)]
Test instance for matching class hierarchy on get_from_identity
Fixed issue where a lazyload that uses session-local "get" against a target
many-to-one relationship where an object with the correct primary key is
present, however it's an instance of a sibling class, does not correctly
return None as is the case when the lazy loader actually emits a load for
that row.
Mike Bayer [Sat, 21 Mar 2020 14:16:47 +0000 (10:16 -0400)]
Remove pyproject.toml from distribution
SQLAlchemy does not want to opt-in to pep-517 at this time
as this would require a custom build backend interface
which we have not built yet, and the standard is not
widely adopted at this time in any case. Per
[1] [2], the presence of this file indicates a positive opt-in
to pep-517, so it must be omitted from source distributions.
Mike Bayer [Fri, 20 Mar 2020 20:40:47 +0000 (16:40 -0400)]
Clarify Alembic vs. Migrate
Migrate is fully legacy at this point so continue to mention
it to provide clarity, but ensure it isn't suggested
as a real alternative. It's unclear if Migrate will be able
to support SQLAlchemy 2.0.
Mike Bayer [Fri, 20 Mar 2020 19:21:10 +0000 (15:21 -0400)]
Emphasize context managers when working with Core
Prep the main documentation for the changes coming up in 1.4
by first removing references to engine.execute() outside
of the "connectionless" section, and using context managers
in all cases. For features that have always been confusing
and are going away, add a note that this feature will
be going away.
Mike Bayer [Fri, 20 Mar 2020 15:17:47 +0000 (11:17 -0400)]
Correct misleading guidance on multiprocessing
Link the connections intro to the dedicated section
on multiprocessing rather than stating that a separate
engine per process is needed, since this is inaccurate
and vague.
mike bayer [Wed, 18 Mar 2020 23:05:20 +0000 (19:05 -0400)]
Don't include PG INCLUDE columns as regular index columns
Fixed issue where a "covering" index, e.g. those which have an INCLUDE
clause, would be reflected including all the columns in INCLUDE clause as
regular columns. A warning is now emitted if these additional columns are
detected indicating that they are currently ignored. Note that full
support for "covering" indexes is part of :ticket:`4458`. Pull request
courtesy Marat Sharafutdinov.
Federico Caselli [Sat, 14 Mar 2020 12:57:42 +0000 (13:57 +0100)]
Support inspection of computed column
Added support for reflection of "computed" columns, which are now returned
as part of the structure returned by :meth:`.Inspector.get_columns`.
When reflecting full :class:`.Table` objects, computed columns will
be represented using the :class:`.Computed` construct.
Also improve the documentation in :meth:`Inspector.get_columns`, correctly
listing all the returned keys.
2. add some exclusion rules to allow the sqlite_file target to work;
add to tox.
3. add reap dbs target for SQLite, repair SQLite drop_db routine
which also wasn't doing the right thing for memory databases
etc.
4. Fix logging in provision files, as the main provision logger
is the one that's enabled by reap_dbs and maybe others, have all
the provision files use the provision logger.
Mike Bayer [Fri, 13 Mar 2020 03:11:18 +0000 (23:11 -0400)]
Include schema in all_tab_comments query
Fixed regression / incorrect fix caused by fix for :ticket:`5146` where the
Oracle dialect reads from the "all_tab_comments" view to get table comments
but fails to accommodate for the current owner of the table being
requested, causing it to read the wrong comment if multiple tables of the
same name exist in multiple schemas.
Mike Bayer [Thu, 12 Mar 2020 23:44:37 +0000 (19:44 -0400)]
Dont raise on pytest deprecation warnings
py.test 5.4.0 emits deprecation warnings for pytest.Class.
make sure we don't raise for these, and log the code that will
be used for 5.4.0 when we bump requirements.
Mike Bayer [Wed, 11 Mar 2020 14:41:12 +0000 (10:41 -0400)]
Repair broken call to sys.exc_info()
Fixed regression in 1.3.14 due to :ticket:`4849` where a sys.exc_info()
call failed to be invoked correctly when a flush error would occur. Test
coverage has been added for this exception case.
Mike Bayer [Tue, 10 Mar 2020 22:48:42 +0000 (18:48 -0400)]
Reword implicit left join error; ensure deterministic FROM for columns
Adjusted the error message emitted by :meth:`.Query.join` when a left hand
side can't be located that the :meth:`.Query.select_from` method is the
best way to resolve the issue. Also, within the 1.3 series, used a
deterministic ordering when determining the FROM clause from a given column
entity passed to :class:`.Query` so that the same expression is determined
each time.
Gord Thompson [Mon, 9 Mar 2020 21:50:53 +0000 (17:50 -0400)]
Reflect comments from any table accessible by the current user
Fixed a reflection bug where table comments could only be retrieved for
tables actually owned by the user but not for tables visible to the user
but owned by someone else. Pull request courtesy Dave Hirschfeld.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
