Mike Bayer [Sat, 30 Jan 2021 21:57:50 +0000 (16:57 -0500)]
set identifier length for MySQL constraints to 64
The rule to limit index names to 64 also applies to all
DDL names, such as those coming from naming conventions.
Add another limiting variable for constraint names and
create test cases against all constraint types.
Additionally, codified in the test suite MySQL's lack of
support for naming of a FOREIGN KEY constraint after
the name was given, which apparently assigns the name to an
associated KEY but not the constraint itself, until MySQL 8
and MariaDB 10.5 which appear to have resolved the
behavior. However it's not clear how Alembic hasn't had
issues reported with this so far.
Fixed long-lived bug in MySQL dialect where the maximum identifier length
of 255 was too long for names of all types of constraints, not just
indexes, all of which have a size limit of 64. As metadata naming
conventions can create too-long names in this area, apply the limit to the
identifier generator within the DDL compiler.
Gord Thompson [Mon, 25 Jan 2021 18:24:25 +0000 (11:24 -0700)]
Use schema._copy_expression() fully in column collection constraints
Fixed issue where using :meth:`_schema.Table.to_metadata` (called
:meth:`_schema.Table.tometadata` in 1.3) in conjunction with a PostgreSQL
:class:`_postgresql.ExcludeConstraint` that made use of ad-hoc column
expressions would fail to copy correctly.
Mike Bayer [Fri, 29 Jan 2021 18:43:38 +0000 (13:43 -0500)]
Clarify Column.index / Column.unique parameters
These parameters need to be more clear that they cause a
constraint / index object to be generated. Clarify the rules
by which this occurs and include contextual information about
naming conventions as well.
Mike Bayer [Thu, 28 Jan 2021 16:04:29 +0000 (11:04 -0500)]
Allow Oracle CLOB/NCLOB/BLOB in returning
Fixed bug in Oracle dialect where retriving a CLOB/BLOB column via
:meth:`_dml.Insert.returning` would fail as the LOB value would need to be
read when returned; additionally, repaired support for retrieval of Unicode
values via RETURNING under Python 2.
As of yet, we still don't know how to reproduce the
ORA-24813 error indicated in the issue.
Also backporting the statement cache clear added to master
in f1e96cb087 , as we are testing in CI against two oracle
versions now there are sporadic failures that appear to be
memory related.
Mike Bayer [Mon, 25 Jan 2021 18:22:34 +0000 (13:22 -0500)]
Add more docs on engine connecting w/ URL parameters
We have a special section for connect_args but didn't appropriately
introduce that URL keywords may be used as well, which otherwise
does not seem to be documented in a generalized way. Add
this additional detail and contrast between URL query string
and connect_args including providing some transparency for
how to see what URL query string parameters will do for a given
dialect, as we do not handle these parameters consistently
right now.
Mike Bayer [Sun, 24 Jan 2021 22:30:32 +0000 (17:30 -0500)]
un-ignore mike's favorite testing filenames
As im using vscode I'd like these filenames to show up
in filesearch but I'd like to keep "ignore .gitignore files"
turned on. I've moved these names to my own local
.git/info/exclude instead.
Giuseppe Lumia [Sat, 23 Jan 2021 18:04:44 +0000 (13:04 -0500)]
Set upper bound to pg8000 version
For SQLAlchemy 1.3 only, setup.py pins pg8000 to a version lower than
1.16.6. Version 1.16.6 and above is supported by SQLAlchemy 1.4. Pull
request courtesy Giuseppe Lumia.
Mike Bayer [Thu, 14 Jan 2021 16:18:58 +0000 (11:18 -0500)]
Use UnsupportedCompilationError for no default compiler
Fixed issue where the stringification that is sometimes called when
attempting to generate the "key" for the ``.c`` collection on a selectable
would fail if the column were an unlabeled custom SQL construct using the
``sqlalchemy.ext.compiler`` extension, and did not provide a default
compilation form; while this seems like an unusual case, it can get invoked
for some ORM scenarios such as when the expression is used in an "order by"
in combination with joined eager loading. The issue is that the lack of a
default compiler function was raising :class:`.CompileError` and not
:class:`.UnsupportedCompilationError`.
Mike Bayer [Thu, 7 Jan 2021 17:20:51 +0000 (12:20 -0500)]
Update connect args for pymysql 1.0.0; aiomysql fixes
Fixed deprecation warnings that arose as a result of the release of PyMySQL
1.0, including deprecation warnings for the "db" and "passwd" parameters
now replaced with "database" and "password".
For the 1.4 version of this patch, we are also changing tox.ini
to refer to a local branch of aiomysql that fixes pymysql
compatibility issues.
Mike Bayer [Tue, 5 Jan 2021 13:48:36 +0000 (08:48 -0500)]
Remove special rule for TypeDecorator of TypeDecorator
Removing this check for "TypeDecorator" in impl seems to not
break anything and allows TypeDecorator.with_variant() to
work correctly. The line has been traced back to 2007 and
does not appear to have relevance today.
Fixed bug where making use of the :meth:`.TypeEngine.with_variant` method
on a :class:`.TypeDecorator` type would fail to take into account the
dialect-specific mappings in use, due to a rule in :class:`.TypeDecorator`
that was instead attempting to check for chains of :class:`.TypeDecorator`
instances.
Mike Bayer [Mon, 4 Jan 2021 22:05:46 +0000 (17:05 -0500)]
Check for column expr in Oracle RETURNING check
Fixed regression in Oracle dialect introduced by :ticket:`4894` in
SQLAlchemy 1.3.11 where use of a SQL expression in RETURNING for an UPDATE
would fail to compile, due to a check for "server_default" when an
arbitrary SQL expression is not a column.
Mike Bayer [Sat, 26 Dec 2020 16:16:51 +0000 (11:16 -0500)]
narrow the check for double-paren exprs in mysql create_index
Fixed regression from SQLAlchemy 1.3.20 caused by the fix for
:ticket:`5462` which adds double-parenthesis for MySQL functional
expressions in indexes, as is required by the backend, this inadvertently
extended to include arbitrary :func:`_sql.text` expressions as well as
Alembic's internal textual component, which are required by Alembic for
arbitrary index expressions which don't imply double parenthesis. The
check has been narrowed to include only binary/ unary/functional
expressions directly.
Mike Bayer [Fri, 18 Dec 2020 14:28:06 +0000 (09:28 -0500)]
Gracefully degrade on v$transaction not readable
Fixed regression which occured due to [ticket:5755] which implemented
isolation level support for Oracle. It has been reported that many Oracle
accounts don't actually have permission to query the ``v$transaction``
view so this feature has been altered to gracefully fallback when it fails
upon database connect, where the dialect will assume "READ COMMITTED" is
the default isolation level as was the case prior to SQLAlchemy 1.3.21.
However, explicit use of the :meth:`_engine.Connection.get_isolation_level`
method must now necessarily raise an exception, as Oracle databases with
this restriction explicitly disallow the user from reading the current
isolation level.
Mike Bayer [Tue, 15 Dec 2020 14:45:48 +0000 (09:45 -0500)]
Check explicitly for mapped class as secondary
Added a comprehensive check and an informative error message for the case
where a mapped class, or a string mapped class name, is passed to
:paramref:`_orm.relationship.secondary`. This is an extremely common error
which warrants a clear message.
Additionally, added a new rule to the class registry resolution such that
with regards to the :paramref:`_orm.relationship.secondary` parameter, if a
mapped class and its table are of the identical string name, the
:class:`.Table` will be favored when resolving this parameter. In all
other cases, the class continues to be favored if a class and table
share the identical name.
Mike Bayer [Tue, 15 Dec 2020 13:52:15 +0000 (08:52 -0500)]
Use .expression accessor for hybrid example
hybrids since 1.1 use InstrumentedAttribute at the expression
level, so this doc needed to illustrate how to get at the
SQL expression. Also added a docstring for
QueryableAttribute.expression.
Mike Bayer [Wed, 9 Dec 2020 03:07:48 +0000 (22:07 -0500)]
Implement Oracle SERIALIZABLE + real read of isolation level
There's some significant awkwardness in that we can't
read the level unless a transaction is started, which normally
does not occur unless DML is emitted. The implementation
uses the local_transaction_id function to start a transaction.
It is not known what the performance impact of this might
have, however by default the function is called only once
on first connect and later only if the get_isolation_level()
method is used.
Mike Bayer [Thu, 3 Dec 2020 16:17:08 +0000 (11:17 -0500)]
Reflect decimal points in MariaDB non-quoted numeric defaults
Fixed issue where reflecting a server default on MariaDB only that
contained a decimal point in the value would fail to be reflected
correctly, leading towards a reflected table that lacked any server
default.
Mike Bayer [Wed, 2 Dec 2020 14:26:18 +0000 (09:26 -0500)]
modernize contains_eager() docs
Along with other loader options that are likely
to require usage of populate_existing(), make sure
contains_eager() documentation makes it absolutely
clear that already-loaded collections are not overwritten.
consolidate contains_eager() documentation into the narrative docs.
Additionally, remove the "arbitrary statements" section, this is
not a useful case and seems to be left over.
Mike Bayer [Tue, 1 Dec 2020 21:03:35 +0000 (16:03 -0500)]
Pass along other keyword args in _EventsHold.populate
Fixed bug involving the ``restore_load_context`` option of ORM events such
as :meth:`_orm.InstanceEvents.load` such that the flag would not be carried
along to subclasses which were mapped after the event handler were first
established.
Mike Bayer [Thu, 26 Nov 2020 17:10:25 +0000 (12:10 -0500)]
Don't discard leftovers from surface_selectables
Fixed regression introduced in 1.3.2 for the PostgreSQL dialect, also
copied out to the MySQL dialect's feature in 1.3.18, where usage of a non
:class:`_schema.Table` construct such as :func:`_sql.text` as the argument
to :paramref:`_sql.Select.with_for_update.of` would fail to be accommodated
correctly within the PostgreSQL or MySQL compilers.
AlonM [Sun, 15 Nov 2020 13:13:25 +0000 (08:13 -0500)]
Add opsclass to exclusion constraint
Added new parameter :paramref:`_postgresql.ExcludeConstraint.ops` to the
:class:`_postgresql.ExcludeConstraint` object, to support operator class
specification with this constraint. Pull request courtesy Alon Menczer.
Oscar Batori [Wed, 16 Sep 2020 17:41:34 +0000 (13:41 -0400)]
Added missing keywords for MySQL dialect
Fixes: #5696
Add missing MySQL keywords.
### Description
Adds some missing keywords from the MySQL dialect. While it's hard to describe this as a documentation/typographical fix, there are no current test for presence of the correct keywords, nor a realistic way to implement them those tests.
### Checklist
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Wed, 11 Nov 2020 16:13:27 +0000 (11:13 -0500)]
Warn / raise for returning() / return_defaults() combinations
A warning is emmitted if a returning() method such as
:meth:`_sql.Insert.returning` is called multiple times, as this does not
yet support additive operation. Version 1.4 will support additive
operation for this. Additionally, any combination of the
:meth:`_sql.Insert.returning` and :meth:`_sql.Insert.return_defaults`
methods now raises an error as these methods are mutually exclusive;
previously the operation would fail silently.
Sean Anderson [Sun, 8 Nov 2020 00:52:04 +0000 (19:52 -0500)]
Support SQLite WITHOUT ROWID tables
This adds support for creating tables WITHOUT ROWID in the SQLite
dialect. WITHOUT ROWID tables were introduced in SQLite version 3.8.2
(2013-12-06). They do not use an implicit rowid column as the primary
key. This may result in space and performance savings for tables without
INTEGER primary keys and tables with composite primary keys. For more
information about this feature, see the sqlite documentation [1].
[1] https://www.sqlite.org/withoutrowid.html
Fixes: #5685
### Checklist
This pull request is:
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Sat, 31 Oct 2020 23:08:28 +0000 (19:08 -0400)]
update selectin docs
* correct many-to-one example that doesnt use JOIN or ORDER BY
anymore
* Oracle does tuple IN, let's test it
* many-to-many is supported but joins all the way right now
* remove verbiage about yield_per for the moment to simplify
updates to how yield_per works w/ new style execution. yield_per
is difficult to explain and the section seems kind of complicated
with those details added at the moment.
Mike Bayer [Wed, 21 Oct 2020 17:58:22 +0000 (13:58 -0400)]
Don't populate expired attrs w/ evaluator
Fixed bug in :meth:`_orm.Query.update` where objects in the
:class:`_orm.Session` that were already expired would be unnecessarily
SELECTed individually when they were refreshed by the "evaluate"
synchronize strategy.
For 1.4 there was also a similar issue with fetch that would actually
get the wrong data back, as the new value would be loaded, then
applied with the evaluator.
Mike Bayer [Mon, 19 Oct 2020 14:19:29 +0000 (10:19 -0400)]
Ensure no compiler visit method tries to access .statement
Fixed structural compiler issue where some constructs such as MySQL /
PostgreSQL "on conflict / on duplicate key" would rely upon the state of
the :class:`_sql.Compiler` object being fixed against their statement as
the top level statement, which would fail in cases where those statements
are branched from a different context, such as a DDL construct linked to a
SQL statement.
Support indexing on expressions and functions for the MySQL dialect
A user noticed that creating an index where the "key part" was an expression
or function would raise an error for MySQL because the key part was not
parenthesized. The proposed change will check whether a key part is not a
Column or Unary Expression and parenthesize if the case is False.
This fix also contains a minor fix to a test case that was previously incorrect
(`def test_create_index_expr():`).
Mike Bayer [Wed, 30 Sep 2020 12:37:57 +0000 (08:37 -0400)]
raise on lower-case column shared to multiple tables
Fixed bug where an error was not raised for lower-case
:func:`_column` added to lower-case :func:`_table` object. This now raises
:class:`_exc.ArgumentError` which has always been the case for
upper-case :class:`_schema.Column` and :class:`_schema.Table`.
Mike Bayer [Tue, 29 Sep 2020 18:17:42 +0000 (14:17 -0400)]
Scan for tables without relying upon whereclause
Fixed bug where an UPDATE statement against a JOIN using MySQL multi-table
format would fail to include the table prefix for the target table if the
statement had no WHERE clause, as only the WHERE clause were scanned to
detect a "multi table update" at that particular point. The target
is now also scanned if it's a JOIN to get the leftmost table as the
primary table and the additional entries as additional FROM entries.
Mike Bayer [Tue, 29 Sep 2020 19:44:33 +0000 (15:44 -0400)]
bump variance on test_string, test_unicode
a recent rerun of profiles added more profiling data that's
failing over small differences. 15% variance is fine for these
tests that are looking for thousands of encode calls.
Support for multiple hosts in PostgreSQL connection string
Provide support for multiple hosts in the PostgreSQL connection string.
A user requested for SQLAlchemy to support multiple hosts within a PostgreSQL URL string. The proposed fix allows this. In the event that the url contains multiple hosts the proposed code will convert the query["hosts"] tuple into a single string. This allows the hosts to then get converted into a valid dsn variable in the psycopg2 connect function.
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Stringify correctly for non-str exception argument
Fixed issue where a non-string object sent to
:class:`_exc.SQLAlchemyError` or a subclass, as occurs with some third
party dialects, would fail to stringify correctly. Pull request
courtesy Andrzej Bartosiński.
The "skip_locked" keyword used with ``with_for_update()`` will emit a
warning when used on MariaDB backends, and will then be ignored. This is
a deprecated behavior that will raise in SQLAlchemy 1.4, as an application
that requests "skip locked" is looking for a non-blocking operation which
is not available on those backends.
Mike Bayer [Mon, 14 Sep 2020 14:14:48 +0000 (10:14 -0400)]
Pass all pool parameters in recreate()
The following pool parameters were not being propagated to the new pool
created when :meth:`_engine.Engine.dispose` were called: ``pre_ping``,
``use_lifo``. Additionally the ``recycle`` and ``reset_on_return``
parameters were not propagated for the :class:`_engine.AssertionPool`
class. These issues have been fixed.
Mike Bayer [Thu, 10 Sep 2020 15:56:34 +0000 (11:56 -0400)]
Add more docs for populate_existing(); link with_for_update
The populate_existing() method is actually changing
to be an execution option, however it has almost no
mention in the narrative docs so add docs in terms of the
1.x version first, including that we mention you almost
definitely want to use this method if you are also using
with_for_update().
Mike Bayer [Tue, 8 Sep 2020 15:01:28 +0000 (11:01 -0400)]
PostgreSQL dialect-level isolation_level parameter is legacy
The isolation level section in the docs inadvertently
copied the PostgreSQL example using the PGDialect.isolation_level
parameter and not the execution_options. ensure only
the execution_options version is documented.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
