Mike Bayer [Thu, 1 Apr 2021 16:26:06 +0000 (12:26 -0400)]
Correct for Variant + ARRAY cases in psycopg2
Fixed regression caused by :ticket:`6023` where the PostgreSQL cast
operator applied to elements within an :class:`_types.ARRAY` when using
psycopg2 would fail to use the correct type in the case that the datatype
were also embedded within an instance of the :class:`_types.Variant`
adapter.
Additionally, repairs support for the correct CREATE TYPE to be emitted
when using a ``Variant(ARRAY(some_schema_type))``.
Mike Bayer [Mon, 29 Mar 2021 21:57:20 +0000 (17:57 -0400)]
TypeDecorator passes "outer" flag to itself for set_parent accounting
Fixed bug first introduced in as some combination of :ticket:`2892`,
:ticket:`2919` nnd :ticket:`3832` where the attachment events for a
:class:`_types.TypeDecorator` would be doubled up against the "impl" class,
if the "impl" were also a :class:`_types.SchemaType`. The real-world case
is any :class:`_types.TypeDecorator` against :class:`_types.Enum` or
:class:`_types.Boolean` would get a doubled
:class:`_schema.CheckConstraint` when the ``create_constraint=True`` flag
is set.
Federico Caselli [Wed, 10 Mar 2021 22:54:52 +0000 (23:54 +0100)]
CAST the elements in ARRAYs when using psycopg2
Adjusted the psycopg2 dialect to emit an explicit PostgreSQL-style cast for
bound parameters that contain ARRAY elements. This allows the full range of
datatypes to function correctly within arrays. The asyncpg dialect already
generated these internal casts in the final statement. This also includes
support for array slice updates as well as the PostgreSQL-specific
:meth:`_postgresql.ARRAY.contains` method.
Gord Thompson [Sat, 6 Mar 2021 19:19:13 +0000 (12:19 -0700)]
Fix named CHECK constraint name omitted on repeated creates
Fixed issue where the CHECK constraint generated by :class:`_types.Boolean`
or :class:`_types.Enum` would fail to render the naming convention
correctly after the first compilation, due to an unintended change of state
within the name given to the constraint. This issue was first introduced in
0.9 in the fix for issue #3067, and the fix revises the approach taken at
that time which appears to have been more involved than what was needed.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #6007
Change-Id: I7ecff0a9d86191520f6841b3922a5af5a6971fba
(cherry picked from commit 506b88de5e428fd4ad2feff663ba53e2dbb28891)
Fixed issue where the :class:`_mutable.MutableComposite` construct could be
placed into an invalid state when the parent object was already loaded, and
then covered by a subsequent query, due to the composite properties'
refresh handler replacing the object with a new one not handled by the
mutable extension.
Mike Bayer [Tue, 2 Mar 2021 23:48:08 +0000 (18:48 -0500)]
Remove passive_deletes warning for many-to-one
Removed very old warning that states that passive_deletes is not intended
for many-to-one relationships. While it is likely that in many cases
placing this parameter on a many-to-one relationship is not what was
intended, there are use cases where delete cascade may want to be
disallowed following from such a relationship.
Mike Bayer [Thu, 18 Feb 2021 15:43:16 +0000 (10:43 -0500)]
Extract table names when comparing to nrte error
Fixed issue where the process of joining two tables could fail if one of
the tables had an unrelated, unresolvable foreign key constraint which
would raise :class:`_exc.NoReferenceError` within the join process, which
nonetheless could be bypassed to allow the join to complete. The logic
which tested the exception for signficance within the process would make
assumptions about the construct which would fail.
Mike Bayer [Mon, 8 Feb 2021 16:58:15 +0000 (11:58 -0500)]
Add identifier_preparer per-execution context for schema translates
Fixed bug where the "schema_translate_map" feature failed to be taken into
account for the use case of direct execution of
:class:`_schema.DefaultGenerator` objects such as sequences, which included
the case where they were "pre-executed" in order to generate primary key
values when implicit_returning was disabled.
Mike Bayer [Fri, 5 Feb 2021 14:34:25 +0000 (09:34 -0500)]
Precede sp_columns with EXEC
Fixed issue regarding SQL Server reflection for older SQL Server 2005
version, a call to sp_columns would not proceed correctly without being
prefixed with the EXEC keyword. This method is not used in current 1.4
series.
Mike Bayer [Thu, 4 Feb 2021 17:09:54 +0000 (12:09 -0500)]
Accommodate column-based naming conventions for pk constraint
Repaired / implemented support for primary key constraint naming
conventions that use column names/keys/etc as part of the convention. In
particular, this includes that the :class:`.PrimaryKeyConstraint` object
that's automatically associated with a :class:`.schema.Table` will update
its name as new primary key :class:`_schema.Column` objects are added to
the table and then to the constraint. Internal failure modes related to
this constraint construction process including no columns present, no name
present or blank name present are now accommodated.
Mike Bayer [Sat, 30 Jan 2021 21:57:50 +0000 (16:57 -0500)]
set identifier length for MySQL constraints to 64
The rule to limit index names to 64 also applies to all
DDL names, such as those coming from naming conventions.
Add another limiting variable for constraint names and
create test cases against all constraint types.
Additionally, codified in the test suite MySQL's lack of
support for naming of a FOREIGN KEY constraint after
the name was given, which apparently assigns the name to an
associated KEY but not the constraint itself, until MySQL 8
and MariaDB 10.5 which appear to have resolved the
behavior. However it's not clear how Alembic hasn't had
issues reported with this so far.
Fixed long-lived bug in MySQL dialect where the maximum identifier length
of 255 was too long for names of all types of constraints, not just
indexes, all of which have a size limit of 64. As metadata naming
conventions can create too-long names in this area, apply the limit to the
identifier generator within the DDL compiler.
Gord Thompson [Mon, 25 Jan 2021 18:24:25 +0000 (11:24 -0700)]
Use schema._copy_expression() fully in column collection constraints
Fixed issue where using :meth:`_schema.Table.to_metadata` (called
:meth:`_schema.Table.tometadata` in 1.3) in conjunction with a PostgreSQL
:class:`_postgresql.ExcludeConstraint` that made use of ad-hoc column
expressions would fail to copy correctly.
Mike Bayer [Fri, 29 Jan 2021 18:43:38 +0000 (13:43 -0500)]
Clarify Column.index / Column.unique parameters
These parameters need to be more clear that they cause a
constraint / index object to be generated. Clarify the rules
by which this occurs and include contextual information about
naming conventions as well.
Mike Bayer [Thu, 28 Jan 2021 16:04:29 +0000 (11:04 -0500)]
Allow Oracle CLOB/NCLOB/BLOB in returning
Fixed bug in Oracle dialect where retriving a CLOB/BLOB column via
:meth:`_dml.Insert.returning` would fail as the LOB value would need to be
read when returned; additionally, repaired support for retrieval of Unicode
values via RETURNING under Python 2.
As of yet, we still don't know how to reproduce the
ORA-24813 error indicated in the issue.
Also backporting the statement cache clear added to master
in f1e96cb087 , as we are testing in CI against two oracle
versions now there are sporadic failures that appear to be
memory related.
Mike Bayer [Mon, 25 Jan 2021 18:22:34 +0000 (13:22 -0500)]
Add more docs on engine connecting w/ URL parameters
We have a special section for connect_args but didn't appropriately
introduce that URL keywords may be used as well, which otherwise
does not seem to be documented in a generalized way. Add
this additional detail and contrast between URL query string
and connect_args including providing some transparency for
how to see what URL query string parameters will do for a given
dialect, as we do not handle these parameters consistently
right now.
Mike Bayer [Sun, 24 Jan 2021 22:30:32 +0000 (17:30 -0500)]
un-ignore mike's favorite testing filenames
As im using vscode I'd like these filenames to show up
in filesearch but I'd like to keep "ignore .gitignore files"
turned on. I've moved these names to my own local
.git/info/exclude instead.
Giuseppe Lumia [Sat, 23 Jan 2021 18:04:44 +0000 (13:04 -0500)]
Set upper bound to pg8000 version
For SQLAlchemy 1.3 only, setup.py pins pg8000 to a version lower than
1.16.6. Version 1.16.6 and above is supported by SQLAlchemy 1.4. Pull
request courtesy Giuseppe Lumia.
Mike Bayer [Thu, 14 Jan 2021 16:18:58 +0000 (11:18 -0500)]
Use UnsupportedCompilationError for no default compiler
Fixed issue where the stringification that is sometimes called when
attempting to generate the "key" for the ``.c`` collection on a selectable
would fail if the column were an unlabeled custom SQL construct using the
``sqlalchemy.ext.compiler`` extension, and did not provide a default
compilation form; while this seems like an unusual case, it can get invoked
for some ORM scenarios such as when the expression is used in an "order by"
in combination with joined eager loading. The issue is that the lack of a
default compiler function was raising :class:`.CompileError` and not
:class:`.UnsupportedCompilationError`.
Mike Bayer [Thu, 7 Jan 2021 17:20:51 +0000 (12:20 -0500)]
Update connect args for pymysql 1.0.0; aiomysql fixes
Fixed deprecation warnings that arose as a result of the release of PyMySQL
1.0, including deprecation warnings for the "db" and "passwd" parameters
now replaced with "database" and "password".
For the 1.4 version of this patch, we are also changing tox.ini
to refer to a local branch of aiomysql that fixes pymysql
compatibility issues.
Mike Bayer [Tue, 5 Jan 2021 13:48:36 +0000 (08:48 -0500)]
Remove special rule for TypeDecorator of TypeDecorator
Removing this check for "TypeDecorator" in impl seems to not
break anything and allows TypeDecorator.with_variant() to
work correctly. The line has been traced back to 2007 and
does not appear to have relevance today.
Fixed bug where making use of the :meth:`.TypeEngine.with_variant` method
on a :class:`.TypeDecorator` type would fail to take into account the
dialect-specific mappings in use, due to a rule in :class:`.TypeDecorator`
that was instead attempting to check for chains of :class:`.TypeDecorator`
instances.
Mike Bayer [Mon, 4 Jan 2021 22:05:46 +0000 (17:05 -0500)]
Check for column expr in Oracle RETURNING check
Fixed regression in Oracle dialect introduced by :ticket:`4894` in
SQLAlchemy 1.3.11 where use of a SQL expression in RETURNING for an UPDATE
would fail to compile, due to a check for "server_default" when an
arbitrary SQL expression is not a column.
Mike Bayer [Sat, 26 Dec 2020 16:16:51 +0000 (11:16 -0500)]
narrow the check for double-paren exprs in mysql create_index
Fixed regression from SQLAlchemy 1.3.20 caused by the fix for
:ticket:`5462` which adds double-parenthesis for MySQL functional
expressions in indexes, as is required by the backend, this inadvertently
extended to include arbitrary :func:`_sql.text` expressions as well as
Alembic's internal textual component, which are required by Alembic for
arbitrary index expressions which don't imply double parenthesis. The
check has been narrowed to include only binary/ unary/functional
expressions directly.
Mike Bayer [Fri, 18 Dec 2020 14:28:06 +0000 (09:28 -0500)]
Gracefully degrade on v$transaction not readable
Fixed regression which occured due to [ticket:5755] which implemented
isolation level support for Oracle. It has been reported that many Oracle
accounts don't actually have permission to query the ``v$transaction``
view so this feature has been altered to gracefully fallback when it fails
upon database connect, where the dialect will assume "READ COMMITTED" is
the default isolation level as was the case prior to SQLAlchemy 1.3.21.
However, explicit use of the :meth:`_engine.Connection.get_isolation_level`
method must now necessarily raise an exception, as Oracle databases with
this restriction explicitly disallow the user from reading the current
isolation level.
Mike Bayer [Tue, 15 Dec 2020 14:45:48 +0000 (09:45 -0500)]
Check explicitly for mapped class as secondary
Added a comprehensive check and an informative error message for the case
where a mapped class, or a string mapped class name, is passed to
:paramref:`_orm.relationship.secondary`. This is an extremely common error
which warrants a clear message.
Additionally, added a new rule to the class registry resolution such that
with regards to the :paramref:`_orm.relationship.secondary` parameter, if a
mapped class and its table are of the identical string name, the
:class:`.Table` will be favored when resolving this parameter. In all
other cases, the class continues to be favored if a class and table
share the identical name.
Mike Bayer [Tue, 15 Dec 2020 13:52:15 +0000 (08:52 -0500)]
Use .expression accessor for hybrid example
hybrids since 1.1 use InstrumentedAttribute at the expression
level, so this doc needed to illustrate how to get at the
SQL expression. Also added a docstring for
QueryableAttribute.expression.
Mike Bayer [Wed, 9 Dec 2020 03:07:48 +0000 (22:07 -0500)]
Implement Oracle SERIALIZABLE + real read of isolation level
There's some significant awkwardness in that we can't
read the level unless a transaction is started, which normally
does not occur unless DML is emitted. The implementation
uses the local_transaction_id function to start a transaction.
It is not known what the performance impact of this might
have, however by default the function is called only once
on first connect and later only if the get_isolation_level()
method is used.
Mike Bayer [Thu, 3 Dec 2020 16:17:08 +0000 (11:17 -0500)]
Reflect decimal points in MariaDB non-quoted numeric defaults
Fixed issue where reflecting a server default on MariaDB only that
contained a decimal point in the value would fail to be reflected
correctly, leading towards a reflected table that lacked any server
default.
Mike Bayer [Wed, 2 Dec 2020 14:26:18 +0000 (09:26 -0500)]
modernize contains_eager() docs
Along with other loader options that are likely
to require usage of populate_existing(), make sure
contains_eager() documentation makes it absolutely
clear that already-loaded collections are not overwritten.
consolidate contains_eager() documentation into the narrative docs.
Additionally, remove the "arbitrary statements" section, this is
not a useful case and seems to be left over.
Mike Bayer [Tue, 1 Dec 2020 21:03:35 +0000 (16:03 -0500)]
Pass along other keyword args in _EventsHold.populate
Fixed bug involving the ``restore_load_context`` option of ORM events such
as :meth:`_orm.InstanceEvents.load` such that the flag would not be carried
along to subclasses which were mapped after the event handler were first
established.
Mike Bayer [Thu, 26 Nov 2020 17:10:25 +0000 (12:10 -0500)]
Don't discard leftovers from surface_selectables
Fixed regression introduced in 1.3.2 for the PostgreSQL dialect, also
copied out to the MySQL dialect's feature in 1.3.18, where usage of a non
:class:`_schema.Table` construct such as :func:`_sql.text` as the argument
to :paramref:`_sql.Select.with_for_update.of` would fail to be accommodated
correctly within the PostgreSQL or MySQL compilers.
AlonM [Sun, 15 Nov 2020 13:13:25 +0000 (08:13 -0500)]
Add opsclass to exclusion constraint
Added new parameter :paramref:`_postgresql.ExcludeConstraint.ops` to the
:class:`_postgresql.ExcludeConstraint` object, to support operator class
specification with this constraint. Pull request courtesy Alon Menczer.
Oscar Batori [Wed, 16 Sep 2020 17:41:34 +0000 (13:41 -0400)]
Added missing keywords for MySQL dialect
Fixes: #5696
Add missing MySQL keywords.
### Description
Adds some missing keywords from the MySQL dialect. While it's hard to describe this as a documentation/typographical fix, there are no current test for presence of the correct keywords, nor a realistic way to implement them those tests.
### Checklist
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Wed, 11 Nov 2020 16:13:27 +0000 (11:13 -0500)]
Warn / raise for returning() / return_defaults() combinations
A warning is emmitted if a returning() method such as
:meth:`_sql.Insert.returning` is called multiple times, as this does not
yet support additive operation. Version 1.4 will support additive
operation for this. Additionally, any combination of the
:meth:`_sql.Insert.returning` and :meth:`_sql.Insert.return_defaults`
methods now raises an error as these methods are mutually exclusive;
previously the operation would fail silently.
Sean Anderson [Sun, 8 Nov 2020 00:52:04 +0000 (19:52 -0500)]
Support SQLite WITHOUT ROWID tables
This adds support for creating tables WITHOUT ROWID in the SQLite
dialect. WITHOUT ROWID tables were introduced in SQLite version 3.8.2
(2013-12-06). They do not use an implicit rowid column as the primary
key. This may result in space and performance savings for tables without
INTEGER primary keys and tables with composite primary keys. For more
information about this feature, see the sqlite documentation [1].
[1] https://www.sqlite.org/withoutrowid.html
Fixes: #5685
### Checklist
This pull request is:
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Sat, 31 Oct 2020 23:08:28 +0000 (19:08 -0400)]
update selectin docs
* correct many-to-one example that doesnt use JOIN or ORDER BY
anymore
* Oracle does tuple IN, let's test it
* many-to-many is supported but joins all the way right now
* remove verbiage about yield_per for the moment to simplify
updates to how yield_per works w/ new style execution. yield_per
is difficult to explain and the section seems kind of complicated
with those details added at the moment.
Mike Bayer [Wed, 21 Oct 2020 17:58:22 +0000 (13:58 -0400)]
Don't populate expired attrs w/ evaluator
Fixed bug in :meth:`_orm.Query.update` where objects in the
:class:`_orm.Session` that were already expired would be unnecessarily
SELECTed individually when they were refreshed by the "evaluate"
synchronize strategy.
For 1.4 there was also a similar issue with fetch that would actually
get the wrong data back, as the new value would be loaded, then
applied with the evaluator.
Mike Bayer [Mon, 19 Oct 2020 14:19:29 +0000 (10:19 -0400)]
Ensure no compiler visit method tries to access .statement
Fixed structural compiler issue where some constructs such as MySQL /
PostgreSQL "on conflict / on duplicate key" would rely upon the state of
the :class:`_sql.Compiler` object being fixed against their statement as
the top level statement, which would fail in cases where those statements
are branched from a different context, such as a DDL construct linked to a
SQL statement.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
