Mike Bayer [Wed, 30 Dec 2020 18:56:20 +0000 (13:56 -0500)]
Support TypeDecorator.get_dbapi_type() for setinpusizes
Adjusted the "setinputsizes" logic relied upon by the cx_Oracle, asyncpg
and pg8000 dialects to support a :class:`.TypeDecorator` that includes
an override the :meth:`.TypeDecorator.get_dbapi_type()` method.
Mike Bayer [Wed, 30 Dec 2020 02:06:15 +0000 (21:06 -0500)]
disambiguate SQL server temp table constraint names
This seems to be raising errors lately which was not the
case earlier, however SQL Server seems to produce name conflicts
for named constraints against temp tables in different databases.
I can raise the error every time here running two tests
from ComponentReflectionTest with -n2. Unknown why the issue
is happening now and didn't occur for several months.
https://www.arbinada.com/en/node/1645 has some background.
Mike Bayer [Sat, 26 Dec 2020 16:46:42 +0000 (11:46 -0500)]
implement sessionmaker.begin(), scalar() for async session
Added :meth:`_asyncio.AsyncSession.scalar` as well as support for
:meth:`_orm.sessionmaker.begin` to work as an async context manager with
:class:`_asyncio.AsyncSession`. Also added
:meth:`_asyncio.AsyncSession.in_transaction` accessor.
Mike Bayer [Sat, 26 Dec 2020 16:16:51 +0000 (11:16 -0500)]
narrow the check for double-paren exprs in mysql create_index
Fixed regression from SQLAlchemy 1.3.20 caused by the fix for
:ticket:`5462` which adds double-parenthesis for MySQL functional
expressions in indexes, as is required by the backend, this inadvertently
extended to include arbitrary :func:`_sql.text` expressions as well as
Alembic's internal textual component, which are required by Alembic for
arbitrary index expressions which don't imply double parenthesis. The
check has been narrowed to include only binary/ unary/functional
expressions directly.
Mike Bayer [Wed, 23 Dec 2020 15:43:51 +0000 (10:43 -0500)]
Add ORMExecuteState mapper accessors
Added :attr:`_orm.ORMExecuteState.bind_mapper` and
:attr:`_orm.ORMExecuteState.all_mappers` accessors to
:class:`_orm.ORMExecuteState` event object, so that handlers can respond to
the target mapper and/or mapped class or classes involved in an ORM
statement execution.
Gord Thompson [Sun, 20 Dec 2020 17:20:10 +0000 (10:20 -0700)]
Fix issues with JSON and float/numeric
Decimal accuracy and behavior has been improved when extracting floating
point and/or decimal values from JSON strings using the
:meth:`_sql.sqltypes.JSON.Comparator.as_float` method, when the numeric
value inside of the JSON string has many significant digits; previously,
MySQL backends would truncate values with many significant digits and SQL
Server backends would raise an exception due to a DECIMAL cast with
insufficient significant digits. Both backends now use a FLOAT-compatible
approach that does not hardcode significant digits for floating point
values. For precision numerics, a new method
:meth:`_sql.sqltypes.JSON.Comparator.as_numeric` has been added which
accepts arguments for precision and scale, and will return values as Python
``Decimal`` objects with no floating point conversion assuming the DBAPI
supports it (all but pysqlite).
Mike Bayer [Thu, 3 Dec 2020 19:35:30 +0000 (14:35 -0500)]
Allow Declarative to extract class attr from field
Added an alternate resolution scheme to Declarative that will extract the
SQLAlchemy column or mapped property from the "metadata" dictionary of a
dataclasses.Field object. This allows full declarative mappings to be
combined with dataclass fields.
esoh [Sat, 19 Dec 2020 02:59:31 +0000 (21:59 -0500)]
Repair and cover adaption call w/ ORM having()
Fixed 1.4 regression where the use of :meth:`_orm.Query.having` in
conjunction with queries with internally adapted SQL elements (common in
inheritance scenarios) would fail due to an incorrect function call. Pull
request courtesy esoh.
Mike Bayer [Sat, 19 Dec 2020 15:59:42 +0000 (10:59 -0500)]
Repair mssql dep tests; have __only_on__ imply __backend__
CI missed a few SQL Server tests because we run mssql-backendonly
in the gerrit job. As there was a test that was "only on"
mssql but didn't have backendonly, it never got run and then
fails in master where we run mssql fully.
Any suite that has an __only_on__ is inherently specific to
a backend, so if present this should imply __backend__ so that
it definitely runs when we have that backend present.
This in turn meant we had to fix a few sqlite_file tests that
weren't cleaning up or sharing well as they suddenly became
backend tests under sqlite_file. Added a sqlite_file cleanup
to test class cleanup for now.
Mike Bayer [Fri, 18 Dec 2020 17:22:12 +0000 (12:22 -0500)]
Improve type detection for Values / Tuple
Fixed issue in new :class:`_sql.Values` construct where passing tuples of
objects would fall back to per-value type detection rather than making use
of the :class:`_schema.Column` objects passed directly to
:class:`_sql.Values` that tells SQLAlchemy what the expected type is. This
would lead to issues for objects such as enumerations and numpy strings
that are not actually necessary since the expected type is given.
note this changes NullType() to raise CompileError for
literal_processor; NullType() does not imply the actual value
NULL as much as it does "unknown type" so this should make failure
modes more clear.
Mike Bayer [Fri, 18 Dec 2020 14:28:06 +0000 (09:28 -0500)]
Gracefully degrade on v$transaction not readable
Fixed regression which occured due to [ticket:5755] which implemented
isolation level support for Oracle. It has been reported that many Oracle
accounts don't actually have permission to query the ``v$transaction``
view so this feature has been altered to gracefully fallback when it fails
upon database connect, where the dialect will assume "READ COMMITTED" is
the default isolation level as was the case prior to SQLAlchemy 1.3.21.
However, explicit use of the :meth:`_engine.Connection.get_isolation_level`
method must now necessarily raise an exception, as Oracle databases with
this restriction explicitly disallow the user from reading the current
isolation level.
Mike Bayer [Sat, 12 Dec 2020 23:56:58 +0000 (18:56 -0500)]
Major revisals to lambdas
1. Improve coercions._deep_is_literal to check sequences
for clause elements, thus allowing a phrase like
lambda: col.in_([literal("x"), literal("y")]) to be handled
2. revise closure variable caching completely. All variables
entering must be part of a closure cache key or rejected.
only objects that can be resolved to HasCacheKey or FunctionType
are accepted; all other types are rejected. This adds a high
degree of strictness to lambdas and will make them a little more
awkward to use in some cases, however prevents several classes
of critical issues:
a. previously, a lambda that had an expression derived from
some kind of state, like "self.x", or "execution_context.session.foo"
would produce a closure cache key from "self" or "execution_context",
objects that can very well be per-execution and would therefore
cause a AnalyzedFunction objects to overflow. (memory won't leak
as it looks like an LRUCache is already used for these)
b. a lambda, such as one used within DeferredLamdaElement, that
produces different SQL expressions based on the arguments
(which is in fact what it's supposed to do), however it would
through the use of conditionals produce different bound parameter
combinations, leading to literal parameters not tracked properly.
These are now rejected as uncacheable whereas previously they would
again be part of the closure cache key, causing an overflow of
AnalyizedFunction objects.
3. Ensure non-mapped mixins are handled correctly by
with_loader_criteria().
4. Fixed bug in lambda SQL system where we are not supposed to allow a Python
function to be embedded in the lambda, since we can't predict a bound value
from it. While there was an error condition added for this, it was not
tested and wasn't working; an informative error is now raised.
Michael Oliver [Wed, 16 Dec 2020 16:49:07 +0000 (11:49 -0500)]
Improved PEP-484 for AsyncSession.run_sync
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
I don't think this needs any test added.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 15 Dec 2020 14:45:48 +0000 (09:45 -0500)]
Check explicitly for mapped class as secondary
Added a comprehensive check and an informative error message for the case
where a mapped class, or a string mapped class name, is passed to
:paramref:`_orm.relationship.secondary`. This is an extremely common error
which warrants a clear message.
Additionally, added a new rule to the class registry resolution such that
with regards to the :paramref:`_orm.relationship.secondary` parameter, if a
mapped class and its table are of the identical string name, the
:class:`.Table` will be favored when resolving this parameter. In all
other cases, the class continues to be favored if a class and table
share the identical name.
Mike Bayer [Tue, 15 Dec 2020 13:52:15 +0000 (08:52 -0500)]
Use .expression accessor for hybrid example
hybrids since 1.1 use InstrumentedAttribute at the expression
level, so this doc needed to illustrate how to get at the
SQL expression. Also added a docstring for
QueryableAttribute.expression.
RamonWill [Wed, 21 Oct 2020 12:24:27 +0000 (08:24 -0400)]
Support IF EXISTS/IF NOT EXISTS for DDL constructs
Added parameters :paramref:`_ddl.CreateTable.if_not_exists`,
:paramref:`_ddl.CreateIndex.if_not_exists`,
:paramref:`_ddl.DropTable.if_exists` and
:paramref:`_ddl.DropIndex.if_exists` to the :class:`_ddl.CreateTable`,
:class:`_ddl.DropTable`, :class:`_ddl.CreateIndex` and
:class:`_ddl.DropIndex` constructs which result in "IF NOT EXISTS" / "IF
EXISTS" DDL being added to the CREATE/DROP. These phrases are not accepted
by all databases and the operation will fail on a database that does not
support it as there is no similarly compatible fallback within the scope of
a single DDL statement. Pull request courtesy Ramon Williams.
Gord Thompson [Sat, 12 Dec 2020 16:29:12 +0000 (09:29 -0700)]
Remove autoload=True references
Remove references to deprecated ``autoload=True`` in docs and code samples.
Also remove test/dialect/test_firebird.py and test/dialect/test_sybase.py
as those tests have already been incorporated into the external dialects.
Mike Bayer [Sat, 12 Dec 2020 00:01:12 +0000 (19:01 -0500)]
Fixes for lambda expressions and relationship loaders
Fixed bug in lambda SQL feature, used by ORM
:meth:`_orm.with_loader_criteria` as well as available generally in the SQL
expression language, where assigning a boolean value True/False to a
variable would cause the query-time expression calculation to fail, as it
would produce a SQL expression not compatible with a bound value.
Fixed issue where the :attr:`_orm.ORMExecuteState.is_relationship_load`
parameter would not be set correctly for many lazy loads, all
selectinloads, etc. The flag is essential in order to test if options
should be added to statements or if they would already have been propagated
via relationship loads.
Mike Bayer [Mon, 30 Nov 2020 17:50:51 +0000 (12:50 -0500)]
Send deterministic ordering into unit of work topological
Improved the unit of work topological sorting system such that the
toplogical sort is now deterministic based on the sorting of the input set,
which itself is now sorted at the level of mappers, so that the same inputs
of affected mappers should produce the same output every time, among
mappers / tables that don't have any dependency on each other. This further
reduces the chance of deadlocks as can be observed in a flush that UPDATEs
among multiple, unrelated tables such that row locks are generated.
topological.sort() has been made "deterministic" in all cases by
using a separate list + set.
Mike Bayer [Wed, 9 Dec 2020 16:39:45 +0000 (11:39 -0500)]
Emit deprecation warnings for plain text under Session
Deprecation warnings are emitted under "SQLALCHEMY_WARN_20" mode when
passing a plain string to :meth:`_orm.Session.execute`.
It was also considered to have DDL string expressions to include
this as well, however this leaves us with no backwards-compatible
way of handling reflection of elemens, such as an Index() which
reflects "postgresql_where='x > 5'", there's no place for a rule
that will turn those into text() within the reflection process
that would be separate from when the user passes postgresql_where
to the Index. Not worth it right now.
Mike Bayer [Sun, 15 Nov 2020 21:58:50 +0000 (16:58 -0500)]
correct for "autocommit" deprecation warning
Ensure no autocommit warnings occur internally or
within tests.
Also includes fixes for SQL Server full text tests
which apparently have not been working at all for a long
time, as it used long removed APIs. CI has not had
fulltext running for some years and is now installed.
Mike Bayer [Fri, 11 Dec 2020 15:34:44 +0000 (10:34 -0500)]
Revise attribute refresh for with_loader_criteria, related
Added new attribute :attr:`_orm.ORMExecuteState.is_column_load` to indicate
that a :meth:`_orm.SessionEvents.do_orm_execute` handler that a particular
operation is a primary-key-directed column attribute load, such as from an
expiration or a deferred attribute, and that WHERE criteria or additional
loader options should not be added to the query. This has been added to
the examples which illustrate the :func:`_orm.with_loader_criteria` option.
The :func:`_orm.with_loader_criteria` option has been modified so that it
will never apply its criteria to the SELECT statement for an ORM refresh
operation, such as that invoked by :meth:`_orm.Session.refresh` or whenever
an expired attribute is loaded. These queries are only against the
primary key row of the object that is already present in memory so there
should not be additional criteria added.
Added doc caveats for using lambdas.
Added test coverage for most ORMExecuteState flags and fixed a few
basic access issues.
Mike Bayer [Wed, 9 Dec 2020 03:07:48 +0000 (22:07 -0500)]
Implement Oracle SERIALIZABLE + real read of isolation level
There's some significant awkwardness in that we can't
read the level unless a transaction is started, which normally
does not occur unless DML is emitted. The implementation
uses the local_transaction_id function to start a transaction.
It is not known what the performance impact of this might
have, however by default the function is called only once
on first connect and later only if the get_isolation_level()
method is used.
The crashes that occur in jenkins have not been solved and are
now impacting master. I am not able to reproduce the failure,
including running on the CI machines directly, and a few runs
where I sat there for 20 minutes and watched, it didn't happen.
it is the ultimate heisenbug.
Additionally, there's a reference to "arraysize" that doesn't
exist in fetchmany() and there seem to be no tests that exercise
this for any DBAPI which is also a major bug to be fixed.
Gord Thompson [Mon, 7 Dec 2020 23:37:29 +0000 (18:37 -0500)]
Implement `TypeEngine.as_generic`
Added :meth:`_types.TypeEngine.as_generic` to map dialect-specific types,
such as :class:`sqlalchemy.dialects.mysql.INTEGER`, with the "best match"
generic SQLAlchemy type, in this case :class:`_types.Integer`. Pull
request courtesy Andrew Hannigan.
Abstract away how we check for "overridden methods" so it is more
clear what the intent is and that the methodology can be
independently tested.
The SQLAlchemy async mode now detects and raises an informative
error when an non asyncio compatible :term:`DBAPI` is used.
Using a standard ``DBAPI`` with async SQLAlchemy will cause
it to block like any sync call, interrupting the executing asyncio
loop.
Mike Bayer [Mon, 7 Dec 2020 18:40:48 +0000 (13:40 -0500)]
fix non-ascii quotes in sqlite on update feature
Fixed a SQLite source file that had non-ascii characters inside of its
docstring without a source encoding, introduced within the "INSERT..ON
CONFLICT" feature, which would cause failures under Python 2.
Mike Bayer [Thu, 3 Dec 2020 16:17:08 +0000 (11:17 -0500)]
Reflect decimal points in MariaDB non-quoted numeric defaults
Fixed issue where reflecting a server default on MariaDB only that
contained a decimal point in the value would fail to be reflected
correctly, leading towards a reflected table that lacked any server
default.
Mike Bayer [Wed, 2 Dec 2020 14:26:18 +0000 (09:26 -0500)]
modernize contains_eager() docs
Along with other loader options that are likely
to require usage of populate_existing(), make sure
contains_eager() documentation makes it absolutely
clear that already-loaded collections are not overwritten.
consolidate contains_eager() documentation into the narrative docs.
Additionally, remove the "arbitrary statements" section, this is
not a useful case and seems to be left over.
Mike Bayer [Wed, 2 Dec 2020 01:55:06 +0000 (20:55 -0500)]
Make sure asyncpgfallback has __slots__
the test suite failed to find that we started accessing
a non-existent slot "_isolation_setting" added by me in 9b779611f9, as the test suite makes use of the
AsyncAdaptFallback_asyncpg_connection subclass, which didn't
include __slots__ and therefore didn't catch that
_isolation_setting wasn't added to slots.
Mike Bayer [Tue, 1 Dec 2020 21:03:35 +0000 (16:03 -0500)]
Pass along other keyword args in _EventsHold.populate
Fixed bug involving the ``restore_load_context`` option of ORM events such
as :meth:`_orm.InstanceEvents.load` such that the flag would not be carried
along to subclasses which were mapped after the event handler were first
established.
Mike Bayer [Thu, 26 Nov 2020 17:10:25 +0000 (12:10 -0500)]
Don't discard leftovers from surface_selectables
Fixed regression introduced in 1.3.2 for the PostgreSQL dialect, also
copied out to the MySQL dialect's feature in 1.3.18, where usage of a non
:class:`_schema.Table` construct such as :func:`_sql.text` as the argument
to :paramref:`_sql.Select.with_for_update.of` would fail to be accommodated
correctly within the PostgreSQL or MySQL compilers.
Brendan McCollam [Thu, 26 Nov 2020 16:04:42 +0000 (11:04 -0500)]
Fix typo in sessionmaker example: s/wih/with
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Fix typo in sessionmaker documentation example: s/wih/with
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 24 Nov 2020 01:13:20 +0000 (20:13 -0500)]
Support Column objects in the SET clause for upsert
Established support for :class:`_schema.Column` objects as well as ORM
instrumented attributes as keys in the ``set_`` dictionary passed to the
:meth:`_postgresql.Insert.on_conflict_do_update` and
:meth:`_sqlite.Insert.on_conflict_do_update` methods, which match to the
:class:`_schema.Column` objects in the ``.c`` collection of the target
:class:`_schema.Table`. Previously, only string column names were
expected; a column expression would be assumed to be an out-of-table
expression that would render fully along with a warning.
Federico Caselli [Tue, 10 Nov 2020 22:13:55 +0000 (23:13 +0100)]
Add summary table for orm 2.0 migration
A new formatting option is added to zzzeeksphinx
to allow the table to slide for horizontal overflow
without blowing up the page. overall though trying
to keep the table narrow as the code examples are side by
side, other configurations didn't work as well.
Jordan Pittier [Wed, 18 Nov 2020 14:57:43 +0000 (09:57 -0500)]
QueuePool: support subsecond timeout
Fixes: #5719
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Make it explicit in the documentation and in the default value for the 'timeout'
parameter that `timeout` can be a float. Because Python timing is not
very accurate, warn about the precision.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Fri, 20 Nov 2020 15:12:18 +0000 (10:12 -0500)]
Don't emit warnings on descriptor access
This commit is revising 5162f2bc5fc0ac239f26a76fc9f0c2, which
when I did it felt a little rushed but I couldn't find anything
wrong. Well here we are :).
Fixed issue where a :class:`.RemovedIn20Warning` would erroneously emit
when the ``.bind`` attribute were accessed internally on objects,
particularly when stringifying a SQL construct.
Alter the deprecated() decorator so that we can use it just to add
docstring warnings but not actually warn when the function is
accessed, adding new argument enable_warnings that can be
set to False.
Added a safety feature to deprecated_20() that will disallow an
":attr:" from proceeding if enable_warnings=False isn't present,
unless there's an extra flag
warn_on_attribute_access, since we want Session.transaction to
emit a deprecation warning. This is a little hacky but it's essentially
modifying the decorator to require a positive assertion that
a deprecation decorator on a descriptor should actually warn
on access.
Remove the warning filter for session.transaction and get
tests to pass to ensure this is not also being called
internally.
Added tests to ensure that common places .bind can be passed
as a parameter definitely warn as I was not able to find
this otherwise.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
