Václav Klusák [Mon, 17 Aug 2020 15:58:56 +0000 (11:58 -0400)]
Add support for classical mapping of dataclasses
Added support for direct mapping of Python classes that are defined using
the Python ``dataclasses`` decorator. See the section
:ref:`mapping_dataclasses` for background. Pull request courtesy Václav
Klusák.
RamonWill [Tue, 25 Aug 2020 00:14:15 +0000 (20:14 -0400)]
Provide a more detailed error message for Query.join()
An :class:`.ArgumentError` with more detail is now raised if the target
parameter for :meth:`_query.Query.join` is set to an unmapped object.
Prior to this change a less detailed ``AttributeError`` was raised.
Pull request courtesy Ramon Williams.
Fantix King [Sat, 29 Aug 2020 16:37:34 +0000 (12:37 -0400)]
Fix a mis-reference in create_async_engine().
`AsyncMethodRequired` is actually from
`sqlalchemy.ext.asyncio.exc`, so here it
should be referenced as `async_exc.AsyncMethodRequired`,
instead of `exc.AsyncMethodRequired`.
RamonWill [Sun, 30 Aug 2020 13:05:06 +0000 (09:05 -0400)]
Include PostgreSQL in trigger test and correct documentation example
Include PostgreSQL dialect in trigger test and correct DDL example in documentation
A user highlighted that the syntax in the DDL trigger example was incorrect for PostgreSQL. The trigger tests where also skipping the PostgreSQL dialect until the syntax was corrected. This PR fixes both of these issues.
This pull request is:
- [X ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Sun, 30 Aug 2020 22:13:36 +0000 (18:13 -0400)]
Support extra / single inh criteria with ORM update/delete
The ORM bulk update and delete operations, historically available via the
:meth:`_orm.Query.update` and :meth:`_orm.Query.delete` methods as well as
via the :class:`_dml.Update` and :class:`_dml.Delete` constructs for
:term:`2.0 style` execution, will now automatically accommodate for the
additional WHERE criteria needed for a single-table inheritance
discrminiator. Joined-table inheritance is still not directly
supported. The new :func:`_orm.with_loader_criteria` construct is also
supported for all mappings with bulk update/delete.
Mike Bayer [Fri, 28 Aug 2020 21:56:43 +0000 (17:56 -0400)]
See if the future is here
The docs are going to talk a lot about session.execute(select())
for ORM queries, and additionally it's much easier to help
users with queries and such if we can use this new syntax.
I'm hoping to see how hard it is to get a unified tutorial
started that switches to new syntax. Basically, new syntax
is much easier to explain and less buggy. But, if we
are starting to present new syntax with the explicit goal
of being easier to explain for less experienced programmers,
the "future" thing is going to just be an impediment
to that.
See if we can remove "future" from session.execute(),
so that ORM-enabled select() statements return ORM results
at that level. This does not change the presence of the
"future" flag for the Session's construction and for its
transactional behaviors.
The only perceptible change of the future flag for
session.execute() is that session.execute(select()) where the
statement has ORM entities in it now returns ORM new
style tuples rather than old style tuples. Like
mutating a URL, it's hopefully not very common that people
are doing this.
Gord Thompson [Fri, 21 Aug 2020 16:29:29 +0000 (10:29 -0600)]
Emit v2.0 deprecation warning for "implicit autocommit"
"Implicit autocommit", which is the COMMIT that occurs when a DML or DDL
statement is emitted on a connection, is deprecated and won't be part of
SQLAlchemy 2.0. A 2.0-style warning is emitted when autocommit takes
effect, so that the calling code may be adjusted to use an explicit
transaction.
As part of this change, DDL methods such as
:meth:`_schema.MetaData.create_all` when used against a
:class:`_engine.Engine` or :class:`_engine.Connection` will run the
operation in a BEGIN block if one is not started already.
The MySQL and MariaDB dialects now query from the information_schema.tables
system view in order to determine if a particular table exists or not.
Previously, the "DESCRIBE" command was used with an exception catch to
detect non-existent, which would have the undesirable effect of emitting a
ROLLBACK on the connection. There appeared to be legacy encoding issues
which prevented the use of "SHOW TABLES", for this, but as MySQL support is
now at 5.0.2 or above due to :ticket:`4189`, the information_schema tables
are now available in all cases.
Federico Caselli [Thu, 20 Aug 2020 20:42:28 +0000 (22:42 +0200)]
Update authors file
Add all authors with more than 10 commits. The email are
omitted to avoid possible spam (even if they
are not private since they are in the git history, they
are at least not grep friendly)
Output obtained by `git shortlog --summary | sort -r`
Add support for regular expression on supported backend.
Two operations have been defined:
* :meth:`~.ColumnOperators.regexp_match` implementing a regular
expression match like function.
* :meth:`~.ColumnOperators.regexp_replace` implementing a regular
expression string replace function.
Mike Bayer [Thu, 27 Aug 2020 19:50:47 +0000 (15:50 -0400)]
Raise NotImplemenedError for association proxy __clause_element__
It's not possible right now to use an association proxy element as a plain
column expression to be SELECTed from or used in a SQL function. An
informative error is now raised when this occurs.
Mike Bayer [Fri, 21 Aug 2020 18:44:04 +0000 (14:44 -0400)]
make URL immutable
it's not really correct that URL is mutable and doesn't do
any argument checking. propose replacing it with an immutable
named tuple with rich copy-and-mutate methods.
At the moment this makes a hard change to the CreateEnginePlugin
docs that previously recommended url.query.pop(). I can't find
any plugins on github other than my own that are using this
feature, so see if we can just make a hard change on this one.
jonathan vanasco [Mon, 20 Apr 2020 17:56:49 +0000 (13:56 -0400)]
More descriptive error for non-mapped string prop name
Fixed issue where using a loader option against a string attribute name
that is not actually a mapped attribute, such as a plain Python descriptor,
would raise an uninformative AttributeError; a descriptive error is now
raised.
Federico Caselli [Fri, 21 Aug 2020 22:30:44 +0000 (00:30 +0200)]
Updates for MariaDB sequences
MariaDB should not run a Sequence if it has optional=True.
Additionally, rework the rules in crud.py to accommodate the
new combination MariaDB brings us, which is a dialect
that supports both cursor.lastrowid, explicit sequences,
*and* no support for returning.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #5528
Change-Id: I9a8ea69a34983affa95dfd22186e2908fdf0d58c
Mike Bayer [Wed, 19 Aug 2020 16:08:26 +0000 (12:08 -0400)]
normalize execute style for events, 2.0
The _execute_20 and exec_driver_sql methods should wrap
up the parameters so that they represent the single list / single
dictionary style of invocation into the legacy methods. then
the before_ after_ execute event handlers should be receiving
the parameter dictionary as a single dictionary. this requires
that we break out distill_params to work differently if event
handlers are present.
additionally, add deprecation warnings for old argument passing
styles.
Gord Thompson [Sat, 1 Aug 2020 21:56:12 +0000 (15:56 -0600)]
Add JSON support for mssql
Added support for the :class:`_types.JSON` datatype on the SQL Server
dialect using the :class:`_mssql.JSON` implementation, which implements SQL
Server's JSON functionality against the ``NVARCHAR(max)`` datatype as per
SQL Server documentation. Implementation courtesy Gord Thompson.
Mike Bayer [Wed, 19 Aug 2020 02:53:09 +0000 (22:53 -0400)]
Implement DDL visitor for PG ENUM with schema translate support
Fixed issue where the :class:`_postgresql.ENUM` type would not consult the
schema translate map when emitting a CREATE TYPE or DROP TYPE during the
test to see if the type exists or not. Additionally, repaired an issue
where if the same enum were encountered multiple times in a single DDL
sequence, the "check" query would run repeatedly rather than relying upon a
cached value.
Federico Caselli [Sat, 30 May 2020 12:45:00 +0000 (14:45 +0200)]
Add support for identity columns
Added the :class:`_schema.Identity` construct that can be used to
configure identity columns rendered with GENERATED { ALWAYS |
BY DEFAULT } AS IDENTITY. Currently the supported backends are
PostgreSQL >= 10, Oracle >= 12 and MSSQL (with different syntax
and a subset of functionalities).
Mike Bayer [Tue, 18 Aug 2020 18:17:06 +0000 (14:17 -0400)]
Deliver straight BinaryExpr w/ no negate for any() / all()
Adjusted the :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT"
operation for negation, rather than negating the comparison operator.
Tony Locke [Sun, 2 Aug 2020 19:19:26 +0000 (15:19 -0400)]
Update dialect for pg8000 version 1.16.0
The pg8000 dialect has been revised and modernized for the most recent
version of the pg8000 driver for PostgreSQL. Changes to the dialect
include:
* All data types are now sent as text rather than binary.
* Using adapters, custom types can be plugged in to pg8000.
* Previously, named prepared statements were used for all statements.
Now unnamed prepared statements are used by default, and named
prepared statements can be used explicitly by calling the
Connection.prepare() method, which returns a PreparedStatement
object.
Pull request courtesy Tony Locke.
Notes by Mike: to get this all working it was needed to break
up JSONIndexType into "str" and "int" subtypes; this will be
needed for any dialect that is dependent on setinputsizes().
also includes @caselit's idea to include query params
in the dbdriver parameter.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #5451
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5451
Pull-request-sha: 639751ca9c7544801b9ede02e6cbe15a16c59c82
Jesse Bakker [Mon, 17 Aug 2020 14:35:35 +0000 (10:35 -0400)]
Make discriminator column used by ConcreteBase configurable
The name of the virtual column used when using the
:class:`_declarative.AbstractConcreteBase` and
:class:`_declarative.ConcreteBase` classes can now be customized, to allow
for models that have a column that is actually named ``type``. Pull
request courtesy Jesse-Bakker.
Mike Bayer [Sat, 15 Aug 2020 19:08:09 +0000 (15:08 -0400)]
Create a real type for Tuple() and handle appropriately in compiler
Improved the :func:`_sql.tuple_` construct such that it behaves predictably
when used in a columns-clause context. The SQL tuple is not supported as a
"SELECT" columns clause element on most backends; on those that do
(PostgreSQL, not surprisingly), the Python DBAPI does not have a "nested
type" concept so there are still challenges in fetching rows for such an
object. Use of :func:`_sql.tuple_` in a :func:`_sql.select` or
:class:`_orm.Query` will now raise a :class:`_exc.CompileError` at the
point at which the :func:`_sql.tuple_` object is seen as presenting itself
for fetching rows (i.e., if the tuple is in the columns clause of a
subquery, no error is raised). For ORM use,the :class:`_orm.Bundle` object
is an explicit directive that a series of columns should be returned as a
sub-tuple per row and is suggested by the error message. Additionally ,the
tuple will now render with parenthesis in all contexts. Previously, the
parenthesization would not render in a columns context leading to
non-defined behavior.
As part of this change, Tuple receives a dedicated datatype
which appears to allow us the very desirable change of removing
the bindparam._expanding_in_types attribute as well as
ClauseList._tuple_values (which might already have not been
needed due to #4645).
RamonWill [Wed, 12 Aug 2020 18:50:20 +0000 (14:50 -0400)]
error message for Lookup
Thr proposed change will provide the user with the target Enum Class name as well as up to four possible enum values when a LookupError is raised in the Enum Class.
A user requested that the enum name and possible values are included to the LookupError message to make debugging easier. The criteria included using ellipses for Enums containing more than four values and using ellipses for enum values that were greater than a certain number of characters (for this resolution the limit is 11 characters).
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Fri, 14 Aug 2020 04:58:56 +0000 (00:58 -0400)]
Bump minimum MySQL version to 5.0.2; use all-numeric server version
MySQL dialect's server_version_info tuple is now all numeric. String
tokens like "MariaDB" are no longer present so that numeric comparison
works in all cases. The .is_mariadb flag on the dialect should be
consulted for whether or not mariadb was detected. Additionally removed
structures meant to support extremely old MySQL versions 3.x and 4.x;
the minimum MySQL version supported is now version 5.0.2.
In addition, as the "MariaDB" name goes away from server version,
expand upon the change in I330815ebe572b6a9818377da56621397335fa702
to support the name "mariadb" throughout the dialect and test suite
when mariadb-only mode is used. This changes the "name" field
on the MariaDB dialect to "mariadb", which then implies a change
throughout the testing requirements system as well as all the
dialect-specific DDL argument names such as "mysql_engine" is
now specified as "mariadb_engine", etc. Make use of the
recent additions to test suite URL provisioning so that we can
force MariaDB databases to have a "mariadb-only" dialect which
allows us to test this name change fully.
Update documentation to refer to MySQL / MariaDB explicitly
as well as indicating the "mariadb_" prefix used for options.
It seems likely that MySQL and MariaDB version numbers are going to
start colliding at some point so having the "mariadb" name
be available as a totally separate dialect name should give us
some options in this regard.
Currently also includes a date related fix to a test for
the postgresql dialect that was implicitly assuming a
non-UTC timezone
Mike Bayer [Fri, 14 Aug 2020 16:07:14 +0000 (12:07 -0400)]
Provision on different drivers dynamically
We want TOX_POSTGRESQL and similar to be the fixed variable
that is configured from CI environment. These variables should refer
to database servers but individual drivers like asyncpg mysqlconnector
etc. should come from local tox.ini. add a new system to generate
per-driver URLs from a simple list of hostname-based URLs delivered
from CI environment.
Mike Bayer [Fri, 14 Aug 2020 02:44:33 +0000 (22:44 -0400)]
Use TOX_POSTGRESQL_MASTER for master
since 1.3 doesn't have updated pg8000 support nor
asyncpg support it needs to stay on the previous
TOX_POSTGRESQL variable, so make a new one that will
have the newer drivers in it.
We can now create asyncio endpoints that are then handled
in "implicit IO" form within the majority of the Core internals.
Then coroutines are re-exposed at the point at which we call
into asyncpg methods.
Patch includes:
* asyncpg dialect
* asyncio package
* engine, result, ORM session classes
* new test fixtures, tests
* some work with pep-484 and a short plugin for the
pyannotate package, which seems to have so-so results
RamonWill [Thu, 13 Aug 2020 21:23:16 +0000 (17:23 -0400)]
Raise UnmappedInstanceError if the attribute of a database object is an unmapped object.
The proposed change will raise an UnmappedInstanceError instead of an attribute error if an instance an Instrumented Attribute is unmapped.
### Description
If a user tries to access an attribute of a database object that is unmapped then an error is raised. The reason for this is because the __get__ descriptor uses instance_state(instance) which uses the operator.attrgetter method to see if the object in question has the attribute "_sa_instance_state" that mapped objects have. If it doesn't contain this attribute it means that the object is unmapped.
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
RamonWill [Thu, 13 Aug 2020 21:04:13 +0000 (17:04 -0400)]
Test execution_options on Query object before compilation
A test that checks if the execution options are being set on the Query object before compilation
### Description
Since Issue #4670 has been open there have changes to the execution mechanics that resolved it. I have just created a test case that will enable the issue to be closed.
This pull request is:
- [ X] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Thu, 13 Aug 2020 14:43:53 +0000 (10:43 -0400)]
Further fixes for ticket 5470
The fix for #5470 didn't actually take into account that
the "distinct" logic in query was also doubling up the criteria.
Added many more tests. the 1.3 version here will be different
than 1.4 as the regression is not quite the same.
Mike Bayer [Wed, 12 Aug 2020 22:46:25 +0000 (18:46 -0400)]
Use importlib_metadata; add namespace for mariadb
The ``importlib_metadata`` library is used to scan for setuptools
entrypoints rather than pkg_resources. as importlib_metadata is a small
library that is included as of Python 3.8, the compatibility library is
installed as a dependency for Python versions older than 3.8.
Unfortunately setuptools "attr:" is broken because it tries to import
the module; seems like this is fixed as part of
https://github.com/pypa/setuptools/pull/1753 however this is too recent
to rely upon for now.
Added a new dialect token "mariadb" that may be used in place of "mysql" in
the :func:`_sa.create_engine` URL. This will deliver a MariaDB dialect
subclass of the MySQLDialect in use that forces the "is_mariadb" flag to
True. The dialect will raise an error if a server version string that does
not indicate MariaDB in use is received. This is useful for
MariaDB-specific testing scenarios as well as to support applications that
are hardcoding to MariaDB-only concepts. As MariaDB and MySQL featuresets
and usage patterns continue to diverge, this pattern may become more
prominent.
Mike Bayer [Thu, 13 Aug 2020 00:14:15 +0000 (20:14 -0400)]
Sweep through UPDATE ordered_values a second time
The fix in 180ae7c1a53385f72b0047496ac001ec5099cc3e
didn't do much as the code was not preserving parameter
order at all, in fact. Reworked stmt_parameters to be
delivered in the correct order up front and preserve
throughout crud.py which was not being done at all
before.
Mike Bayer [Mon, 10 Aug 2020 22:40:49 +0000 (18:40 -0400)]
Break scalars() and mappings() into separate objects
The issue of Result.fetchXXX() methods returning Row
objects unless filtering is applied will not provide a
clear enough API story when type annotations are applied,
so break out scalars/mappings into separate wrapper objects.
this makes some things more intuitive and other things a little
more bumpy. however the return type story is now clearer.
Mike Bayer [Tue, 11 Aug 2020 21:46:11 +0000 (17:46 -0400)]
Correct for update.ordered_values() and use in new test
The test for the parameter fix in c0685e5f41 was not working
deterministically on Python 2, so use ordered_values(), however
a second issue in ordered_values() was assuming each element was a
column, so also test for array-assignment expressions with
ordered_values.
Mike Bayer [Sat, 8 Aug 2020 17:03:17 +0000 (13:03 -0400)]
render INSERT/UPDATE column expressions up front; pass state
Fixes related to rendering of complex UPDATE DML
which was not correctly preserving positional parameter
order in conjunction with DML features that are only known
to work on the PostgreSQL database. Both pg8000
and asyncpg use positional parameters which is why these
issues are suddenly apparent.
crud.py now takes on the task of rendering the column
expressions for SET or VALUES so that for the very unusual
case that the column expression is a compound expression
that includes a bound parameter (namely an array index),
the bound parameter order is preserved.
Additionally, crud.py passes through the positional_names
keyword argument into bindparam_string() which is necessary
when CTEs are being rendered, as PG supports complex
CTE / INSERT / UPDATE scenarios.
Mike Bayer [Fri, 7 Aug 2020 18:51:33 +0000 (14:51 -0400)]
Don't link on_connect to first_connect event handler
Adjusted the dialect initialization process such that the
:meth:`_engine.Dialect.on_connect` is not called a second time on the first
connection. The hook is called first, then the
:meth:`_engine.Dialect.initialize` is called if that connection is the
first for that dialect, then no more events are called. This eliminates
the two calls to the "on_connect" function which can produce very difficult
debugging situations.
Mike Bayer [Fri, 7 Aug 2020 15:24:29 +0000 (11:24 -0400)]
Pass schema_translate_map from DDLCompiler to SQLCompiler
Fixed issue where the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would not take effect when the :meth:`_schema.Sequence.next_value`
function function for a :class:`_schema.Sequence` were used in the
:paramref:`_schema.Column.server_default` parameter and the create table
DDL were emitted.
Mike Bayer [Thu, 6 Aug 2020 19:53:17 +0000 (15:53 -0400)]
base all_orm_descriptors ordering on cls.__dict__ + cls.__mro__
Adjusted the workings of the :meth:`_orm.Mapper.all_orm_descriptors`
accessor to represent the attributes in the order that they are located in
a deterministic way, assuming the use of Python 3.6 or higher which
maintains the sorting order of class attributes based on how they were
declared. This sorting is not guaranteed to match the declared order of
attributes in all cases however; see the method documentation for the exact
scheme.
Mike Bayer [Sun, 28 Jun 2020 15:59:34 +0000 (11:59 -0400)]
Documentation updates for 1.4
* major additions to 1.4 migration doc; removed additional
verbosity regarding caching methodology and reorganized the
doc to present itself more as a "what's changed" guide
* as we now have a path for asyncio, update that doc so that
we aren't spreading obsolete information
* updates to the 2.0 migration guide with latest info, however
this is still an architecture doc and not a migration guide
yet, will need further rework.
* start really talking about 1.x vs. 2.0 style everywhere. Querying
is most of the docs so this is going to be a prominent
theme, start getting it to fit in
* Add introductory documentation for ORM example sections as these
are too sparse
* new documentation for do_orm_execute(), many separate sections,
adding deprecation notes to before_compile() and similar
* new example suites to illustrate do_orm_execute(),
with_loader_criteria()
* modernized horizontal sharding examples and added a separate
example to distinguish between multiple databases and single
database w/ multiple tables use case
* introducing DEEP ALCHEMY, will use zzzeeksphinx 1.1.6
* no name for the alchemist yet however the dragon's name
is Flambé
Mike Bayer [Thu, 6 Aug 2020 01:47:43 +0000 (21:47 -0400)]
Implement relationship AND criteria; global loader criteria
Added the ability to add arbitrary criteria to the ON clause generated
by a relationship attribute in a query, which applies to methods such
as :meth:`_query.Query.join` as well as loader options like
:func:`_orm.joinedload`. Additionally, a "global" version of the option
allows limiting criteria to be applied to particular entities in
a query globally.
Documentation is minimal at this point, new examples will
be coming in a subsequent commit.
Some adjustments to execution options in how they are represented
in the ORMExecuteState as well as well as a few ORM tests that
forgot to get merged in a preceding commit.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
