Mike Bayer [Fri, 21 Aug 2020 18:44:04 +0000 (14:44 -0400)]
make URL immutable
it's not really correct that URL is mutable and doesn't do
any argument checking. propose replacing it with an immutable
named tuple with rich copy-and-mutate methods.
At the moment this makes a hard change to the CreateEnginePlugin
docs that previously recommended url.query.pop(). I can't find
any plugins on github other than my own that are using this
feature, so see if we can just make a hard change on this one.
jonathan vanasco [Mon, 20 Apr 2020 17:56:49 +0000 (13:56 -0400)]
More descriptive error for non-mapped string prop name
Fixed issue where using a loader option against a string attribute name
that is not actually a mapped attribute, such as a plain Python descriptor,
would raise an uninformative AttributeError; a descriptive error is now
raised.
Federico Caselli [Fri, 21 Aug 2020 22:30:44 +0000 (00:30 +0200)]
Updates for MariaDB sequences
MariaDB should not run a Sequence if it has optional=True.
Additionally, rework the rules in crud.py to accommodate the
new combination MariaDB brings us, which is a dialect
that supports both cursor.lastrowid, explicit sequences,
*and* no support for returning.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Fixes: #5528
Change-Id: I9a8ea69a34983affa95dfd22186e2908fdf0d58c
Mike Bayer [Wed, 19 Aug 2020 16:08:26 +0000 (12:08 -0400)]
normalize execute style for events, 2.0
The _execute_20 and exec_driver_sql methods should wrap
up the parameters so that they represent the single list / single
dictionary style of invocation into the legacy methods. then
the before_ after_ execute event handlers should be receiving
the parameter dictionary as a single dictionary. this requires
that we break out distill_params to work differently if event
handlers are present.
additionally, add deprecation warnings for old argument passing
styles.
Gord Thompson [Sat, 1 Aug 2020 21:56:12 +0000 (15:56 -0600)]
Add JSON support for mssql
Added support for the :class:`_types.JSON` datatype on the SQL Server
dialect using the :class:`_mssql.JSON` implementation, which implements SQL
Server's JSON functionality against the ``NVARCHAR(max)`` datatype as per
SQL Server documentation. Implementation courtesy Gord Thompson.
Mike Bayer [Wed, 19 Aug 2020 02:53:09 +0000 (22:53 -0400)]
Implement DDL visitor for PG ENUM with schema translate support
Fixed issue where the :class:`_postgresql.ENUM` type would not consult the
schema translate map when emitting a CREATE TYPE or DROP TYPE during the
test to see if the type exists or not. Additionally, repaired an issue
where if the same enum were encountered multiple times in a single DDL
sequence, the "check" query would run repeatedly rather than relying upon a
cached value.
Federico Caselli [Sat, 30 May 2020 12:45:00 +0000 (14:45 +0200)]
Add support for identity columns
Added the :class:`_schema.Identity` construct that can be used to
configure identity columns rendered with GENERATED { ALWAYS |
BY DEFAULT } AS IDENTITY. Currently the supported backends are
PostgreSQL >= 10, Oracle >= 12 and MSSQL (with different syntax
and a subset of functionalities).
Mike Bayer [Tue, 18 Aug 2020 18:17:06 +0000 (14:17 -0400)]
Deliver straight BinaryExpr w/ no negate for any() / all()
Adjusted the :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT"
operation for negation, rather than negating the comparison operator.
Tony Locke [Sun, 2 Aug 2020 19:19:26 +0000 (15:19 -0400)]
Update dialect for pg8000 version 1.16.0
The pg8000 dialect has been revised and modernized for the most recent
version of the pg8000 driver for PostgreSQL. Changes to the dialect
include:
* All data types are now sent as text rather than binary.
* Using adapters, custom types can be plugged in to pg8000.
* Previously, named prepared statements were used for all statements.
Now unnamed prepared statements are used by default, and named
prepared statements can be used explicitly by calling the
Connection.prepare() method, which returns a PreparedStatement
object.
Pull request courtesy Tony Locke.
Notes by Mike: to get this all working it was needed to break
up JSONIndexType into "str" and "int" subtypes; this will be
needed for any dialect that is dependent on setinputsizes().
also includes @caselit's idea to include query params
in the dbdriver parameter.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com> Closes: #5451
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5451
Pull-request-sha: 639751ca9c7544801b9ede02e6cbe15a16c59c82
Jesse Bakker [Mon, 17 Aug 2020 14:35:35 +0000 (10:35 -0400)]
Make discriminator column used by ConcreteBase configurable
The name of the virtual column used when using the
:class:`_declarative.AbstractConcreteBase` and
:class:`_declarative.ConcreteBase` classes can now be customized, to allow
for models that have a column that is actually named ``type``. Pull
request courtesy Jesse-Bakker.
Mike Bayer [Sat, 15 Aug 2020 19:08:09 +0000 (15:08 -0400)]
Create a real type for Tuple() and handle appropriately in compiler
Improved the :func:`_sql.tuple_` construct such that it behaves predictably
when used in a columns-clause context. The SQL tuple is not supported as a
"SELECT" columns clause element on most backends; on those that do
(PostgreSQL, not surprisingly), the Python DBAPI does not have a "nested
type" concept so there are still challenges in fetching rows for such an
object. Use of :func:`_sql.tuple_` in a :func:`_sql.select` or
:class:`_orm.Query` will now raise a :class:`_exc.CompileError` at the
point at which the :func:`_sql.tuple_` object is seen as presenting itself
for fetching rows (i.e., if the tuple is in the columns clause of a
subquery, no error is raised). For ORM use,the :class:`_orm.Bundle` object
is an explicit directive that a series of columns should be returned as a
sub-tuple per row and is suggested by the error message. Additionally ,the
tuple will now render with parenthesis in all contexts. Previously, the
parenthesization would not render in a columns context leading to
non-defined behavior.
As part of this change, Tuple receives a dedicated datatype
which appears to allow us the very desirable change of removing
the bindparam._expanding_in_types attribute as well as
ClauseList._tuple_values (which might already have not been
needed due to #4645).
RamonWill [Wed, 12 Aug 2020 18:50:20 +0000 (14:50 -0400)]
error message for Lookup
Thr proposed change will provide the user with the target Enum Class name as well as up to four possible enum values when a LookupError is raised in the Enum Class.
A user requested that the enum name and possible values are included to the LookupError message to make debugging easier. The criteria included using ellipses for Enums containing more than four values and using ellipses for enum values that were greater than a certain number of characters (for this resolution the limit is 11 characters).
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Fri, 14 Aug 2020 04:58:56 +0000 (00:58 -0400)]
Bump minimum MySQL version to 5.0.2; use all-numeric server version
MySQL dialect's server_version_info tuple is now all numeric. String
tokens like "MariaDB" are no longer present so that numeric comparison
works in all cases. The .is_mariadb flag on the dialect should be
consulted for whether or not mariadb was detected. Additionally removed
structures meant to support extremely old MySQL versions 3.x and 4.x;
the minimum MySQL version supported is now version 5.0.2.
In addition, as the "MariaDB" name goes away from server version,
expand upon the change in I330815ebe572b6a9818377da56621397335fa702
to support the name "mariadb" throughout the dialect and test suite
when mariadb-only mode is used. This changes the "name" field
on the MariaDB dialect to "mariadb", which then implies a change
throughout the testing requirements system as well as all the
dialect-specific DDL argument names such as "mysql_engine" is
now specified as "mariadb_engine", etc. Make use of the
recent additions to test suite URL provisioning so that we can
force MariaDB databases to have a "mariadb-only" dialect which
allows us to test this name change fully.
Update documentation to refer to MySQL / MariaDB explicitly
as well as indicating the "mariadb_" prefix used for options.
It seems likely that MySQL and MariaDB version numbers are going to
start colliding at some point so having the "mariadb" name
be available as a totally separate dialect name should give us
some options in this regard.
Currently also includes a date related fix to a test for
the postgresql dialect that was implicitly assuming a
non-UTC timezone
Mike Bayer [Fri, 14 Aug 2020 16:07:14 +0000 (12:07 -0400)]
Provision on different drivers dynamically
We want TOX_POSTGRESQL and similar to be the fixed variable
that is configured from CI environment. These variables should refer
to database servers but individual drivers like asyncpg mysqlconnector
etc. should come from local tox.ini. add a new system to generate
per-driver URLs from a simple list of hostname-based URLs delivered
from CI environment.
Mike Bayer [Fri, 14 Aug 2020 02:44:33 +0000 (22:44 -0400)]
Use TOX_POSTGRESQL_MASTER for master
since 1.3 doesn't have updated pg8000 support nor
asyncpg support it needs to stay on the previous
TOX_POSTGRESQL variable, so make a new one that will
have the newer drivers in it.
We can now create asyncio endpoints that are then handled
in "implicit IO" form within the majority of the Core internals.
Then coroutines are re-exposed at the point at which we call
into asyncpg methods.
Patch includes:
* asyncpg dialect
* asyncio package
* engine, result, ORM session classes
* new test fixtures, tests
* some work with pep-484 and a short plugin for the
pyannotate package, which seems to have so-so results
RamonWill [Thu, 13 Aug 2020 21:23:16 +0000 (17:23 -0400)]
Raise UnmappedInstanceError if the attribute of a database object is an unmapped object.
The proposed change will raise an UnmappedInstanceError instead of an attribute error if an instance an Instrumented Attribute is unmapped.
### Description
If a user tries to access an attribute of a database object that is unmapped then an error is raised. The reason for this is because the __get__ descriptor uses instance_state(instance) which uses the operator.attrgetter method to see if the object in question has the attribute "_sa_instance_state" that mapped objects have. If it doesn't contain this attribute it means that the object is unmapped.
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
RamonWill [Thu, 13 Aug 2020 21:04:13 +0000 (17:04 -0400)]
Test execution_options on Query object before compilation
A test that checks if the execution options are being set on the Query object before compilation
### Description
Since Issue #4670 has been open there have changes to the execution mechanics that resolved it. I have just created a test case that will enable the issue to be closed.
This pull request is:
- [ X] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Thu, 13 Aug 2020 14:43:53 +0000 (10:43 -0400)]
Further fixes for ticket 5470
The fix for #5470 didn't actually take into account that
the "distinct" logic in query was also doubling up the criteria.
Added many more tests. the 1.3 version here will be different
than 1.4 as the regression is not quite the same.
Mike Bayer [Wed, 12 Aug 2020 22:46:25 +0000 (18:46 -0400)]
Use importlib_metadata; add namespace for mariadb
The ``importlib_metadata`` library is used to scan for setuptools
entrypoints rather than pkg_resources. as importlib_metadata is a small
library that is included as of Python 3.8, the compatibility library is
installed as a dependency for Python versions older than 3.8.
Unfortunately setuptools "attr:" is broken because it tries to import
the module; seems like this is fixed as part of
https://github.com/pypa/setuptools/pull/1753 however this is too recent
to rely upon for now.
Added a new dialect token "mariadb" that may be used in place of "mysql" in
the :func:`_sa.create_engine` URL. This will deliver a MariaDB dialect
subclass of the MySQLDialect in use that forces the "is_mariadb" flag to
True. The dialect will raise an error if a server version string that does
not indicate MariaDB in use is received. This is useful for
MariaDB-specific testing scenarios as well as to support applications that
are hardcoding to MariaDB-only concepts. As MariaDB and MySQL featuresets
and usage patterns continue to diverge, this pattern may become more
prominent.
Mike Bayer [Thu, 13 Aug 2020 00:14:15 +0000 (20:14 -0400)]
Sweep through UPDATE ordered_values a second time
The fix in 180ae7c1a53385f72b0047496ac001ec5099cc3e
didn't do much as the code was not preserving parameter
order at all, in fact. Reworked stmt_parameters to be
delivered in the correct order up front and preserve
throughout crud.py which was not being done at all
before.
Mike Bayer [Mon, 10 Aug 2020 22:40:49 +0000 (18:40 -0400)]
Break scalars() and mappings() into separate objects
The issue of Result.fetchXXX() methods returning Row
objects unless filtering is applied will not provide a
clear enough API story when type annotations are applied,
so break out scalars/mappings into separate wrapper objects.
this makes some things more intuitive and other things a little
more bumpy. however the return type story is now clearer.
Mike Bayer [Tue, 11 Aug 2020 21:46:11 +0000 (17:46 -0400)]
Correct for update.ordered_values() and use in new test
The test for the parameter fix in c0685e5f41 was not working
deterministically on Python 2, so use ordered_values(), however
a second issue in ordered_values() was assuming each element was a
column, so also test for array-assignment expressions with
ordered_values.
Mike Bayer [Sat, 8 Aug 2020 17:03:17 +0000 (13:03 -0400)]
render INSERT/UPDATE column expressions up front; pass state
Fixes related to rendering of complex UPDATE DML
which was not correctly preserving positional parameter
order in conjunction with DML features that are only known
to work on the PostgreSQL database. Both pg8000
and asyncpg use positional parameters which is why these
issues are suddenly apparent.
crud.py now takes on the task of rendering the column
expressions for SET or VALUES so that for the very unusual
case that the column expression is a compound expression
that includes a bound parameter (namely an array index),
the bound parameter order is preserved.
Additionally, crud.py passes through the positional_names
keyword argument into bindparam_string() which is necessary
when CTEs are being rendered, as PG supports complex
CTE / INSERT / UPDATE scenarios.
Mike Bayer [Fri, 7 Aug 2020 18:51:33 +0000 (14:51 -0400)]
Don't link on_connect to first_connect event handler
Adjusted the dialect initialization process such that the
:meth:`_engine.Dialect.on_connect` is not called a second time on the first
connection. The hook is called first, then the
:meth:`_engine.Dialect.initialize` is called if that connection is the
first for that dialect, then no more events are called. This eliminates
the two calls to the "on_connect" function which can produce very difficult
debugging situations.
Mike Bayer [Fri, 7 Aug 2020 15:24:29 +0000 (11:24 -0400)]
Pass schema_translate_map from DDLCompiler to SQLCompiler
Fixed issue where the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would not take effect when the :meth:`_schema.Sequence.next_value`
function function for a :class:`_schema.Sequence` were used in the
:paramref:`_schema.Column.server_default` parameter and the create table
DDL were emitted.
Mike Bayer [Thu, 6 Aug 2020 19:53:17 +0000 (15:53 -0400)]
base all_orm_descriptors ordering on cls.__dict__ + cls.__mro__
Adjusted the workings of the :meth:`_orm.Mapper.all_orm_descriptors`
accessor to represent the attributes in the order that they are located in
a deterministic way, assuming the use of Python 3.6 or higher which
maintains the sorting order of class attributes based on how they were
declared. This sorting is not guaranteed to match the declared order of
attributes in all cases however; see the method documentation for the exact
scheme.
Mike Bayer [Sun, 28 Jun 2020 15:59:34 +0000 (11:59 -0400)]
Documentation updates for 1.4
* major additions to 1.4 migration doc; removed additional
verbosity regarding caching methodology and reorganized the
doc to present itself more as a "what's changed" guide
* as we now have a path for asyncio, update that doc so that
we aren't spreading obsolete information
* updates to the 2.0 migration guide with latest info, however
this is still an architecture doc and not a migration guide
yet, will need further rework.
* start really talking about 1.x vs. 2.0 style everywhere. Querying
is most of the docs so this is going to be a prominent
theme, start getting it to fit in
* Add introductory documentation for ORM example sections as these
are too sparse
* new documentation for do_orm_execute(), many separate sections,
adding deprecation notes to before_compile() and similar
* new example suites to illustrate do_orm_execute(),
with_loader_criteria()
* modernized horizontal sharding examples and added a separate
example to distinguish between multiple databases and single
database w/ multiple tables use case
* introducing DEEP ALCHEMY, will use zzzeeksphinx 1.1.6
* no name for the alchemist yet however the dragon's name
is Flambé
Mike Bayer [Thu, 6 Aug 2020 01:47:43 +0000 (21:47 -0400)]
Implement relationship AND criteria; global loader criteria
Added the ability to add arbitrary criteria to the ON clause generated
by a relationship attribute in a query, which applies to methods such
as :meth:`_query.Query.join` as well as loader options like
:func:`_orm.joinedload`. Additionally, a "global" version of the option
allows limiting criteria to be applied to particular entities in
a query globally.
Documentation is minimal at this point, new examples will
be coming in a subsequent commit.
Some adjustments to execution options in how they are represented
in the ORMExecuteState as well as well as a few ORM tests that
forgot to get merged in a preceding commit.
Mike Bayer [Thu, 6 Aug 2020 00:32:02 +0000 (20:32 -0400)]
Fixes for MySQL 8
MySQL 8.0.19 has some changes to how it reports on display widths
for int types (seems like it omits it in some cases), and also
no longer cares about the length for YEAR. Another adjustment
to the ordering of constraints reported in one case also.
At least one CI machine is at 8.0.21 now.
Mike Bayer [Wed, 5 Aug 2020 21:58:48 +0000 (17:58 -0400)]
Convert lazy loader, selectinload, load_on_ident to lambda statements
Building on newly robust lambdas in
I29a513c98917b1d503abfdd61e6b6e8800851aa8,
convert key loading off of the "baked" system so that baked
is no longer used by the ORM.
Mike Bayer [Wed, 5 Aug 2020 20:42:26 +0000 (16:42 -0400)]
Robustness for lambdas, lambda statements
in order to accommodate relationship loaders
with lambda caching, a lot more is needed. This is
a full refactor of the lambda system such that it
now has two levels of caching; the first level caches what
can be known from the __code__ element, then the next level
of caching is against the lambda itself and the contents
of __closure__. This allows for the elements inside
the lambdas, like columns and entities, to change and
then be part of the cache key. Lazy/selectinloads' use of
baked queries had to add distinct cache key elements,
which was attempted here but overall things needed to be
more robust than that.
This commit is broken out from the very long and sprawling
commit at Id6b5c03b1ce9ddb7b280f66792212a0ef0a1c541 .
Mike Bayer [Sat, 1 Aug 2020 17:57:04 +0000 (13:57 -0400)]
Genericize str() for types
Remove lookup logic that attempts to locate a dialect for a type,
just use StrSQLTypeCompiler.
Cleaned up the internal ``str()`` for datatypes so that all types produce a
string representation without any dialect present, including that it works
for third-party dialect types without that dialect being present. The
string representation defaults to being the UPPERCASE name of that type
with nothing else.
Mike Bayer [Sat, 1 Aug 2020 19:05:53 +0000 (15:05 -0400)]
Establish future behavior for Session cascade backrefs, bind
The behavior of the :paramref:`_orm.relationship.cascade_backrefs` flag
will be reversed in 2.0 and set to ``False`` unconditionally, such that
backrefs don't cascade save-update operations from a forwards-assignment to
a backwards assignment. A 2.0 deprecation warning is emitted when the
parameter is left at its default of ``True`` at the point at which such a
cascade operation actually takes place. The new behavior can be
established as always by setting the flag to ``False`` on a specific
:func:`_orm.relationship`, or more generally can be set up across the board
by setting the the :paramref:`_orm.Session.future` flag to True.
Additionally in the interests of expediency, this commit will also
move Session away from making use of bound metadata if the future=True
flag is set. An application that sets future=True should ideally
have to change as little else as possible for full 2.0 behavior.
Mike Bayer [Sat, 9 May 2020 14:55:19 +0000 (10:55 -0400)]
Unpin pydocstyle, minimum for flake8-docstrings 1.3.1
Per https://gitlab.com/pycqa/flake8-docstrings/-/issues/36#note_199635329
flake8-docstrings has repaired the pydocstyle issue as of
1.3.1, so unpin pydocstyle.
Mike Bayer [Tue, 28 Jul 2020 20:46:53 +0000 (16:46 -0400)]
Consider default FROM DUAL for MySQL
MySQL claims it doesn't require FROM DUAL for no
FROM clause even though the issue at #5481 locates
a case which requires one. See if FROM DUAL the same
way as Oracle without attempting to guess is potentially
feasible.
- in create wheel set minimum versions of setuptools and wheel to avoid
failure in python 3.5 with metadata configured in setup.cfg
- update action versions
- test also cext in the pull requests
Jim Bosch [Sun, 26 Jul 2020 20:50:14 +0000 (16:50 -0400)]
Ensure is_comparison passed for PG RANGE op() methods
Fixed issue where the return type for the various RANGE comparison
operators would itself be the same RANGE type rather than BOOLEAN, which
would cause an undesirable result in the case that a
:class:`.TypeDecorator` that defined result-processing behavior were in
use. Pull request courtesy Jim Bosch.
Mike Bayer [Sun, 26 Jul 2020 17:17:56 +0000 (13:17 -0400)]
Improve delete cascade and passive deletes sections
Add cross linking from Core to ORM and also add new sections
and examples regarding many-to-many. Move the section
out of "collections" and into the "cascades" chapter
where it's more likely to be found.
PyODBCConnector.initialize just super from Connector, no need to redeclare, so I guess that's why they are commented before.
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
Remove the useless comment code lines.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
