Mike Bayer [Sat, 8 Aug 2020 17:03:17 +0000 (13:03 -0400)]
render INSERT/UPDATE column expressions up front; pass state
Fixes related to rendering of complex UPDATE DML
which was not correctly preserving positional parameter
order in conjunction with DML features that are only known
to work on the PostgreSQL database. Both pg8000
and asyncpg use positional parameters which is why these
issues are suddenly apparent.
crud.py now takes on the task of rendering the column
expressions for SET or VALUES so that for the very unusual
case that the column expression is a compound expression
that includes a bound parameter (namely an array index),
the bound parameter order is preserved.
Additionally, crud.py passes through the positional_names
keyword argument into bindparam_string() which is necessary
when CTEs are being rendered, as PG supports complex
CTE / INSERT / UPDATE scenarios.
Mike Bayer [Fri, 7 Aug 2020 18:51:33 +0000 (14:51 -0400)]
Don't link on_connect to first_connect event handler
Adjusted the dialect initialization process such that the
:meth:`_engine.Dialect.on_connect` is not called a second time on the first
connection. The hook is called first, then the
:meth:`_engine.Dialect.initialize` is called if that connection is the
first for that dialect, then no more events are called. This eliminates
the two calls to the "on_connect" function which can produce very difficult
debugging situations.
Mike Bayer [Fri, 7 Aug 2020 15:24:29 +0000 (11:24 -0400)]
Pass schema_translate_map from DDLCompiler to SQLCompiler
Fixed issue where the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would not take effect when the :meth:`_schema.Sequence.next_value`
function function for a :class:`_schema.Sequence` were used in the
:paramref:`_schema.Column.server_default` parameter and the create table
DDL were emitted.
Mike Bayer [Thu, 6 Aug 2020 19:53:17 +0000 (15:53 -0400)]
base all_orm_descriptors ordering on cls.__dict__ + cls.__mro__
Adjusted the workings of the :meth:`_orm.Mapper.all_orm_descriptors`
accessor to represent the attributes in the order that they are located in
a deterministic way, assuming the use of Python 3.6 or higher which
maintains the sorting order of class attributes based on how they were
declared. This sorting is not guaranteed to match the declared order of
attributes in all cases however; see the method documentation for the exact
scheme.
Mike Bayer [Sun, 28 Jun 2020 15:59:34 +0000 (11:59 -0400)]
Documentation updates for 1.4
* major additions to 1.4 migration doc; removed additional
verbosity regarding caching methodology and reorganized the
doc to present itself more as a "what's changed" guide
* as we now have a path for asyncio, update that doc so that
we aren't spreading obsolete information
* updates to the 2.0 migration guide with latest info, however
this is still an architecture doc and not a migration guide
yet, will need further rework.
* start really talking about 1.x vs. 2.0 style everywhere. Querying
is most of the docs so this is going to be a prominent
theme, start getting it to fit in
* Add introductory documentation for ORM example sections as these
are too sparse
* new documentation for do_orm_execute(), many separate sections,
adding deprecation notes to before_compile() and similar
* new example suites to illustrate do_orm_execute(),
with_loader_criteria()
* modernized horizontal sharding examples and added a separate
example to distinguish between multiple databases and single
database w/ multiple tables use case
* introducing DEEP ALCHEMY, will use zzzeeksphinx 1.1.6
* no name for the alchemist yet however the dragon's name
is Flambé
Mike Bayer [Thu, 6 Aug 2020 01:47:43 +0000 (21:47 -0400)]
Implement relationship AND criteria; global loader criteria
Added the ability to add arbitrary criteria to the ON clause generated
by a relationship attribute in a query, which applies to methods such
as :meth:`_query.Query.join` as well as loader options like
:func:`_orm.joinedload`. Additionally, a "global" version of the option
allows limiting criteria to be applied to particular entities in
a query globally.
Documentation is minimal at this point, new examples will
be coming in a subsequent commit.
Some adjustments to execution options in how they are represented
in the ORMExecuteState as well as well as a few ORM tests that
forgot to get merged in a preceding commit.
Mike Bayer [Thu, 6 Aug 2020 00:32:02 +0000 (20:32 -0400)]
Fixes for MySQL 8
MySQL 8.0.19 has some changes to how it reports on display widths
for int types (seems like it omits it in some cases), and also
no longer cares about the length for YEAR. Another adjustment
to the ordering of constraints reported in one case also.
At least one CI machine is at 8.0.21 now.
Mike Bayer [Wed, 5 Aug 2020 21:58:48 +0000 (17:58 -0400)]
Convert lazy loader, selectinload, load_on_ident to lambda statements
Building on newly robust lambdas in
I29a513c98917b1d503abfdd61e6b6e8800851aa8,
convert key loading off of the "baked" system so that baked
is no longer used by the ORM.
Mike Bayer [Wed, 5 Aug 2020 20:42:26 +0000 (16:42 -0400)]
Robustness for lambdas, lambda statements
in order to accommodate relationship loaders
with lambda caching, a lot more is needed. This is
a full refactor of the lambda system such that it
now has two levels of caching; the first level caches what
can be known from the __code__ element, then the next level
of caching is against the lambda itself and the contents
of __closure__. This allows for the elements inside
the lambdas, like columns and entities, to change and
then be part of the cache key. Lazy/selectinloads' use of
baked queries had to add distinct cache key elements,
which was attempted here but overall things needed to be
more robust than that.
This commit is broken out from the very long and sprawling
commit at Id6b5c03b1ce9ddb7b280f66792212a0ef0a1c541 .
Mike Bayer [Sat, 1 Aug 2020 17:57:04 +0000 (13:57 -0400)]
Genericize str() for types
Remove lookup logic that attempts to locate a dialect for a type,
just use StrSQLTypeCompiler.
Cleaned up the internal ``str()`` for datatypes so that all types produce a
string representation without any dialect present, including that it works
for third-party dialect types without that dialect being present. The
string representation defaults to being the UPPERCASE name of that type
with nothing else.
Mike Bayer [Sat, 1 Aug 2020 19:05:53 +0000 (15:05 -0400)]
Establish future behavior for Session cascade backrefs, bind
The behavior of the :paramref:`_orm.relationship.cascade_backrefs` flag
will be reversed in 2.0 and set to ``False`` unconditionally, such that
backrefs don't cascade save-update operations from a forwards-assignment to
a backwards assignment. A 2.0 deprecation warning is emitted when the
parameter is left at its default of ``True`` at the point at which such a
cascade operation actually takes place. The new behavior can be
established as always by setting the flag to ``False`` on a specific
:func:`_orm.relationship`, or more generally can be set up across the board
by setting the the :paramref:`_orm.Session.future` flag to True.
Additionally in the interests of expediency, this commit will also
move Session away from making use of bound metadata if the future=True
flag is set. An application that sets future=True should ideally
have to change as little else as possible for full 2.0 behavior.
Mike Bayer [Sat, 9 May 2020 14:55:19 +0000 (10:55 -0400)]
Unpin pydocstyle, minimum for flake8-docstrings 1.3.1
Per https://gitlab.com/pycqa/flake8-docstrings/-/issues/36#note_199635329
flake8-docstrings has repaired the pydocstyle issue as of
1.3.1, so unpin pydocstyle.
Mike Bayer [Tue, 28 Jul 2020 20:46:53 +0000 (16:46 -0400)]
Consider default FROM DUAL for MySQL
MySQL claims it doesn't require FROM DUAL for no
FROM clause even though the issue at #5481 locates
a case which requires one. See if FROM DUAL the same
way as Oracle without attempting to guess is potentially
feasible.
- in create wheel set minimum versions of setuptools and wheel to avoid
failure in python 3.5 with metadata configured in setup.cfg
- update action versions
- test also cext in the pull requests
Jim Bosch [Sun, 26 Jul 2020 20:50:14 +0000 (16:50 -0400)]
Ensure is_comparison passed for PG RANGE op() methods
Fixed issue where the return type for the various RANGE comparison
operators would itself be the same RANGE type rather than BOOLEAN, which
would cause an undesirable result in the case that a
:class:`.TypeDecorator` that defined result-processing behavior were in
use. Pull request courtesy Jim Bosch.
Mike Bayer [Sun, 26 Jul 2020 17:17:56 +0000 (13:17 -0400)]
Improve delete cascade and passive deletes sections
Add cross linking from Core to ORM and also add new sections
and examples regarding many-to-many. Move the section
out of "collections" and into the "cascades" chapter
where it's more likely to be found.
PyODBCConnector.initialize just super from Connector, no need to redeclare, so I guess that's why they are commented before.
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
Remove the useless comment code lines.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Tue, 21 Jul 2020 16:36:20 +0000 (12:36 -0400)]
Allow Grouping to pass along proxy_set of element
Repaired an issue where the "ORDER BY" clause rendering a label name rather
than a complete expression, which is particularly important for SQL Server,
would fail to occur if the expression were enclosed in a parenthesized
grouping in some cases. This case has been added to test support.
Mike Bayer [Sun, 19 Jul 2020 21:39:14 +0000 (17:39 -0400)]
Revise setinputsizes approach
in order to support asyncpg as well as pg8000,
we need to revise setinputsizes to work for more cases as well
as adjust NativeForEmulated a bit to work more completely with
the INTERVAL datatype.
- put most of the setinputsizes work into the compiler where
the computation can be cached.
- support per-element setinputsizes for a tuple
- adjust TypeDecorator so that _unwrapped_dialect_impl
will honor a type that the dialect links to directly in
it's adaption mapping. Decouble _unwrapped_dialect_impl
from TypeDecorator._gen_dialect_impl() which has a different
purpose. This allows setinputsizes to do the right thing
with the INTERVAL datatype.
- test cases for Oracle with Variant continue to work
Mike Bayer [Mon, 13 Jul 2020 14:49:57 +0000 (10:49 -0400)]
test single and double quote inspection scenarios
Applied a sweep through all included dialects to ensure names that contain
single or double quotes are properly escaped when querying system tables,
for all :class:`.Inspector` methods that accept object names as an argument
(e.g. table names, view names, etc). SQLite and MSSQL contained two
quoting issues that were repaired.
mock in python 2.7 / 3.8 seems to have an "args" accessor
but in prior python 3 versions this is not the case,
so this is likely a recent addition to "mock" but wasn't
present in the python3 vendored versions.
Mike Bayer [Sat, 11 Jul 2020 22:54:15 +0000 (18:54 -0400)]
Further corrections to session docs
many more to come.
One issue is that references to documentation sections
inside of decorator-configured deprecation warnings will
also generate a meaningless token when the runtime warning
is emitted. It would be nice to improve upon this somehow.
Mike Bayer [Wed, 8 Jul 2020 18:31:17 +0000 (14:31 -0400)]
Convert remaining ORM APIs to support 2.0 style
This is kind of a mixed bag of all kinds to help get us
to 1.4 betas. The documentation stuff is a work in
progress. Lots of other relatively small changes to
APIs and things. More commits will follow to continue
improving the documentation and transitioning to the
1.4/2.0 hybrid documentation. In particular some refinements
to Session usage models so that it can match Engine's
scoping / transactional patterns, and a decision to
start moving away from "subtransactions" completely.
* add select().from_statement() to produce FromStatement in an
ORM context
* begin referring to select() that has "plugins" for the few edge
cases where select() will have ORM-only behaviors
* convert dynamic.AppenderQuery to its own object that can use
select(), though at the moment it uses Query to support legacy
join calling forms.
* custom query classes for AppenderQuery are replaced by
do_orm_execute() hooks for custom actions, a separate gerrit
will document this
* add Session.get() to replace query.get()
* Deprecate session.begin->subtransaction. propose within the
test suite a hypothetical recipe for apps that rely on this
pattern
* introduce Session construction level context manager,
sessionmaker context manager, rewrite the whole top of the
session_transaction.rst documentation. Establish context manager
patterns for Session that are identical to engine
* ensure same begin_nested() / commit() behavior as engine
* devise all new "join into an external transaction" recipe,
add test support for it, add rules into Session so it
just works, write new docs. need to ensure this doesn't
break anything
* vastly reduce the verbosity of lots of session docs as
I dont think people read this stuff and it's difficult
to keep current in any case
* constructs like case(), with_only_columns() really need
to move to *columns, add a coercion rule to just change
these.
* docs need changes everywhere I look. in_() is not in
the Core tutorial? how do people even know about it?
Remove tons of cruft from Select docs, etc.
* build a system for common ORM options like populate_existing
and autoflush to populate from execution options.
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
- Update the test so it looks at computed_default_table instead of
computed_column_table.
- Stop inspecting autoincrement; instead directly check for
default/non-default values for the relevant columns.
Fixes: #5414
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Wed, 8 Jul 2020 19:48:32 +0000 (15:48 -0400)]
Limit init_subclass test above python 3.6
The commit for I1654befe9eb1c8b8e7fc0784bdbe64284614f0ea #5357
runs the test on all Python 3 versions, however we need to limit
at least python 3.6 for this.
Mike Bayer [Wed, 8 Jul 2020 19:32:38 +0000 (15:32 -0400)]
Propose --nomemory by default
the memory tests are solely responsible for the largest chunk of
time and CPU energy taken up in running the tests. Regressions
in this area are nonexistent unless major changes are being
taken up. Try to revert them to False and see if a single
gerrit job can perhaps run these.
Mike Bayer [Wed, 8 Jul 2020 19:12:30 +0000 (15:12 -0400)]
Repair change of "future_select" caught in merge conflict
the recent change to select() apparently seems to have
been merged with other statements in the test_update_delete
test that was still using the old term.
Mike Bayer [Fri, 26 Jun 2020 20:15:19 +0000 (16:15 -0400)]
Add future=True to create_engine/Session; unify select()
Several weeks of using the future_select() construct
has led to the proposal there be just one select() construct
again which features the new join() method, and otherwise accepts
both the 1.x and 2.x argument styles. This would make
migration simpler and reduce confusion.
However, confusion may be increased by the fact that select().join()
is different Current thinking is we may be better off
with a few hard behavioral changes to old and relatively unknown APIs
rather than trying to play both sides within two extremely similar
but subtly different APIs. At the moment, the .join() thing seems
to be the only behavioral change that occurs without the user
taking any explicit steps. Session.execute() will still
behave the old way as we are adding a future flag.
This change also adds the "future" flag to Session() and
session.execute(), so that interpretation of the incoming statement,
as well as that the new style result is returned, does not
occur for existing applications unless they add the use
of this flag.
The change in general is moving the "removed in 2.0" system
further along where we want the test suite to fully pass
even if the SQLALCHEMY_WARN_20 flag is set.
Get many tests to pass when SQLALCHEMY_WARN_20 is set; this
should be ongoing after this patch merges.
Improve the RemovedIn20 warning; these are all deprecated
"since" 1.4, so ensure that's what the messages read.
Make sure the inforamtion link is on all warnings.
Add deprecation warnings for parameters present and
add warnings to all FromClause.select() types of methods.
Imply `sync_backref` flag in a viewonly relationship
Update :paramref:`_orm.relationship.sync_backref` flag in a relationship
to make it implicitly set to False in ``viewonly=True`` relationships,
preventing synchronization events in all cases.
Mike Bayer [Tue, 7 Jul 2020 15:26:39 +0000 (11:26 -0400)]
ensure we unwrap desc() /label() all the way w/ order by
The deprecated logic to move order_by expressions
up into the columns clause needed adjustment to accommodate
for a more deeply-wrapped structure when desc() + label()
are combined in an order by column. This structure
now comes from coercions in 1.4. it's not clear to me
at the moment why it's different from 1.3 but
this shouldn't really matter.
Added a ``**kw`` argument to the :meth:`.DeclarativeMeta.__init__` method.
This allows a class to support the :pep:`487` metaclass hook
``__init_subclass__``. Pull request courtesy Ewen Gillies.
1. move all discussion of how to customize connection
to the engines.rst section. have the section in pooling.rst
point to engines.
2. ensure all of the common use cases for connection modification
are listed out in order of level of modification. the use
case of modifying an existing connection is separate from that of
modifying how connect is called; ensure that poolevents.connect
is referred to just as prominently as dialectevents.do_connect.
3. completely replace any discussion of create_engine.creator, as this
hook does not offer anything beyond what do_connect() does.
"creator" is more about using a Pool object directly without an
Engine, which is no longer a documented use case.
Mike Bayer [Sun, 5 Jul 2020 17:44:58 +0000 (13:44 -0400)]
Ensure synchronize_session works with lambda statements
A few places have logic that assumes the top-level statement
is the actual UPDATE or DELETE which is not the case with a
lambda. Ensure the correct object is used. This
fixes issues specific to both "fetch" strategy
as well as "evaluate" strategy.
Mike Bayer [Mon, 16 Dec 2019 22:06:43 +0000 (17:06 -0500)]
introduce deferred lambdas
The coercions system allows us to add in lambdas as arguments
to Core and ORM elements without changing them at all. By allowing
the lambda to produce a deterministic cache key where we can also
cheat and yank out literal parameters means we can move towards
having 90% of "baked" functionality in a clearer way right in
Core / ORM.
As a second step, we can have whole statements inside the lambda,
and can then add generation with __add__(), so then we have
100% of "baked" functionality with full support of ad-hoc
literal values.
Adds some more short_selects tests for the moment for comparison.
Other tweaks inside cache key generation as we're trying to
approach a certain level of performance such that we can
remove the use of "baked" from the loader strategies.
As we have not yet closed #4639, however the caching feature
has been fully integrated as of b0cfa7379cf8513a821a3dbe3028c4965d9f85bd, we will also
add complete caching documentation here and close that issue
as well.
jonathan vanasco [Tue, 30 Jun 2020 16:31:58 +0000 (12:31 -0400)]
added semicolon (;) to improve warning message clarity
### Description
Added a semicolon to improve the clarity of warning message. I actually had a table named `backend`, and thought it was involved!
While updating the code, I noticed no test that directly tests for this warning message. There are tests for the `Can't sort tables for DROP;` prefix of this message and the `exc.CircularDependencyError`; and some tests for the `exc.CircularDependencyError` message itself. I couldn't find any test for this particular message though. (Just thought I'd bring that up)
Mike Bayer [Mon, 29 Jun 2020 19:45:20 +0000 (15:45 -0400)]
Include DATETIME / DateTime with the MySQL TIMESTAMP examples
To eliminate any remaining confusion, clarify that
DATETIME (as well as DateTime) and TIMESTAMP are treated
similarly with the MySQL dialect regarding ON UPDATE.
Mike Bayer [Mon, 29 Jun 2020 18:35:22 +0000 (14:35 -0400)]
Docuemnt current workarounds for MySQL TIMESTAMP
MySQL 8 no longer generates the DEFAULT or ON UPDATE
clauses for TIMESTAMP by default, hence users will begin to hit
this regularly. add warnings that this non-standard SQL is not
accommodated by server_onupdate and docuemnt the workaround
used in issues such as #4652.
Mike Bayer [Sun, 28 Jun 2020 16:47:32 +0000 (12:47 -0400)]
Remove _generate_path_cache_key()
loader options can now make a deterministic cache key based
on the structure they are given, and this accommodates for
aliased classes as well so that these cache keys are now
"safe". Have baked query call upon
the regular cache key method.
Mike Bayer [Sun, 28 Jun 2020 02:57:52 +0000 (22:57 -0400)]
Add an extra step to pool test_sync
Have observed CI failure with windows where not all
three connections got pulled out at the same time here
as the threads got serialized. make sure all three
connections get used.
Mike Bayer [Fri, 17 Apr 2020 14:55:08 +0000 (10:55 -0400)]
ORM executemany returning
Build on #5401 to allow the ORM to take advanage
of executemany INSERT + RETURNING.
Implemented the feature
updated tests
to support INSERT DEFAULT VALUES, needed to come up with
a new syntax for compiler INSERT INTO table (anycol) VALUES (DEFAULT)
which can then be iterated out for executemany.
Added graceful degrade to plain executemany for PostgreSQL <= 8.2
Renamed EXECUTEMANY_DEFAULT to EXECUTEMANY_PLAIN
Fix issue where unicode identifiers or parameter names wouldn't
work with execute_values() under Py2K, because we have to
encode the statement and therefore have to encode the
insert_single_values_expr too.
Correct issue from #5401 to support executemany + return_defaults
for a PK that is explicitly pre-generated, meaning we aren't actually
getting RETURNING but need to return it from compiled_parameters.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
