Mike Bayer [Mon, 4 Apr 2022 14:13:23 +0000 (10:13 -0400)]
cx_Oracle modernize
Full "RETURNING" support is implemented for the cx_Oracle dialect, meaning
multiple RETURNING rows are now recived for DML statements that produce
more than one row for RETURNING.
cx_Oracle 7 is now the minimum version for cx_Oracle.
Getting Oracle to do multirow returning took about 5 minutes. however,
getting Oracle's RETURNING system to integrate with ORM-enabled
insert, update, delete, is a big deal because that architecture wasn't
really working very robustly, including some recent changes in 1.4
for FromStatement were done in a hurry, so this patch also cleans up
the FromStatement situation and begins to establish it more concretely
as the base for all ReturnsRows / TextClause ORM scenarios.
Mike Bayer [Wed, 6 Apr 2022 13:41:11 +0000 (09:41 -0400)]
maintain complete cloned_set for BindParameter
Fixed regression caused by :ticket:`7823` which impacted the caching
system, such that bound parameters that had been "cloned" within ORM
operations, such as polymorphic loading, would in some cases not acquire
their correct execution-time value leading to incorrect bind values being
rendered.
Mike Bayer [Wed, 30 Mar 2022 22:01:58 +0000 (18:01 -0400)]
pep484 - sql.selectable
the pep484 task becomes more intense as there is mounting
pressure to come up with a consistency in how data moves
from end-user to instance variable.
current thinking is coming into:
1. there are _typing._XYZArgument objects that represent "what the
user sent"
2. there's the roles, which represent a kind of "filter" for different
kinds of objects. These are mostly important as the argument
we pass to coerce().
3. there's the thing that coerce() returns, which should be what the
construct uses as its internal representation of the thing.
This is _typing._XYZElement.
but there's some controversy over whether or
not we should pass actual ClauseElements around by their role
or not. I think we shouldn't at the moment, but this makes the
"role-ness" of something a little less portable. Like, we have
to set DMLTableRole for TableClause, Join, and Alias, but then
also we have to repeat those three types in order to set up
_DMLTableElement.
Other change introduced here, there was a deannotate=True
for the left/right of a sql.join(). All tests pass without that.
I'd rather not have that there as if we have a join(A, B) where
A, B are mapped classes, we want them inside of the _annotations.
The rationale seems to be performance, but this performance can
be illustrated to be on the compile side which we hope is cached
in the normal case.
CTEs now accommodate for text selects including recursive.
Get typing to accommodate "util.preloaded" cleanly; add "preloaded"
as a real module. This seemed like we would have needed
pep562 `__getattr__()` but we don't, just set names in
globals() as we import them.
Mike Bayer [Sun, 3 Apr 2022 17:44:57 +0000 (13:44 -0400)]
use .fromisoformat() for sqlite datetime, date, time parsing
SQLite datetime, date, and time datatypes now use Python standard lib
``fromisoformat()`` methods in order to parse incoming datetime, date, and
time string values. This improves performance vs. the previous regular
expression-based approach, and also automatically accommodates for datetime
and time formats that contain either a six-digit "microseconds" format or a
three-digit "milliseconds" format.
Mike Bayer [Sun, 3 Apr 2022 15:28:57 +0000 (11:28 -0400)]
TableValuedAlias generation fixes
Fixed bug in newly implemented
:paramref:`.FunctionElement.table_valued.joins_implicitly` feature where
the parameter would not automatically propagate from the original
:class:`.TableValuedAlias` object to the secondary object produced when
calling upon :meth:`.TableValuedAlias.render_derived` or
:meth:`.TableValuedAlias.alias`.
Additionally repaired these issues in :class:`.TableValuedAlias`:
* repaired a potential memory issue which could occur when
repeatedly calling :meth:`.TableValuedAlias.render_derived` against
successive copies of the same object (for .alias(), we currently
have to still continue chaining from the previous element. not sure
if this can be improved but this is standard behavior for .alias()
elsewhere)
* repaired issue where the individual element types would be lost when
calling upon :meth:`.TableValuedAlias.render_derived` or
:meth:`.TableValuedAlias.alias`.
Mike Bayer [Sun, 3 Apr 2022 14:39:19 +0000 (10:39 -0400)]
clarify alternative mapping example
this second example is not part of the doctest steps,
clarify that it's not part of code examples to be present
in execution steps. Add an extra registry +
declarative base on top
so that even if someone does run it, the Base will have
been reset and the examples will continue to work
(noting that column order in statements may change, but
probably nothing else).
Mike Bayer [Thu, 31 Mar 2022 20:53:43 +0000 (16:53 -0400)]
allow executemany values for ON CONFLICT DO NOTHING
Scaled back a fix made for :ticket:`6581` where "executemany values" mode
for psycopg2 were disabled for all "ON CONFLICT" styles of INSERT, to
not apply to the "ON CONFLICT DO NOTHING" clause, which does not include
any parameters and is safe for "executemany values" mode. "ON CONFLICT
DO UPDATE" is still blocked from "executemany values" as there may
be additional parameters in the DO UPDATE clause that cannot be batched
(which is the original issue fixed by :ticket:`6581`).
Mike Bayer [Thu, 31 Mar 2022 18:56:52 +0000 (14:56 -0400)]
add template methods for ORMInsert
Fixed regression caused by :ticket:`7861` where invoking an
:class:`.Insert` construct which contained ORM entities via
:meth:`_orm.Session.execute` would fail.
Mike Bayer [Thu, 31 Mar 2022 13:08:11 +0000 (09:08 -0400)]
add close=False parameter to engine.dispose()
Added new parameter :paramref:`.Engine.dispose.close`, defaulting to True.
When False, the engine disposal does not touch the connections in the old
pool at all, simply dropping the pool and replacing it. This use case is so
that when the original pool is transferred from a parent process, the
parent process may continue to use those connections.
Federico Caselli [Wed, 30 Mar 2022 20:50:18 +0000 (22:50 +0200)]
Update bindparam cache key
The ``literal_execute`` parameter now takes part of the cache
generation of a bindparam, since it changes the sql string generated
by the compiler.
Previously the correct bind values were used, but the ``literal_execute``
would be ignored on subsequent executions of the same query.
Mike Bayer [Fri, 25 Mar 2022 21:08:48 +0000 (17:08 -0400)]
pep-484: the pep-484ening, SQL part three
hitting DML which is causing us to open up the
ColumnCollection structure a bit, as we do put anonymous
column expressions with None here. However, we still want
Table /TableClause to have named column collections that
don't return None, so parametrize the "key" in this
collection also.
* rename some "immutable" elements to "readonly". we change
the contents of immutablecolumncollection underneath, so it's
not "immutable"
Mike Bayer [Wed, 30 Mar 2022 15:40:21 +0000 (11:40 -0400)]
dont use cyext for sphinx builds
if these files are stale, imports fail and the build
silently excludes docstrings. also , typing / docstrings
are generally in the _py version of things, so just
don't include cyexts in doc builds.
Mike Bayer [Tue, 29 Mar 2022 13:48:24 +0000 (09:48 -0400)]
use annotated entity when adding secondary
Fixed regression in "dynamic" loader strategy where the
:meth:`_orm.Query.filter_by` method would not be given an appropriate
entity to filter from, in the case where a "secondary" table were present
in the relationship being queried and the mapping were against something
complex such as a "with polymorphic".
Mike Bayer [Mon, 28 Mar 2022 22:39:19 +0000 (18:39 -0400)]
apply loader criteria more specifically when refresh is true
Fixed bug in :func:`_orm.with_loader_criteria` function where loader
criteria would not be applied to a joined eager load that were invoked
within the scope of a refresh operation for the parent object.
Mike Bayer [Sat, 26 Mar 2022 20:20:34 +0000 (16:20 -0400)]
column_descriptions or equiv for DML, core select
Added new attributes :attr:`.ValuesBase.returning_column_descriptions` and
:attr:`.ValuesBase.entity_description` to allow for inspection of ORM
attributes and entities that are installed as part of an :class:`.Insert`,
:class:`.Update`, or :class:`.Delete` construct. The
:attr:`.Select.column_descriptions` accessor is also now implemented for
Core-only selectables.
Mike Bayer [Mon, 28 Mar 2022 17:46:24 +0000 (13:46 -0400)]
fix quotes regexp for SQLite CHECK constraints
Fixed bug where the name of CHECK constraints under SQLite would not be
reflected if the name were created using quotes, as is the case when the
name uses mixed case or special characters.
Mike Bayer [Mon, 28 Mar 2022 19:52:27 +0000 (15:52 -0400)]
bump black to 22.3.0
both black and click were released in the past
few hours, and black 21.5b1 seems to suddenly
be failing on a missing symbol from click. just
update to the latest
Mike Bayer [Mon, 28 Mar 2022 15:23:39 +0000 (11:23 -0400)]
block asyncmy 0.2.4
asyncmy made a backwards incompatible change to the
TIME datatype in 0.2.4 which hopefully will be reverted
as this change does not belong in a minor point release
Mike Bayer [Sat, 26 Mar 2022 14:09:10 +0000 (10:09 -0400)]
support BLANK_SCHEMA, RETAIN_SCHEMA FK schema on copy
Added support so that the :paramref:`.Table.tometadata.referred_schema_fn`
callable passed to :meth:`.Table.to_metadata` may return the value
:data:`.BLANK_SCHEMA` to indicate that the referenced foreign key should be
reset to None. The :data.`RETAIN_SCHEMA` symbol may also be returned from
this function to indicate "no change", which will behave the same as
``None`` currently does which also indicates no change.
Mike Bayer [Thu, 27 Jan 2022 20:07:17 +0000 (15:07 -0500)]
generalize conditional DDL throughout schema / DDL
Expanded on the "conditional DDL" system implemented by the
:class:`_schema.DDLElement` class to be directly available on
:class:`_schema.SchemaItem` constructs such as :class:`_schema.Index`,
:class:`_schema.ForeignKeyConstraint`, etc. such that the conditional logic
for generating these elements is included within the default DDL emitting
process. This system can also be accommodated by a future release of
Alembic to support conditional DDL elements within all schema-management
systems.
Mike Bayer [Thu, 24 Mar 2022 17:58:20 +0000 (13:58 -0400)]
more autocommit messaging
Further clarified connection-level logging to indicate the BEGIN, ROLLBACK
and COMMIT log messages do not actually indicate a real transaction when
the AUTOCOMMIT isolation level is in use; messaging has been extended to
include the BEGIN message itself, and the messaging has also been fixed to
accommodate when the :class:`.Engine` level
:paramref:`.create_engine.isolation_level` parameter was used directly.
Mike Bayer [Wed, 23 Mar 2022 00:14:04 +0000 (20:14 -0400)]
trust user PK argument as given; don't reduce
Fixed issue where the :class:`_orm.Mapper` would reduce a user-defined
:paramref:`_orm.Mapper.primary_key` argument too aggressively, in the case
of mapping to a ``UNION`` where for some of the SELECT entries, two columns
are essentially equivalent, but in another, they are not, such as in a
recursive CTE. The logic here has been changed to accept a given
user-defined PK as given, where columns will be related to the mapped
selectable but no longer "reduced" as this heuristic can't accommodate for
all situations.
Mike Bayer [Wed, 23 Mar 2022 14:07:13 +0000 (10:07 -0400)]
Add option to disable from linting for table valued function
Added new parameter
:paramref:`.FunctionElement.table_valued.joins_implicitly`, for the
:meth:`.FunctionElement.table_valued` construct. This parameter indicates
that the given table-valued function implicitly joins to the table it
refers towards, essentially disabling the "from linting" feature, i.e. the
"cartesian product" warning, from taking effect due to the presence of this
parameter. May be used for functions such as ``func.json_each()``.
provinzkraut [Tue, 22 Mar 2022 17:17:56 +0000 (13:17 -0400)]
Upgrade parts of the documentation to 2.0 style
<!-- Provide a general summary of your proposed changes in the Title field above -->
I've started to work on #7659, implementing the low hanging fruit changes for now. Some still remain, which I've outlined as a [comment](https://github.com/sqlalchemy/sqlalchemy/issues/7659#issuecomment-1073029151), and probably also some that I didn't catch.
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Mon, 21 Mar 2022 21:35:26 +0000 (17:35 -0400)]
note that horizontal sharding supports multi schema translates
the horizontal sharding API needs some work as it is
still exposing some legacy details, but in any case illustrate
how we can, for the moment, to use multiple schema translate
maps in a single session.
A lot more cleanup is needed in horizontal sharding, see #7837
Mike Bayer [Fri, 18 Mar 2022 14:33:40 +0000 (10:33 -0400)]
catch unexpected errors when accessing clslevel attribute
Improved the error message that's raised for the case where the
:func:`.association_proxy` construct attempts to access a target attribute
at the class level, and this access fails. The particular use case here is
when proxying to a hybrid attribute that does not include a working
class-level implementation.
Mike Bayer [Wed, 16 Mar 2022 16:37:20 +0000 (12:37 -0400)]
remove intermediary _is_clone_of entries when cloning
Improvements in memory usage by the ORM, removing a significant set of
intermediary expression objects that are typically stored when a copy of an
expression object is created. These clones have been greatly reduced,
reducing the number of total expression objects stored in memory by
ORM mappings by about 30%.
note this change causes the tests to have a bit of a harder time with
GC, which we would assume is because mappings now have a lot more
garbage to clean up after mappers are configured. it remains
to be seen what the long term effects of this are.
Mike Bayer [Sun, 13 Mar 2022 17:37:11 +0000 (13:37 -0400)]
pep-484 - SQL column operations
note we are taking out the
ColumnOperartors[SQLCoreOperations] thing; not really clear
why that was needed and at the moment it seems I was likely
confused.
Mike Bayer [Mon, 14 Mar 2022 18:09:03 +0000 (14:09 -0400)]
test #7820
There was an apparent improvement in the distill params
methodology used in exec_driver_sql which allows raw tuples to
pass through. In 1.4 there seems to be a _distill_cursor_params()
function that says it can handle this kind of parameter, but it isn't
used and when I tried to substitute it in for exec_driver_sql(),
things still fail.
In any case, add coverage here for the use case of passing
direct tuple params to exec_driver_sql including as the first
param, to note that it isn't mis-interpreted the way it is
in 1.x.
Mike Bayer [Mon, 14 Mar 2022 04:01:30 +0000 (00:01 -0400)]
restore an updated version of cascade backrefs
these sections were removed totally from SQLAlchemy 2.0
as the ``cascade_backrefs`` option was removed. However,
the current behavior is still one that should be documented,
so restore the two removed sections in a re-worded form
that presents the current behavior as well as some
explaination.
Mike Bayer [Tue, 8 Mar 2022 22:14:41 +0000 (17:14 -0500)]
pep-484: sqlalchemy.sql pass one
sqlalchemy.sql will require many passes to get all
modules even gradually typed. Will have to pick and
choose what modules can be strictly typed vs. which
can be gradual.
in this patch, emphasis is on visitors.py, cache_key.py,
annotations.py for strict typing, compiler.py is on gradual
typing but has much more structure, in particular where it
connects with the outside world.
The work within compiler.py also reached back out to
engine/cursor.py , default.py quite a bit.
Mike Bayer [Thu, 10 Mar 2022 16:57:00 +0000 (11:57 -0500)]
additional mypy strictness
enable type checking within untyped defs. This allowed
some more internals to be fixed up with assertions etc.
some internals that were unnecessary or not even used
at all were removed. BaseCursorResult was no longer
necessary since we only have one kind of CursorResult
now. The different ResultProxy subclasses that had
alternate "strategies" dont appear to be used at all
even in 1.4.x, as there's no code that accesses the
_cursor_strategy_cls attribute, which is also removed.
As these were mostly private constructs that weren't
even functioning correctly in any case,
it's fine to remove these over the 2.0 boundary.
Mike Bayer [Fri, 11 Mar 2022 21:23:14 +0000 (16:23 -0500)]
restore quickstart to toc
this was coming out wrong due to the wrong header level
on the page. it needs to be in a visible toc so that
the inner elements show up on the sidebar / top mobile nav.
Mike Bayer [Fri, 11 Mar 2022 21:01:09 +0000 (16:01 -0500)]
further simplify pool-sharing doc
1. the event based approach doesn't require dispose() to be
called at all, and the note that the "pool will hang" makes
no sense. I have no idea what that refers towards
2. the subsequent paragraph about connections and sessions
is unintelligible. old paragraphs like these are likely
why people complain about the docs so much. try to just
say "don't do this", as that is easier than trying to
explain to use connection.invalidate() etc.
Mike Bayer [Fri, 11 Mar 2022 18:40:16 +0000 (13:40 -0500)]
add copybutton
works great, including for plain code plus prompt code
with SQL (copies only the prompt code when prompts are present).
added some styling to zzzeeksphinx
Mike Bayer [Fri, 11 Mar 2022 15:08:36 +0000 (10:08 -0500)]
ORM quickstart
This is done in 1.4 style so it can be backported to 1.4.
Will put this up as is, we can work on it. For 2.0, the
ORM mapping will be updated to mapped_column() style when
we do the full pass.
Mike Bayer [Tue, 8 Mar 2022 18:40:12 +0000 (13:40 -0500)]
pop the stack that we pushed
Fixed regression caused by :ticket:`7760` where the new capabilities of
:class:`.TextualSelect` were not fully implemented within the compiler
properly, leading to issues with composed INSERT constructs such as "INSERT
FROM SELECT" and "INSERT...ON CONFLICT" when combined with CTE and textual
statements.
Mike Bayer [Tue, 8 Mar 2022 14:34:09 +0000 (09:34 -0500)]
support adapt_on_names for with_polymorphic
Added :paramref:`_orm.with_polymorphic.adapt_on_names` to the
:func:`_orm.with_polymorphic` function, which allows a polymorphic load
(typically with concrete mapping) to be stated against an alternative
selectable that will adapt to the original mapped selectable on column
names alone.
Mike Bayer [Mon, 7 Mar 2022 20:11:29 +0000 (15:11 -0500)]
support selectin_polymorphic w/ no fixed polymorphic_on
Fixed issue where the :func:`_orm.polymorphic_selectin` loader option would
not work with joined inheritance mappers that don't have a fixed
"polymorphic_on" column. Additionally added test support for a wider
variety of usage patterns with this construct.
Fixed bug where :func:`_orm.composite` attributes would not work in
conjunction with the :func:`_orm.selectin_polymorphic` loader strategy for
joined table inheritance.
Mike Bayer [Mon, 7 Mar 2022 17:01:42 +0000 (12:01 -0500)]
adapt create_engine from sqlalchemy2-stubs
this is much simplified, will try to see if _IsolationLevel
can work out, technically some driver can have custom values
here but in practice this might not be a thing
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
