Mike Bayer [Sun, 16 Oct 2022 22:30:44 +0000 (18:30 -0400)]
repair type qualify in _ServerDefaultType; other pyright tweaks
as we haven't done full pylance / pyright strict typing internally,
some of the things pyright reports on specifically will leak
out into user code, such as this mapped_column() issue.
So we will have to look more closely at pyright strict
mode going forward for the release.
Fixed typing issue where pylance strict mode would report "partially
unknown" datatype for the :func:`_orm.mapped_column` construct.
Also repaired a trailing comma and pyright complaining about overloads
for orm.composite.
Mike Bayer [Sat, 15 Oct 2022 19:20:21 +0000 (15:20 -0400)]
accommodate arbitrary embedded params in insertmanyvalues
Fixed bug in new "insertmanyvalues" feature where INSERT that included a
subquery with :func:`_sql.bindparam` inside of it would fail to render
correctly in "insertmanyvalues" format. This affected psycopg2 most
directly as "insertmanyvalues" is used unconditionally with this driver.
Mike Bayer [Sat, 15 Oct 2022 15:12:25 +0000 (11:12 -0400)]
disable isort in pyproject.toml
disable isort, for IDEs that just default isort to be turned on, e.g. vscode.
we use flake8-import-order for import sorting, using zimports to actually
reformat code. isort is nicer in many ways but doesn't have our
"import *" fixer and also is not 100% compatible with flake8-import-order.
Mike Bayer [Tue, 11 Oct 2022 21:01:43 +0000 (17:01 -0400)]
implement autobegin=False option
Added new parameter :paramref:`_orm.Session.autobegin`, which when set to
``False`` will prevent the :class:`_orm.Session` from beginning a
transaction implicitly. The :meth:`_orm.Session.begin` method must be
called explicitly first in order to proceed with operations, otherwise an
error is raised whenever any operation would otherwise have begun
automatically. This option can be used to create a "safe"
:class:`_orm.Session` that won't implicitly start new transactions.
As part of this change, also added a new status variable
:class:`_orm.SessionTransaction.origin` which may be useful for event
handling code to be aware of the origin of a particular
:class:`_orm.SessionTransaction`.
Mike Bayer [Mon, 10 Oct 2022 18:03:04 +0000 (14:03 -0400)]
warn for no polymorphic identity w/ poly hierarchy
A warning is emitted when attempting to configure a mapped class within an
inheritance hierarchy where the mapper is not given any polymorphic
identity, however there is a polymorphic discriminator column assigned.
Such classes should be abstract if they never intend to load directly.
Mike Bayer [Tue, 11 Oct 2022 18:22:40 +0000 (14:22 -0400)]
enable check same thread for aiosqlite
to do this we have to invent our own isolation level
setter based on their current internals. however
now we can ensure thread-safe access. we are trying
to resolve an issue where test suite on CI seems to fail
around the same time each time.
Mike Bayer [Fri, 7 Oct 2022 18:03:16 +0000 (14:03 -0400)]
rename MappedCollection and related
For consistency with the prominent ORM concept :class:`_orm.Mapped`, the
names of the dictionary-oriented collections,
:func:`_orm.attribute_mapped_collection`,
:func:`_orm.column_mapped_collection`, and :class:`_orm.MappedCollection`,
are changed to :func:`_orm.attribute_keyed_dict`,
:func:`_orm.column_keyed_dict` and :class:`_orm.KeyFuncDict`, using the
phrase "dict" to minimize any confusion against the term "mapped". The old
names will remain indefinitely with no schedule for removal.
Docs here are also updated for typing as we can type
these collections as ``Mapped[dict[str, cls]]``, don't need
KeyFuncDict / MappedCollection for these
Mike Bayer [Fri, 7 Oct 2022 15:25:08 +0000 (11:25 -0400)]
dont mutate bind_arguments incoming dictionary
The :paramref:`_orm.Session.execute.bind_arguments` dictionary is no longer
mutated when passed to :meth:`_orm.Session.execute` and similar; instead,
it's copied to an internal dictionary for state changes. Among other
things, this fixes and issue where the "clause" passed to the
:meth:`_orm.Session.get_bind` method would be incorrectly referring to the
:class:`_sql.Select` construct used for the "fetch" synchronization
strategy, when the actual query being emitted was a :class:`_dml.Delete` or
:class:`_dml.Update`. This would interfere with recipes for "routing
sessions".
Mike Bayer [Mon, 26 Sep 2022 18:38:44 +0000 (14:38 -0400)]
implement write-only colletions, typing for dynamic
For 2.0, we provide a truly "larger than memory collection"
implementation, a write-only collection that will never
under any circumstances implicitly load the entire
collection, even during flush.
This is essentially a much more "strict" version
of the "dynamic" loader, which in fact has a lot of
scenarios that it loads the full backing collection
into memory, mostly defeating its purpose.
Typing constructs are added that support
both the new feature WriteOnlyMapping as well as the
legacy feature DynamicMapping. These have been
integrated with "annotion based mapping" so that
relationship() uses these annotations to configure
the loader strategy as well.
additional changes:
* the docs triggered a conflict in hybrid's
"transformers" section, this section is hard-coded
to Query using a pattern that doesnt seem to have
any use and isn't part of the current select()
interface, so just removed this section
* As the docs for WriteOnlyMapping are very long,
collections.rst is broken up into two pages now.
Mike Bayer [Thu, 29 Sep 2022 16:56:23 +0000 (12:56 -0400)]
reorganize Mapped[] super outside of MapperProperty
We made all the MapperProperty classes a subclass
of Mapped[] to allow declarative mappings to name
Mapped[] on the left side. this was cheating a bit because
MapperProperty is not actually a descriptor, and the mapping
process replaces the object with InstrumentedAttribute at
mapping time, which is the actual Mapped[] descriptor.
But now in I6929f3da6e441cad92285e7309030a9bac4e429d we
are considering making the "cheating" a little more extensive
by putting DynamicMapped / WriteOnlyMapped in Relationship's
hierarchy, which need a flat out "type: ignore" to work.
Instead of pushing more cheats into the core classes, move
out the "Declarative"-facing versions of these classes to be
typing only: Relationship, Composite, Synonym, and MappedSQLExpression
added for ColumnProperty. Keep the internals expressed on the
old names, RelationshipProperty, CompositeProperty, SynonymProperty,
ColumnProprerty, which will remain "pure" with fully correct typing.
then have the typing only endpoints be where the "cheating"
and "type: ignores" have to happen, so that these are more or less
slightly better forms of "Any".
Make if_exists and if_not_exists flags on ddl statements match compiler
Added ``if_exists`` and ``if_not_exists`` parameters for all "Create" /
"Drop" constructs including :class:`.CreateSequence`,
:class:`.DropSequence`, :class:`.CreateIndex`, :class:`.DropIndex`, etc.
allowing generic "IF EXISTS" / "IF NOT EXISTS" phrases to be rendered
within DDL. Pull request courtesy Jesse Bakker.
John Bodley [Fri, 30 Sep 2022 01:58:58 +0000 (21:58 -0400)]
adjust MySQL view reflection for non-standard MySQL variants
Adjusted the regular expression used to match "CREATE VIEW" when
testing for views to work more flexibly, no longer requiring the
special keyword "ALGORITHM" in the middle, which was intended to be
optional but was not working correctly. The change allows view reflection
to work more completely on MySQL-compatible variants such as StarRocks.
Pull request courtesy John Bodley.
Mike Bayer [Mon, 3 Oct 2022 01:22:11 +0000 (21:22 -0400)]
the future is here
the autodoc for the "future" Engine / Connection were removed,
so all these links weren't working. Replace all _future
for these with _engine. There was just one _future pointing
to select, changed that separately.
Mike Bayer [Sun, 2 Oct 2022 02:24:38 +0000 (22:24 -0400)]
add disable doctest tag for autodoc test suite
ahead of trying to get everything formatted, some more
flexibility so that we can use doctest for all
python + sql code, while still being able to tell the
test suite to not run doctests on a sample. All of the
"non-console python with SQL" in the docs is because I was
showing an example that I didn't want tested.
Mike Bayer [Fri, 30 Sep 2022 20:58:26 +0000 (16:58 -0400)]
fix "entry points"
the formatting here wasn't round tripping without manually
adjusting the indentation. this suggests that "check" mode
is not doing the exact same thing as "write". maybe
file.write_text() is changing something.
Added script to format code in the rst documentation using black.
This is also added to the lint tox job to ensure that the code
in the docs is properly formatted.
Mike Bayer [Tue, 27 Sep 2022 14:23:28 +0000 (10:23 -0400)]
correct test criteria for mssql
this test was failing on mssql b.c. mssql
reports false for supports_default_metavalue. however
insertmanyvalues is still used in this case, and the
assert SQL is against the default dialect in any case.
Mike Bayer [Mon, 26 Sep 2022 12:54:59 +0000 (08:54 -0400)]
adjust tests for sqlites w/o returning
the sqlite builds on github actions seem to be very
inconsistent about versions and many don't support
RETURNING. ensure any tests that depend on RETURNING present
are marked as such.
Mike Bayer [Thu, 18 Aug 2022 17:56:50 +0000 (13:56 -0400)]
New ORM Query Guide featuring DML support
reviewers: these docs publish periodically at:
https://docs.sqlalchemy.org/en/gerrit/4042/orm/queryguide/index.html
See the "last generated" timestamp near the bottom of the
page to ensure the latest version is up
Change includes some other adjustments:
* small typing fixes for end-user benefit
* removal of a bunch of old examples for patterns that nobody
uses or aren't really what we promote now
* modernization of some examples, including inheritance
Mike Bayer [Sun, 25 Sep 2022 18:56:22 +0000 (14:56 -0400)]
warn for local-only column in remote side
A warning is emitted in ORM configurations when an explicit
:func:`_orm.remote` annotation is applied to columns that are local to the
immediate mapped class, when the referenced class does not include any of
the same table columns. Ideally this would raise an error at some point as
it's not correct from a mapping point of view.
Mike Bayer [Sun, 7 Aug 2022 16:14:19 +0000 (12:14 -0400)]
ORM bulk insert via execute
* ORM Insert now includes "bulk" mode that will run
essentially the same process as session.bulk_insert_mappings;
interprets the given list of values as ORM attributes for
key names
* ORM UPDATE has a similar feature, without RETURNING support,
for session.bulk_update_mappings
* Added support for upserts to do RETURNING ORM objects as well
* ORM UPDATE/DELETE with list of parameters + WHERE criteria
is a not implemented; use connection
* ORM UPDATE/DELETE defaults to "auto" synchronize_session;
use fetch if RETURNING is present, evaluate if not, as
"fetch" is much more efficient (no expired object SELECT problem)
and less error prone if RETURNING is available
UPDATE: howver this is inefficient! please continue to
use evaluate for simple cases, auto can move to fetch
if criteria not evaluable
* "Evaluate" criteria will now not preemptively
unexpire and SELECT attributes that were individually
expired. Instead, if evaluation of the criteria indicates that
the necessary attrs were expired, we expire the object
completely (delete) or expire the SET attrs unconditionally
(update). This keeps the object in the same unloaded state
where it will refresh those attrs on the next pass, for
this generally unusual case. (originally #5664)
* Core change! update/delete rowcount comes from len(rows)
if RETURNING was used. SQLite at least otherwise did not
support this. adjusted test_rowcount accordingly
* ORM DELETE with a list of parameters at all is also a not
implemented as this would imply "bulk", and there is no
bulk_delete_mappings (could be, but we dont have that)
* ORM insert().values() with single or multi-values translates
key names based on ORM attribute names
* ORM returning() implemented for insert, update, delete;
explcit returning clauses now interpret rows in an ORM
context, with support for qualifying loader options as well
* session.bulk_insert_mappings() assigns polymorphic identity
if not set.
* explicit RETURNING + synchronize_session='fetch' is now
supported with UPDATE and DELETE.
* expanded return_defaults() to work with DELETE also.
* added support for composite attributes to be present
in the dictionaries used by bulk_insert_mappings and
bulk_update_mappings, which is also the new ORM bulk
insert/update feature, that will expand the composite
values into their individual mapped attributes the way they'd
be on a mapped instance.
* bulk UPDATE supports "synchronize_session=evaluate", is the
default. this does not apply to session.bulk_update_mappings,
just the new version
* both bulk UPDATE and bulk INSERT, the latter with or without
RETURNING, support *heterogenous* parameter sets.
session.bulk_insert/update_mappings did this, so this feature
is maintained. now cursor result can be both horizontally
and vertically spliced :)
This is now a long story with a lot of options, which in
itself is a problem to be able to document all of this
in some way that makes sense. raising exceptions for
use cases we haven't supported is pretty important here
too, the tradition of letting unsupported things just not work
is likely not a good idea at this point, though there
are still many cases that aren't easily avoidable
Mike Bayer [Mon, 18 Jul 2022 19:08:37 +0000 (15:08 -0400)]
implement batched INSERT..VALUES () () for executemany
the feature is enabled for all built in backends
when RETURNING is used,
except for Oracle that doesn't need it, and on
psycopg2 and mssql+pyodbc it is used for all INSERT statements,
not just those that use RETURNING.
third party dialects would need to opt in to the new feature
by setting use_insertmanyvalues to True.
Also adds dialect-level guards against using returning
with executemany where we dont have an implementation to
suit it. execute single w/ returning still defers to the
server without us checking.
Tighten password security by removing `URL.__str__`
For improved security, the :class:`_url.URL` object will now use password
obfuscation by default when ``str(url)`` is called. To stringify a URL with
cleartext password, the :meth:`_url.URL.render_as_string` may be used,
passing the :paramref:`_url.URL.render_as_string.hide_password` parameter
as ``False``. Thanks to our contributors for this pull request.
Mike Bayer [Fri, 23 Sep 2022 19:17:57 +0000 (15:17 -0400)]
remove should_nest behavior for contains_eager()
Fixed regression for 1.4 in :func:`_orm.contains_eager` where the "wrap in
subquery" logic of :func:`_orm.joinedload` would be inadvertently triggered
for use of the :func:`_orm.contains_eager` function with similar statements
(e.g. those that use ``distinct()``, ``limit()`` or ``offset()``). This is
not appropriate for :func:`_orm.contains_eager` which has always had the
contract that the user-defined SQL statement is unmodified with the
exception of adding the appropriate columns.
Also includes an adjustment to the assertion in Label._make_proxy()
which was there to prevent a fixed label name from being anonymized;
if the label is already anonymous, the change should proceed.
This logic was being hit before the contains_eager behavior was
adjusted. With the adjustment, this code is not used.
Mike Bayer [Fri, 23 Sep 2022 17:20:16 +0000 (13:20 -0400)]
update whatsnew/ migration
The migration20 guide had a lot of "whats new" stuff in it,
which is moved to the whatsnew20 document.
Also moved most of the content regarding the reflection rewrite
into the whatsnew20 document and added some performance comparisons
to 1.4.
The sectioning for these two documents, as has happened for some
other documents I'm working on, has been modified to only use the
double-equals operator at the top. This is strictly because my
vscode seems to be able to show me an .rst outline in the side panel
if I do it this way, and otherwise it's blank.
Mike Bayer [Tue, 20 Sep 2022 16:21:14 +0000 (12:21 -0400)]
auto-cast PG range types
Range type handling has been enhanced so that it automatically
renders type casts, so that in-place round trips for statements that don't
provide the database with any context don't require the :func:`_sql.cast`
construct to be explicit for the database to know the desired type.
Mike Bayer [Mon, 19 Sep 2022 13:40:40 +0000 (09:40 -0400)]
break out text() from TextualSelect for col matching
Fixed issue where mixing "*" with additional explicitly-named column
expressions within the columns clause of a :func:`_sql.select` construct
would cause result-column targeting to sometimes consider the label name or
other non-repeated names to be an ambiguous target.
Mike Bayer [Mon, 19 Sep 2022 16:18:47 +0000 (12:18 -0400)]
add raiseload to load_only()
currently this can't be finessed in another way,
at least not easily. The deferred() that it sets up
doesn't seem to be cancellable. in any case, this is more consistent
API with that of defer().
Mike Bayer [Sat, 17 Sep 2022 14:33:55 +0000 (10:33 -0400)]
change verbiage stating exact compliance with RFC-1738
As long as we aren't using urlparse() to parse URLs,
we are not RFC-1738 compliant. As we accept underscores
in the scheme and not dashes or dots, we are not
RFC-1738 compliant, so emulate language like
that of PostgreSQL [1] that we "generally follow" this
scheme but include some exceptions.
Mike Bayer [Sat, 17 Sep 2022 14:18:56 +0000 (10:18 -0400)]
remove obtuse section about "bundled bind parameters"
Just looking for basics on insert in the first pages
of the tutorial I see this weird detour into something that
nobody ever uses and definitely isn't going to make sense
to the people I see complaining about our docs on twitter,
remove this. the tutorial probably needs a big sweep
for wordy obtuse things. the userbase is changing and
we really have a lot of brand-new-to-programming types coming
in.
The :class:`_functions.array_agg` will now set the array dimensions to 1.
Improved :class:`_types.ARRAY` processing to accept ``None`` values as
value of a multi-array.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
