Mike Bayer [Mon, 19 Sep 2022 13:40:40 +0000 (09:40 -0400)]
break out text() from TextualSelect for col matching
Fixed issue where mixing "*" with additional explicitly-named column
expressions within the columns clause of a :func:`_sql.select` construct
would cause result-column targeting to sometimes consider the label name or
other non-repeated names to be an ambiguous target.
Mike Bayer [Mon, 19 Sep 2022 16:18:47 +0000 (12:18 -0400)]
add raiseload to load_only()
currently this can't be finessed in another way,
at least not easily. The deferred() that it sets up
doesn't seem to be cancellable. in any case, this is more consistent
API with that of defer().
Mike Bayer [Sat, 17 Sep 2022 14:33:55 +0000 (10:33 -0400)]
change verbiage stating exact compliance with RFC-1738
As long as we aren't using urlparse() to parse URLs,
we are not RFC-1738 compliant. As we accept underscores
in the scheme and not dashes or dots, we are not
RFC-1738 compliant, so emulate language like
that of PostgreSQL [1] that we "generally follow" this
scheme but include some exceptions.
Mike Bayer [Sat, 17 Sep 2022 14:18:56 +0000 (10:18 -0400)]
remove obtuse section about "bundled bind parameters"
Just looking for basics on insert in the first pages
of the tutorial I see this weird detour into something that
nobody ever uses and definitely isn't going to make sense
to the people I see complaining about our docs on twitter,
remove this. the tutorial probably needs a big sweep
for wordy obtuse things. the userbase is changing and
we really have a lot of brand-new-to-programming types coming
in.
The :class:`_functions.array_agg` will now set the array dimensions to 1.
Improved :class:`_types.ARRAY` processing to accept ``None`` values as
value of a multi-array.
Mike Bayer [Thu, 15 Sep 2022 12:42:34 +0000 (08:42 -0400)]
catch exception for system_views also
Fixed yet another regression in SQL Server isolation level fetch (see
:ticket:`8231`, :ticket:`8475`), this time with "Microsoft Dynamics CRM
Database via Azure Active Directory", which apparently lacks the
``system_views`` view entirely. Error catching has been extended that under
no circumstances will this method ever fail, provided database connectivity
is present.
Mike Bayer [Tue, 13 Sep 2022 15:00:46 +0000 (11:00 -0400)]
Add type awareness to evaluator
Fixed regression where using ORM update() with synchronize_session='fetch'
would fail due to the use of evaluators that are now used to determine the
in-Python value for expressions in the the SET clause when refreshing
objects; if the evaluators make use of math operators against non-numeric
values such as PostgreSQL JSONB, the non-evaluable condition would fail to
be detected correctly. The evaluator now limits the use of math mutation
operators to numeric types only, with the exception of "+" that continues
to work for strings as well. SQLAlchemy 2.0 may alter this further by
fetching the SET values completely rather than using evaluation.
Mike Bayer [Thu, 8 Sep 2022 17:19:08 +0000 (13:19 -0400)]
additional de-stringify pass for unions
the change in c3cfee5b00a40790c18d took out
a pass for de-stringify that broke some un-tested cases
for Optional with future annotations mode. Adding tests
for this revealed that this was a subset of
a more general case where Union is presented
with ForwardRefs inside of it matching up within the type
map, which wasn't working before either, fixed that as well with
an additional de-stringify for elements within the Union.
Added long-requested case-insensitive string operators
:meth:`_sql.ColumnOperators.icontains`,
:meth:`_sql.ColumnOperators.istartswith`,
:meth:`_sql.ColumnOperators.iendswith`, which produce case-insensitive
LIKE compositions (using ILIKE on PostgreSQL, and the LOWER() function on
all other backends) to complement the existing LIKE composition operators
:meth:`_sql.ColumnOperators.contains`,
:meth:`_sql.ColumnOperators.startswith`, etc. Huge thanks to Matias
Martinez Rebori for their meticulous and complete efforts in implementing
these new methods.
Mike Bayer [Wed, 7 Sep 2022 21:13:23 +0000 (17:13 -0400)]
enable UPDATE..FROM for SQLite
The SQLite dialect now supports UPDATE..FROM syntax, for UPDATE statements
that may refer to additional tables within the WHERE criteria of the
statement without the need to use subqueries. This syntax is invoked
automatically when using the :class:`_dml.Update` construct when more than
one table or other entity or selectable is used.
Ensure that all cython extension are imported by the compied detection logic.
This is required since cython extensions moduels are marked as optional
in the install, so it's possible that only some of them are compiled.
The extensions are enabled only if all of them are correctly compiled
Fixed regression caused by the fix for :ticket:`8231` released in 1.4.40
where connection would fail if the user does not have permission to query
the dm_exec_sessions or dm_pdw_nodes_exec_sessions system view when trying
to determine the current transaction isolation level.
Peter Schutt [Thu, 1 Sep 2022 23:11:40 +0000 (19:11 -0400)]
Detection of PEP 604 union syntax.
### Description
Fixes #8478
Handle `UnionType` as arguments to `Mapped`, e.g., `Mapped[str | None]`:
- adds `utils.typing.is_optional_union()` used to detect if a column should be nullable.
- adds `"UnionType"` to `utils.typing.is_optional()` names.
- uses `get_origin()` in `utils.typing.is_origin_of()` as `UnionType` has no `__origin__` attribute.
- tests with runtime type and postponed annotations and guard the tests running with `compat.py310`.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Wed, 31 Aug 2022 15:07:23 +0000 (11:07 -0400)]
run update_subclass anytime we add new clslevel dispatch
Fixed event listening issue where event listeners added to a superclass
would be lost if a subclass were created which then had its own listeners
associated. The practical example is that of the :class:`.sessionmaker`
class created after events have been associated with the
:class:`_orm.Session` class.
Mike Bayer [Tue, 30 Aug 2022 14:47:24 +0000 (10:47 -0400)]
include TableClause.schema in cache key
Fixed issue where use of the :func:`_sql.table` construct, passing a string
for the :paramref:`_sql.table.schema` parameter, would fail to take the
"schema" string into account when producing a cache key, thus leading to
caching collisions if multiple, same-named :func:`_sql.table` constructs
with different schemas were used.
Mike Bayer [Tue, 30 Aug 2022 14:25:47 +0000 (10:25 -0400)]
implement event for merge/load=False for mutable state setup
Fixed issue in :mod:`sqlalchemy.ext.mutable` extension where collection
links to the parent object would be lost if the object were merged with
:meth:`.Session.merge` while also passing :paramref:`.Session.merge.load`
as False.
The event added here is currently private for expediency, but
is acceptable to become a public event at some point.
Mike Bayer [Tue, 30 Aug 2022 13:50:03 +0000 (09:50 -0400)]
apply consistent ORM mutable notes for all mutable SQL types
in
https://github.com/sqlalchemy/sqlalchemy/discussions/8447
I was surprised that we didnt have any notes about using Mutable
for ARRAY classes, since we have them for HSTORE and JSON.
Add a consistent topic box for these so we have something to
point towards.
Mike Bayer [Mon, 29 Aug 2022 14:43:36 +0000 (10:43 -0400)]
refine ruleset to determine when poly adaption should be used
Fixed regression appearing in the 1.4 series where a joined-inheritance
query placed as a subquery within an enclosing query for that same entity
would fail to render the JOIN correctly for the inner query. The issue
manifested in two different ways prior and subsequent to version 1.4.18
(related issue #6595), in one case rendering JOIN twice, in the other
losing the JOIN entirely. To resolve, the conditions under which
"polymorphic loading" are applied have been scaled back to not be invoked
for simple joined inheritance queries.
Trevor Gross [Wed, 24 Aug 2022 10:32:31 +0000 (06:32 -0400)]
Use cibuildwheel to create wheels
Using cibuildwheel the following wheels are created:
- windows x64 and x84
- macos x64 and arm
- linux x64 and arm on manylinux and mosulinux (for alpine)
- create a pure python wheel (for pypy and other archs)
Mike Bayer [Wed, 24 Aug 2022 17:06:09 +0000 (13:06 -0400)]
fix mock issue in pool test
Due to the change for #5648 in ad14471bc99c2fb2315d3333a95dd,
the mock in test_recycle_pool_no_race needs to implement
an empty handle_error dispatcher.
the exception was in a thread so did not interrupt the
test suite:
File "/home/classic/dev/sqlalchemy/lib/sqlalchemy/engine/base.py", line 2059, in _handle_dbapi_exception
for fn in self.dialect.dispatch.handle_error:
TypeError: 'Mock' object is not iterable
Mike Bayer [Tue, 23 Aug 2022 13:28:06 +0000 (09:28 -0400)]
integrate connection.terminate() for supporting dialects
Integrated support for asyncpg's ``terminate()`` method call for cases
where the connection pool is recycling a possibly timed-out connection,
where a connection is being garbage collected that wasn't gracefully
closed, as well as when the connection has been invalidated. This allows
asyncpg to abandon the connection without waiting for a response that may
incur long timeouts.
Mike Bayer [Mon, 22 Aug 2022 17:43:59 +0000 (13:43 -0400)]
reformat functions.rst; document coalsce
this file was all over the place autodocumenting all the
contents of functions.py with no regards to the heading
paragraph which seemed to be introducing the generic functions.
Use specific autoclass/autofunc docs as automodule is generally
unworkable.
Mike Bayer [Sun, 21 Aug 2022 15:58:20 +0000 (11:58 -0400)]
Column._copy() duplicates "user defined" nullable state exactly
To accommodate how mapped_column() works, after many
attempts to get this working it became clear that _copy()
should just transfer "nullable" state exactly as it was,
including the state where .nullable was set but user_defined_nullable
remains at not user set.
additionally, added a similar step to _merge() that was needed
to preserve the nullability behavior when Identity is present.
server / client default objects are not copied within column._copy()
and this should be fixed.
Mike Bayer [Tue, 16 Aug 2022 14:44:48 +0000 (10:44 -0400)]
support create/drop events for all CREATE/DROP
Implemented the DDL event hooks :meth:`.DDLEvents.before_create`,
:meth:`.DDLEvents.after_create`, :meth:`.DDLEvents.before_drop`,
:meth:`.DDLEvents.after_drop` for all :class:`.SchemaItem` objects that
include a distinct CREATE or DROP step, when that step is invoked as a
distinct SQL statement, including for :class:`.ForeignKeyConstraint`,
:class:`.Sequence`, :class:`.Index`, and PostgreSQL's
:class:`_postgresql.ENUM`.
Mike Bayer [Fri, 19 Aug 2022 15:12:41 +0000 (11:12 -0400)]
remove narrative "reconstructor" document
this event hook is not commonly used and this page does not
fit into the current narrative very well. We should possibly
write a new paragraph regarding how instances load
at some point though the best place to put it is not clear.
Mike Bayer [Fri, 19 Aug 2022 13:49:25 +0000 (09:49 -0400)]
fix some inaccuracies in with_variant doc
* the table wont create on mysql/mariadb b.c. user_name had
no length
* "utf-8" is not recognized by mysql/mariadb, use "utf8"
* mysql or mariadb name match is determined by the URL, not the
actual DB that is detected (I know I made it work that way but
I forgot)
* for the 1.4 backport only, will remove the "mariadb" part
as we dont support that API, #8408
Mike Bayer [Wed, 17 Aug 2022 21:47:19 +0000 (17:47 -0400)]
improve abstractconcretebase
try to limit the attributes on the base and set up wpoly
etc so that things still work the way we want.
It seems like I've tried this in the past before so not sure
if this is actually working or if there are problems. it needs
a little more strictness on how you set up your base since
attributes are no longer implicit. So, it seems like perhaps
the new behavior should be on a flag or something like
"strict_attributes=True", something like that, so that nothing
breaks for existing users and we can slowly deal with the new
way being a little bit less worse than the old way.
Mike Bayer [Tue, 9 Aug 2022 17:31:14 +0000 (13:31 -0400)]
validate mapped collection key is loaded
Changed the attribute access method used by
:func:`_orm.attribute_mapped_collection` and
:func:`_orm.column_mapped_collection`, used when populating the dictionary,
to assert that the data value on the object to be used as the dictionary
key is actually present, and is not instead using "None" due to the
attribute never being actually assigned. This is used to prevent a
mis-population of None for a key when assigning via a backref where the
"key" attribute on the object is not yet assigned.
As the failure mode here is a transitory condition that is not typically
persisted to the database, and is easy to produce via the constructor of
the class based on the order in which parameters are assigned, it is very
possible that many applications include this behavior already which is
silently passed over. To accommodate for applications where this error is
now raised, a new parameter
:paramref:`_orm.attribute_mapped_collection.ignore_unpopulated_attribute`
is also added to both :func:`_orm.attribute_mapped_collection` and
:func:`_orm.column_mapped_collection` that instead causes the erroneous
backref assignment to be skipped.
Mike Bayer [Tue, 16 Aug 2022 18:25:12 +0000 (14:25 -0400)]
refine transfer of cached ORM options for selectin, lazy
Fixed issue involving :func:`_orm.with_loader_criteria` where a closure
variable used as bound parameter value within the lambda would not carry
forward correctly into additional relationship loaders such as
:func:`_orm.selectinload` and :func:`_orm.lazyload` after the statement
were cached, using the stale originally-cached value instead.
This change brings forth a good refinement where we finally realize
we shouldn't be testing every ORM option with lots of switches, we
just let the option itself be given "here is your uncached version,
you are cached, tell us what to do!". the current decision is
that strategy loader options used the cached in all cases as they
always have, with_loader_criteria uses the uncached, because the
uncached will have been invoked with new closure state that we
definitely need. The only
edge that might not work is if with_loader_criteria referenced
an entity that is local to the query, namely a specific AliasedInsp,
however that's not a documented case for this. if we had to do that,
then we perhaps would introduce a more complex reconcilation
logic, and this would also give us the hook to do that.
Mike Bayer [Wed, 17 Aug 2022 17:06:51 +0000 (13:06 -0400)]
fill out all distinguising fields for AliasedInsp
Hardened the cache key strategy for the :func:`_orm.aliased` and
:func:`_orm.with_polymorphic` constructs. While no issue involving actual
statements being cached can easily be demonstrated (if at all), these two
constructs were not including enough of what makes them unique in their
cache keys for caching on the aliased construct alone to be accurate.
Introduced the type :class:`_postgresql.JSONPATH` that can be used
in cast expressions. This is required by some PostgreSQL dialects
when using functions such as ``jsonb_path_exists`` or
``jsonb_path_match`` that accept a ``jsonpath`` as input.
Mike Bayer [Wed, 10 Aug 2022 14:53:11 +0000 (10:53 -0400)]
Propagate key for collection events
Added new parameter :paramref:`_orm.AttributeEvents.include_key`, which
will include the dictionary or list key for operations such as
``__setitem__()`` (e.g. ``obj[key] = value``) and ``__delitem__()`` (e.g.
``del obj[key]``), using a new keyword parameter "key" or "keys", depending
on event, e.g. :paramref:`_orm.AttributeEvents.append.key`,
:paramref:`_orm.AttributeEvents.bulk_replace.keys`. This allows event
handlers to take into account the key that was passed to the operation and
is of particular importance for dictionary operations working with
:class:`_orm.MappedCollection`.
Mike Bayer [Fri, 12 Aug 2022 22:35:14 +0000 (18:35 -0400)]
restore Connectable and other changes for oslo.db
at this point oslo.db should be passing tests however
they seem to be importing "Connectable", which we had
temporarily removed, it's now a different kind of object
but restore it to the engine import space which seems
to be where they're pulling it from to see if oslo.db
can run some tests now.
RamonWill [Tue, 25 Aug 2020 00:17:19 +0000 (20:17 -0400)]
Add support for Partitioning and Sample pages on mysql
Add support for Partitioning and Sample pages on MySQL and MariaDB
reflected options.
The options are stored in the table dialect options dictionary, so
the following keyword need to be prefixed with ``mysql_`` or ``mariadb_``
depending on the backend.
Supported options are:
These options are also reflected when loading a table from database,
and will populate the table :attr:`_schema.Table.dialect_options`.
Pull request courtesy of Ramon Will.
Mike Bayer [Thu, 11 Aug 2022 14:20:49 +0000 (10:20 -0400)]
reorg bulk persistence into a separate module
This restores persistence.py to only functions that are used
by unitofwork.py, and all the "bulk" stuff gets its own
module bulk_persistence.py. Also fixes up the ORM context
class hierarchy for bulk.
This is all ahead of the ORM-insert changes coming in, so that
the later review can be about logic and not about reorganization.
Daniel Hall [Sat, 30 Jul 2022 19:12:20 +0000 (15:12 -0400)]
Support comments on MSSQL
Added support table and column comments on MSSQL when
creating a table. Added support for reflecting table comments.
Thanks to Daniel Hall for the help in this pull request.
Mike Bayer [Wed, 10 Aug 2022 16:40:17 +0000 (12:40 -0400)]
doc fixes
* fixed erroneous use of mapped_column() in m2m relationship Table
* Fill in full imports for some relationship examples that had
partial imports; examples that have no imports, leave empty for now
* converted joined/single inh mappings to annotated style
* We have a problem with @declared_attr in that the error message
is wrong if the mapped_column() returned doesnt have a type, and/or
mapped_column() with @declared_attr doesnt use the annotation
* fix thing where sphinx with undoc-members global setting seems to
no longer tolerate ":attribute:" entries in autodoc classes, which
is fine we can document the annotations now
* Fix mapper params in inheritance to be on Mapper
* add missing changelog file for instances remove
Mike Bayer [Tue, 9 Aug 2022 13:50:57 +0000 (09:50 -0400)]
improve names for datatypes section
Continuing along #8362, if we look at the document here without
"this is the same text I've read for 14 years", we begin to see
that the title "Column and Data types" makes no sense at all,
is there a "column type" and a "Data type"? I guess what I was
thinking at that time is that a type can be placed on a
Column, or it can be used whenever you have, you know, "data".
The phrase "SQL expression" wasn't discovered yet.
"SQL Datatype" is not spectacular but at least it's one term.
the new intro then is focused on the hierarchy layout so let's
name it that.
not amazing, but better than the duplicate name that made
no sense before.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
