Mike Bayer [Thu, 16 Sep 2021 17:38:11 +0000 (13:38 -0400)]
use the stack for insert_from_select
Fixed issue related to new ``add_cte()`` feature where pairing two
"INSERT..FROM SELECT" statements simultaneously would lose track of the two
independent SELECT statements, leading to the wrong SQL.
Daniel Black [Sun, 12 Sep 2021 05:17:50 +0000 (01:17 -0400)]
test update_nowait - added support in MariaDB-10.3
<!-- Provide a general summary of your proposed changes in the Title field above -->
MariaDB supported NOWAIT in 10.3.
ref: https://mariadb.com/kb/en/wait-and-nowait/
This pull request is:
- [X] very trivial test change
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Miguel Grinberg [Mon, 13 Sep 2021 18:41:13 +0000 (14:41 -0400)]
Add scalars method to connection and session classes
Added new methods :meth:`_orm.Session.scalars`,
:meth:`_engine.Connection.scalars`, :meth:`_asyncio.AsyncSession.scalars`
and :meth:`_asyncio.AsyncSession.stream_scalars`, which provide a short cut
to the use case of receiving a row-oriented :class:`_result.Result` object
and converting it to a :class:`_result.ScalarResult` object via the
:meth:`_engine.Result.scalars` method, to return a list of values rather
than a list of rows. The new methods are analogous to the long existing
:meth:`_orm.Session.scalar` and :meth:`_engine.Connection.scalar` methods
used to return a single value from the first row only. Pull request
courtesy Miguel Grinberg.
Ensure that ``str()`` is called on the an ``URL.password`` argument,
allowing usage of objects that implement the ``__str__()`` method
as password attributes.
Also clarified that one such object is not appropriate to dynamically
change the password.
Mike Bayer [Mon, 13 Sep 2021 18:38:02 +0000 (14:38 -0400)]
replace "in" operator with "==" + "or" for greenlet platforms
Further adjusted the "greenlet" package specifier in setup.cfg to use a
long chain of "or" expressions, so that the comparison of
``platform_machine`` to a specific identifier matches only the complete
string.
* test_schema2 isn't needed in MySQL/MariaDB test.
* Only permissions on test_schema are needed.
* User without qualification defaults to user@'%'.
* Use MARIADB_* names with MariaDB container that has
been there most of this year in releases.
* We don't need to specify a database on connection
when creating users/databases.
* On all containers - docker will always pull image
if not available. Also postgres image didn't match
container initialization.
* Remove container versions, just use latest
Mike Bayer [Tue, 30 Mar 2021 19:22:01 +0000 (15:22 -0400)]
ignore and warn for native_enum=False with pg.ENUM datatype
any UPPERCASE datatype refers to that exact type name rendered
on the database. So PG's ENUM must render "ENUM" and is
"native" by definition. warn if this flag is passed.
The :class:`_postgresql.ENUM` datatype is PostgreSQL-native and therefore
should not be used with the ``native_enum=False`` flag. This flag is now
ignored if passed to the :class:`_postgresql.ENUM` datatype and a warning
is emitted; previously the flag would cause the type object to fail to
function correctly.
Mike Bayer [Fri, 10 Sep 2021 14:07:15 +0000 (10:07 -0400)]
bulk_save->return_defaults does not populate server defaults
it fetches them but doesn't put them on the objects, which
is likely a bug, however i have no intention of working
on the bulk methods. Add more documentation to strongly
discourage these methods in favor of more explicit bulk
approaches, e.g. using an insert() construct directly.
Daniel Black [Wed, 8 Sep 2021 07:12:16 +0000 (03:12 -0400)]
test_for_update to recognise skip_locked for MariaDB-10.6
### Description
Alters the test for skip_locked to recognize that `SKIP LOCKED` was added to the MariaDB syntax in 10.6.0 https://mariadb.com/kb/en/select/#skip-locked.
### Checklist
This pull request is:
- [X] A minor test case fix
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
$ pytest --db mariadb --dburi mariadb://scott:tiger@127.0.0.1:43809/test test/dialect/mysql/test_for_update.py
...
INTERNALERROR> File "/home/dan/.py3/lib64/python3.9/site-packages/MySQLdb/connections.py", line 185, in __init__
INTERNALERROR> super().__init__(*args, **kwargs2)
INTERNALERROR> sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2002, "Can't connect to MySQL server on '127.0.0.1' (115)")
INTERNALERROR> (Background on this error at: https://sqlalche.me/e/14/e3q8)
```
Some queries where executed on the container instance however there's something in the test hard-coded to 3306 (observed in strace). And/or I'm doing something incorrectly.
Mike Bayer [Thu, 2 Sep 2021 18:11:38 +0000 (14:11 -0400)]
turn off deduping for col expressions
Fixed ORM issue where column expressions passed to ``query()`` or
ORM-enabled ``select()`` would be deduplicated on the identity of the
object, such as a phrase like ``select(A.id, null(), null())`` would
produce only one "NULL" expression, which previously was not the case in
1.3. However, the change also allows for ORM expressions to render as given
as well, such as ``select(A.data, A.data)`` will produce a result row with
two columns.
Daniel Stone [Mon, 30 Aug 2021 15:15:25 +0000 (11:15 -0400)]
Added loader options to session.merge, asyncsession.merge
Added loader options to :meth:`_orm.Session.merge` and
:meth:`_asyncio.AsyncSession.merge`, which will apply the given loader
options to the ``get()`` used internally by merge, allowing eager loading
of relationships etc. to be applied when the merge process loads a new
object. Pull request courtesy Daniel Stone.
Mike Bayer [Wed, 1 Sep 2021 12:58:06 +0000 (08:58 -0400)]
add asyncio.gather() example; add connection opts
while I dont like this approach very much, people will likely
be asking for it a lot, so represent the most correct and
efficient form we can handle right now.
Added missing ``**kw`` arguments to the
:meth:`_asyncio.AsyncSession.connection` method.
Mike Bayer [Tue, 31 Aug 2021 17:34:43 +0000 (13:34 -0400)]
Fix and test sequences w/ executemany in pre-exec scenarios
Fixed issue where an engine that had ``implicit_returning`` set to False
would fail to function when PostgreSQL's "fast insertmany" feature were
used in conjunction with a ``Sequence``, as well as if any kind of
"executemany" with "return_defaults()" were used in conjunction with a
``Sequence``. Note that PostgreSQL "fast insertmany" uses "RETURNING" by
definition, when the SQL statement is passed to the driver; overall, the
``implicit_returning`` flag is legacy and has no real use in modern
SQLAlchemy, and will be deprecated in a separate change.
Mike Bayer [Tue, 31 Aug 2021 20:41:58 +0000 (16:41 -0400)]
remove async session slots
1798c3cf1c added __slots__ to the base which then
caused af0824fd79 to fail. as we can't use an assignable
class variable with slots, remove slots from AsyncSession
which is how this class was working anyway.
Fixed bug where the error message for SQLite invalid isolation level on the
pysqlite driver would fail to indicate that "AUTOCOMMIT" is one of the
valid isolation levels.
Federico Caselli [Fri, 27 Aug 2021 20:45:56 +0000 (22:45 +0200)]
Allow custom sync session class in ``AsyncSession``.
The :class:`_asyncio.AsyncSession` now supports overriding which
:class:`_orm.Session` it uses as the proxied instance. A custom ``Session``
class can be passed using the :paramref:`.AsyncSession.sync_session_class`
parameter or by subclassing the ``AsyncSession`` and specifying a custom
:attr:`.AsyncSession.sync_session_class`.
Federico Caselli [Thu, 26 Aug 2021 20:00:33 +0000 (22:00 +0200)]
Handle mappings passed to ``execution_options``.
Fixed a bug in :meth:`_asyncio.AsyncSession.execute` and
:meth:`_asyncio.AsyncSession.stream` that required ``execution_options``
to be an instance of ``immutabledict`` when defined. It now
correctly accepts any mapping.
Mike Bayer [Wed, 25 Aug 2021 17:12:55 +0000 (13:12 -0400)]
ensure "sqlalchemy" info set for all considered classes
Fixed issue in mypy plugin where columns on a mixin would not be correctly
interpreted if the mapped class relied upon a ``__tablename__`` routine
that came from a superclass.
jonathan vanasco [Mon, 23 Aug 2021 20:25:21 +0000 (16:25 -0400)]
standardizing docs #6821
(redo of 2999/I5609025feee8cfdecc09b55bfbf1bd13fa2e6602)
This PR is designed to bring more clarity within the docs by renaming object
instances that may be consfusingly similar to class, method, and attribute names.
For example, instances of the class `MetaData` are available on some objects as
`.metadata` property, and had appeared within the docs as both `meta` and
`metadata` which has confused some users in the past. By this PR, the docs now
utilize the following naming convention:
* MetaData - SQLAlchemy class
* .metadata - SQLAlchemy API attributes
* metadata_obj - developer instantiated metadata objects or references
Detailed Changes:
* standardized `meta` and `metadata` instances to `metadata_obj`. note: the docs were evenly split between 'meta' and 'metadata'.
* standardized 'cursor' to 'cursor_obj' to avoid confusion with the method.
* standardized a 'scalar_subquery = ' to 'scalar_subq' to avoid confusion with the method.
* standardized a 'cte = ' to 'cte_obj' to avoid confusion with the method
Mike Bayer [Mon, 23 Aug 2021 15:24:48 +0000 (11:24 -0400)]
qualify compile_state updates for non-current entities
Fixed issue in recently repaired ``Query.with_entities()`` method where the
flag that determines automatic uniquing for legacy ORM ``Query`` objects
only would be set to ``True`` inappropriately in cases where the
``with_entities()`` call would be setting the ``Query`` to return
column-only rows, which are not uniqued.
Mike Bayer [Fri, 20 Aug 2021 15:47:26 +0000 (11:47 -0400)]
restore statement substitution to before_execute()
Fixed issue where the ability of the
:meth:`_engine.ConnectionEvents.before_execute` method to alter the SQL
statement object passed, returning the new object to be invoked, was
inadvertently removed. This behavior has been restored.
The refactor in a1939719a652774a437f69f8d4788b3f08650089 removed this
feature for some reason and there were no tests in place to detect
it. I don't see any indication this was planned.
Jiayang [Thu, 19 Aug 2021 09:23:51 +0000 (02:23 -0700)]
mysql://user:pass@host/dbname instead of pass:host
Summary:
Reading [1 sqlalchemy.future.Engine](https://docs.sqlalchemy.org/en/14/core/future.html#sqlalchemy.future.Engine) and [2 sqlalchemy.create_engine](https://docs.sqlalchemy.org/en/14/core/engines.html#sqlalchemy.create_engine). I fond @ should be used between pass & host.
Test Plan:
I checked with sqlalchemy 1.4.22 locally and I think @ should be used.
Mike Bayer [Wed, 18 Aug 2021 14:48:16 +0000 (10:48 -0400)]
CAST Oracle table_name, owner, others to VARCHAR(128)
Added a CAST(VARCHAR2(128)) to the "table name", "owner", and other
DDL-name parameters as used in reflection queries against Oracle system
views such as ALL_TABLES, ALL_TAB_CONSTRAINTS, etc to better enable
indexing to take place against these columns, as they previously would be
implicitly handled as NVARCHAR2 due to Python's use of Unicode for strings;
these columns are documented in all Oracle versions as being VARCHAR2 with
lengths varying from 30 to 128 characters depending on server version.
Additionally, test support has been enabled for Unicode-named DDL
structures against Oracle databases.
Mike Bayer [Tue, 17 Aug 2021 19:03:57 +0000 (15:03 -0400)]
send user defined options from the current query
Revised the means by which the
:attr:`_orm.ORMExecuteState.user_defined_options` accessor receives
:class:`_orm.UserDefinedOption` and related option objects from the
context, with particular emphasis on the "selectinload" on the loader
strategy where this previously was not working; other strategies did not
have this problem. The objects that are associated with the current query
being executed, and not that of a query being cached, are now propagated
unconditionally. This essentially separates them out from the "loader
strategy" options which are explicitly associated with the compiled state
of a query and need to be used in relation to the cached query.
The effect of this fix is that a user-defined option, such as those used
by the dogpile.caching example as well as for other recipes such as
defining a "shard id" for the horizontal sharing extension, will be
correctly propagated to eager and lazy loaders regardless of whether
a cached query was ultimately invoked.
Mike Bayer [Fri, 13 Aug 2021 15:07:17 +0000 (11:07 -0400)]
rewrite _extra_criteria in selectinload; propagate correctly to Load
Fixed issue in :func:`_orm.selectinload` where use of the new
:meth:`_orm.PropComparator.and_` feature within options that were nested
more than one level deep would fail to update bound parameter values that
were in the nested criteria, as a side effect of SQL statement caching.
Implementation adds a new step that rewrites the parameters
inside of all _extra_criteria when invoking selectinload
as well as subqueryload. Additionally, changed how Load()
gets "extra_criteria", in that it pulls it from
UnboundLoad._extra_criteria instead of re-fetching it from the
path elements, which are not updated by this new step.
This patch also builds upon the removal of lambda queries
for use in loader strategies in #6889. lambdas made this issue
much more difficult to diagnose. An attempt to reintroduce
lambdas here after finally identifying the "extra_criteria"
issue above showed that lambdas still impact the
assertsql fixture, meaning we have a statement structure that
upon calling .compile() still delivers stale data due to lambdas,
even if caching is turned off, and the non-cached test was still
failing due to stale data within the lambdas.
This is basically the complexity that #6889 fixes and as there's
no real performance gain to using lambdas in these strategies
on top of the existing statement caching that does most of the
work, it should be much less likely going forward to have as many
deeply confusing issues as we've had within selectinload/lazyload
in the 1.4 series.
Mike Bayer [Sun, 15 Aug 2021 22:12:42 +0000 (18:12 -0400)]
honor NO_CACHE in lambdas
Fixed issue in lambda caching system where an element of a query that
produces no cache key, like a custom option or clause element, would still
populate the expression in the "lambda cache" inappropriately.
This was discovered as part of :ticket:`6887` but is a separate
issue.
Mike Bayer [Mon, 16 Aug 2021 21:20:48 +0000 (17:20 -0400)]
remove lambda caching from loader strategies
Adjusted ORM loader internals to no longer use the "lambda caching" system
that was added in 1.4, as well as repaired one location that was still
using the previous "baked query" system for a query. The lambda caching
system remains an effective way to reduce the overhead of building up
queries that have relatively fixed usage patterns. In the case of loader
strategies, the queries used are responsible for moving through lots of
arbitrary options and criteria, which is both generated and sometimes
consumed by end-user code, that make the lambda cache concept not any more
efficient than not using it, at the cost of more complexity. In particular
the problems noted by :ticket:`6881` and :ticket:`6887` are made
considerably less complicated by removing this feature internally.
Fixed an issue where the :class:`_orm.Bundle` construct would not create
proper cache keys, leading to inefficient use of the query cache. This
had some impact on the "selectinload" strategy and was identified as
part of :ticket:`6889`.
Added a Select._create_raw_select() method which essentially
performs ``__new__`` and then populates ``__dict__`` directly,
with no coercions. This saves most of the overhead time that
the lambda caching system otherwise seeks to avoid.
Includes removal of bakedquery from
mapper->_subclass_load_via_in() which was overlooked from
the 1.4 refactor.
Mike Bayer [Sun, 15 Aug 2021 16:21:13 +0000 (12:21 -0400)]
fix linter JOIN logic; fix PostgreSQL ARRAY op comparison
Adjusted the "from linter" warning feature to accommodate for a chain of
joins more than one level deep where the ON clauses don't explicitly match
up the targets, such as an expression such as "ON TRUE". This mode of use
is intended to cancel the cartesian product warning simply by the fact that
there's a JOIN from "a to b", which was not working for the case where the
chain of joins had more than one element.
this incurs a bit more compiler overhead that comes out in profiling
but is not extensive.
Added the "is_comparison" flag to the PostgreSQL "overlaps",
"contained_by", "contains" operators, so that they work in relevant ORM
contexts as well as in conjunction with the "from linter" feature.
Mike Bayer [Mon, 26 Jul 2021 22:06:41 +0000 (18:06 -0400)]
limit greenlet dependency to pypi-listed platforms
The setup requirements have been modified such ``greenlet`` is a default
requirement only for those platforms that are well known for ``greenlet``
to be installable and for which there is already a pre-built binary on
pypi; the current list is ``x86_64 aarch64 ppc64le amd64 win32``. For other
platforms, greenlet will not install by default, which should enable
installation and test suite running of SQLAlchemy 1.4 on platforms that
don't support ``greenlet``, excluding any asyncio features. In order to
install with the ``greenlet`` dependency included on a machine architecture
outside of the above list, the ``[asyncio]`` extra may be included by
running ``pip install sqlalchemy[asyncio]`` which will then attempt to
install ``greenlet``.
Additionally, the test suite has been repaired so that tests can complete
fully when greenlet is not installed, with appropriate skips for
asyncio-related tests.
João Sampaio [Wed, 11 Aug 2021 14:36:49 +0000 (11:36 -0300)]
Fix example with deprecated code
Just a few lines above, it is said that creating a mapping with `mapper()` is deprecated in favor of `mapper_registry.map_imperatively()`. This PR fixes an example in the documentation that still uses the old function.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
