Mike Bayer [Wed, 27 Jul 2022 15:36:57 +0000 (11:36 -0400)]
implement tuple-slices from .c collections
Added new syntax to the ``.c`` collection on all :class:`.FromClause`
objects allowing tuples of keys to be passed to ``__getitem__()``, along
with support for ``select()`` handling of ``.c`` collections directly,
allowing the syntax ``select(table.c['a', 'b', 'c'])`` to be possible. The
sub-collection returned is itself a :class:`.ColumnCollection` which is
also directly consumable by :func:`_sql.select` and similar now.
The PostgreSQL dialect now supports reflection of expression based indexes.
The reflection is supported both when using
:meth:`_engine.Inspector.get_indexes` and when reflecting a
:class:`_schema.Table` using :paramref:`_schema.Table.autoload_with`.
Thanks to immerrr and Aidan Kane for the help on this ticket.
Mike Bayer [Thu, 7 Jul 2022 16:07:39 +0000 (12:07 -0400)]
Use FETCH FIRST N ROWS / OFFSET for Oracle LIMIT/OFFSET
Oracle will now use FETCH FIRST N ROWS / OFFSET syntax for limit/offset
support by default for Oracle 12c and above. This syntax was already
available when :meth:`_sql.Select.fetch` were used directly, it's now
implied for :meth:`_sql.Select.limit` and :meth:`_sql.Select.offset` as
well.
I'm currently setting this up so that the new syntax renders
in Oracle using POSTCOMPILE binds. I really have no indication
if Oracle's SQL optimizer would be better with params
here, so that it can cache the SQL plan, or if it expects
hardcoded numbers for these. Since we had reports that the previous
ROWNUM thing really needed hardcoded ints, let's guess
for now that hardcoded ints would be preferable. it can be turned
off with a single boolean if users report that they'd prefer
real bound values.
Mike Bayer [Thu, 23 Jun 2022 20:43:49 +0000 (16:43 -0400)]
add an extra load for non-new but unloaded
Made an improvement to the "deferred" / "load_only" set of strategy options
where if a certain object is loaded from two different logical paths within
one query, attributes that have been configured by at least one of the
options to be populated will be populated in all cases, even if other load
paths for that same object did not set this option. previously, it was
based on randomness as to which "path" addressed the object first.
Mike Bayer [Tue, 19 Jul 2022 17:03:51 +0000 (13:03 -0400)]
fixes for mypy 0.971
things that were passing with 0.961 need adjustment.
it seems mypy has become very pedantic about the difference
between importing from a module vs. accessing members of that
module as instance variables, so adjust the preloaded
typing block to be explicitly instance variables, since that's
how the accessor works in any case.
Mike Bayer [Tue, 19 Jul 2022 14:50:05 +0000 (10:50 -0400)]
check for TypeDecorator when handling getitem
Fixed issue where :class:`.TypeDecorator` would not correctly proxy the
``__getitem__()`` operator when decorating the :class:`.ARRAY` datatype,
without explicit workarounds.
Mike Bayer [Mon, 18 Jul 2022 16:52:28 +0000 (12:52 -0400)]
implement executemany RETURNING for Oracle
this works straight out of the box as we can expand
upon what we did for #6245 to also receive for multiple
statements. Oracle "fast ORM insert" then is basically done.
Federico Caselli [Fri, 17 Jun 2022 21:12:39 +0000 (23:12 +0200)]
add shield() in aexit
Added ``asyncio.shield()`` to the connection and session release process
specifically within the ``__aexit__()`` context manager exit, when using
:class:`.AsyncConnection` or :class:`.AsyncSession` as a context manager
that releases the object when the context manager is complete. This appears
to help with task cancellation when using alternate concurrency libraries
such as ``anyio``, ``uvloop`` that otherwise don't provide an async context
for the connection pool to release the connection properly during task
cancellation.
Mike Bayer [Mon, 18 Jul 2022 12:48:55 +0000 (08:48 -0400)]
add contextmanager typing, open run_sync typing
was missing AsyncConnection type for the async
context manager.
fixing that revealed that _SyncConnectionCallable
and _SyncSessionCallable protocols are infeasible because
the given callable can have a lot of different signatures
that are compatible.
Mike Bayer [Sun, 17 Jul 2022 15:32:27 +0000 (11:32 -0400)]
use concat() directly for contains, startswith, endswith
Adjusted the SQL compilation for string containment functions
``.contains()``, ``.startswith()``, ``.endswith()`` to force the use of the
string concatenation operator, rather than relying upon the overload of the
addition operator, so that non-standard use of these operators with for
example bytestrings still produces string concatenation operators.
To accommodate this, needed to add a new _rconcat operator function,
which is private, as well as a fallback in concat_op() that works
similarly to Python builtin ops.
Mike Bayer [Sat, 16 Jul 2022 20:19:15 +0000 (16:19 -0400)]
implement column._merge()
this takes the user-defined args of one Column and merges
them into the not-user-defined args of another Column.
Implemented within the pep-593 column transfer operation
to begin to make this new feature more robust.
work may still be needed for constraints etc. but
in theory everything from the left side annotated column
should take effect for the right side if not otherwise
specified on the right.
Mike Bayer [Fri, 15 Jul 2022 16:25:22 +0000 (12:25 -0400)]
make anno-only Mapped[] column available for mixins
Documentation is relying on the recently improved
behavior of produce_column_copies() to make sure everything is
available on cls for a declared_attr. therefore for anno-only
attribute, we also need to generate the mapped_column() up front
before scan is called.
noticed in pylance, allow @declared_attr to recognize
@classmethod also which allows letting typing tools know
something is explicitly a classmethod
Mike Bayer [Mon, 11 Jul 2022 01:24:17 +0000 (21:24 -0400)]
support "SELECT *" for ORM queries
A :func:`_sql.select` construct that is passed a sole '*' argument for
``SELECT *``, either via string, :func:`_sql.text`, or
:func:`_sql.literal_column`, will be interpreted as a Core-level SQL
statement rather than as an ORM level statement. This is so that the ``*``,
when expanded to match any number of columns, will result in all columns
returned in the result. the ORM- level interpretation of
:func:`_sql.select` needs to know the names and types of all ORM columns up
front which can't be achieved when ``'*'`` is used.
If ``'*`` is used amongst other expressions simultaneously with an ORM
statement, an error is raised as this can't be interpreted correctly by the
ORM.
Mike Bayer [Thu, 7 Jul 2022 15:44:09 +0000 (11:44 -0400)]
document using fetch() with Oracle
We implemented working FETCH support, but it's not
yet implied by limit/offset. The docs make no mention
that this is available which is very misleading including
to maintainers. Make it clear that fetch() support is
there right now, it's just not yet implicit with
limit/offset.
Mike Bayer [Wed, 6 Jul 2022 01:05:18 +0000 (21:05 -0400)]
generalize sql server check for id col to accommodate ORM cases
Fixed issues that prevented the new usage patterns for using DML with ORM
objects presented at :ref:`orm_dml_returning_objects` from working
correctly with the SQL Server pyodbc dialect.
Here we add a step to look in compile_state._dict_values more thoroughly
for the keys we need to determine "identity insert" or not, and also
add a new compiler variable dml_compile_state so that we can skip the
ORM's compile_state if present.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Mike Bayer [Sun, 3 Jul 2022 17:55:20 +0000 (13:55 -0400)]
test transfer of default, insert_default
right now "default" goes to Column.default unconditionally
if insert_default is not present, including if dataclasses
are in use where the field effectively now does two things.
This generally works out because Python side default
can be assigned to the object or picked up by Core in any case.
However, we might want to look into later on migrating this
to have the fields act more separately. I think it's
"OK" for now, will try to doc that this might change.
Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.
- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions
[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)
Mike Bayer [Thu, 30 Jun 2022 23:10:06 +0000 (19:10 -0400)]
repair yield_per for non-SS dialects and add new options
Implemented new :paramref:`_engine.Connection.execution_options.yield_per`
execution option for :class:`_engine.Connection` in Core, to mirror that of
the same :ref:`yield_per <orm_queryguide_yield_per>` option available in
the ORM. The option sets both the
:paramref:`_engine.Connection.execution_options.stream_results` option at
the same time as invoking :meth:`_engine.Result.yield_per`, to provide the
most common streaming result configuration which also mirrors that of the
ORM use case in its usage pattern.
Fixed bug in :class:`_engine.Result` where the usage of a buffered result
strategy would not be used if the dialect in use did not support an
explicit "server side cursor" setting, when using
:paramref:`_engine.Connection.execution_options.stream_results`. This is in
error as DBAPIs such as that of SQLite and Oracle already use a
non-buffered result fetching scheme, which still benefits from usage of
partial result fetching. The "buffered" strategy is now used in all
cases where :paramref:`_engine.Connection.execution_options.stream_results`
is set.
Added :meth:`.FilterResult.yield_per` so that result implementations
such as :class:`.MappingResult`, :class:`.ScalarResult` and
:class:`.AsyncResult` have access to this method.
Mike Bayer [Tue, 28 Jun 2022 22:55:19 +0000 (18:55 -0400)]
produce column copies up the whole hierarchy first
Fixed issue where a hierarchy of classes set up as an abstract or mixin
declarative classes could not declare standalone columns on a superclass
that would then be copied correctly to a :class:`_orm.declared_attr`
callable that wanted to make use of them on a descendant class.
Originally it looked like this would produce an ordering change,
however an adjustment to the flow for produce_column_copies
has avoided that for now.
Gord Thompson [Sat, 25 Jun 2022 16:34:51 +0000 (10:34 -0600)]
Change setinputsizes behavior for mssql+pyodbc
The ``use_setinputsizes`` parameter for the ``mssql+pyodbc`` dialect now
defaults to ``True``; this is so that non-unicode string comparisons are
bound by pyodbc to pyodbc.SQL_VARCHAR rather than pyodbc.SQL_WVARCHAR,
allowing indexes against VARCHAR columns to take effect. In order for the
``fast_executemany=True`` parameter to continue functioning, the
``use_setinputsizes`` mode now skips the ``cursor.setinputsizes()`` call
specifically when ``fast_executemany`` is True and the specific method in
use is ``cursor.executemany()``, which doesn't support setinputsizes. The
change also adds appropriate pyodbc DBAPI typing to values that are typed
as :class:`_types.Unicode` or :class:`_types.UnicodeText`, as well as
altered the base :class:`_types.JSON` datatype to consider JSON string
values as :class:`_types.Unicode` rather than :class:`_types.String`.
cheremnov [Thu, 24 Feb 2022 07:22:33 +0000 (02:22 -0500)]
Comments on (named) constraints
Adds support for comments on named constraints, including `ForeignKeyConstraint`, `PrimaryKeyConstraint`, `CheckConstraint`, `UniqueConstraint`, solving the [Issue 5667](https://github.com/sqlalchemy/sqlalchemy/issues/5667).
Supports only PostgreSQL backend.
### Description
Following the example of [Issue 1546](https://github.com/sqlalchemy/sqlalchemy/issues/1546), supports comments on constraints. Specifically, enables comments on _named_ ones — as I get it, PostgreSQL prohibits comments on unnamed constraints.
Enables setting the comments for named constraints like this:
```
Table(
'example', metadata,
Column('id', Integer),
Column('data', sa.String(30)),
PrimaryKeyConstraint(
"id", name="id_pk", comment="id_pk comment"
),
CheckConstraint('id < 100', name="cc1", comment="Id value can't exceed 100"),
UniqueConstraint(['data'], name="uc1", comment="Must have unique data field"),
)
```
Provides the DDL representation for constraint comments and routines to create and drop them. Class `.Inspector` reflects constraint comments via methods like `get_check_constraints` .
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- [ ] A short code fix
- [x] A new feature implementation
- Solves the issue 5667.
- The commit message includes `Fixes: 5667`.
- Includes tests based on comment reflection.
Mike Bayer [Mon, 27 Jun 2022 16:56:27 +0000 (12:56 -0400)]
merge column args from Annotated left side
because we are forced by pep-681 to use the argument
"default", we need a way to have client Column default
separate from a dataclasses level default. Also, pep-681
does not support deriving the descriptor function from
Annotated, so allow a brief right side mapped_column() to
be present that will have more column-centric arguments
from the left side Annotated to be merged.
Federico Caselli [Sun, 26 Jun 2022 10:31:45 +0000 (12:31 +0200)]
Ensure type lengths are int in oracle
Repair change introduced by the multi reflection that caused
char length of varchar like types or precisions in numberic
like types to be set as float.
This will fix the test errors in alembic that are
currently broken, as shown in
I9ad803df1d3ccf2a5111266b781061936717b8c8
Mike Bayer [Fri, 24 Jun 2022 14:31:46 +0000 (10:31 -0400)]
add fallback for old mutable format
Fixed regression caused by :ticket:`8133` where the pickle format for
mutable attributes was changed, without a fallback to recognize the old
format, causing in-place upgrades of SQLAlchemy to no longer be able to
read pickled data from previous versions. A check plus a fallback for the
old format is now in place.
Mike Bayer [Thu, 23 Jun 2022 15:15:19 +0000 (11:15 -0400)]
refine _include_fn to not include sibling mappers
Fixed regression caused by :ticket:`8064` where a particular check for
column correspondence was made too liberal, resulting in incorrect
rendering for some ORM subqueries such as those using
:meth:`.PropComparator.has` or :meth:`.PropComparator.any` in conjunction
with joined-inheritance queries that also use legacy aliasing features.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
