Andrew Brookins [Sun, 22 May 2022 20:24:15 +0000 (16:24 -0400)]
Add a note on using server_onupdate=FetchedValue()
Add a note on using `server_onupdate=FetchedValue()` when using SQL expressions with `onupdate`.
My team encountered an issue with using a SQL expression with `onupdate`.
Despite the dialect (PG) supporting `RETURNING`, we needed to mark the column with
`server_onupdate=FetchedValue()` in order to get the column used with `onupdate`
to appear in the `RETURNING` clause of `UPDATE` statements.
This was not clear from the documentation, so I want to make it crystal clear for other
folks defining similar columns.
Mike Bayer [Wed, 18 May 2022 20:06:29 +0000 (16:06 -0400)]
use plainto_tsquery for PG match
The :meth:`.Operators.match` operator now uses ``plainto_tsquery()`` for
PostgreSQL full text search, rather than ``to_tsquery()``. The rationale
for this change is to provide better cross-compatibility with match on
other database backends. Full support for all PostgreSQL full text
functions remains available through the use of :data:`.func` in
conjunction with :meth:`.Operators.bool_op` (an improved version of
:meth:`.Operators.op` for boolean operators).
Additional doc updates here apply to 1.4 so will backport these
out to a separate commit.
Mike Bayer [Sun, 22 May 2022 14:32:02 +0000 (10:32 -0400)]
disable psycopg-c from dbapimain builds
I don't know what this means.
ERROR:
Command errored out with exit status 1:
command: /home/classic/tmp/foo/.venv/bin/python /home/classic/tmp/foo/.venv/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmp1x3hi09h
cwd: /tmp/pip-install-yo6y28dm/psycopg-c_18eb2a0429dc438493454ed1051296ad
Complete output (14 lines):
error: Multiple top-level packages discovered in a flat-layout: ['psycopg', 'psycopg_c', 'psycopg_pool'].
To avoid accidental inclusion of unwanted files or directories,
setuptools will not proceed with this build.
If you are trying to create a single distribution with multiple packages
on purpose, you should not rely on automatic discovery.
Instead, consider the following options:
1. set up custom discovery (`find` directive with `include` or `exclude`)
2. use a `src-layout`
3. explicitly set `py_modules` or `packages` with a list of names
To find more information, look for "package discovery" on setuptools docs.
Mike Bayer [Fri, 18 Feb 2022 15:05:12 +0000 (10:05 -0500)]
implement dataclass_transforms
Implement a new means of creating a mapped dataclass where
instead of applying the `@dataclass` decorator distinctly,
the declarative process itself can create the dataclass.
MapperProperty and MappedColumn objects themselves take
the place of the dataclasses.Field object when constructing
the class.
The overall approach is made possible at the typing level
using pep-681 dataclass transforms [1].
This new approach should be able to completely supersede the
previous "dataclasses" approach of embedding metadata into
Field() objects, which remains a mutually exclusive declarative
setup style (mixing them introduces new issues that are not worth
solving).
valievkarim [Wed, 18 May 2022 20:24:41 +0000 (16:24 -0400)]
Include new MySQL error code 4031 for MySQL disconnect check
Added disconnect code for MySQL error 4031, introduced in MySQL >= 8.0.24,
indicating connection idle timeout exceeded. In particular this repairs an
issue where pre-ping could not reconnect on a timed-out connection. Pull
request courtesy valievkarim.
Mike Bayer [Mon, 16 May 2022 14:57:51 +0000 (10:57 -0400)]
fix most sphinx warnings (1.4)
still can't figure out the warnings with some of the older
changelog files.
this cherry-picks the sphinx fixes from 1.4 and additionally
fixes a small number of new issues in the 2.0 docs. However,
2.0 has many more errors to fix, primarily from the removal
of the legacy tutorials left behind a lot of labels that need
to be re-linked to the new tutorial.
Mike Bayer [Fri, 13 May 2022 20:08:34 +0000 (16:08 -0400)]
render col name in on conflict set clause, not given key
Fixed bug where the PostgreSQL :meth:`_postgresql.Insert.on_conflict`
method and the SQLite :meth:`_sqlite.Insert.on_conflict` method would both
fail to correctly accommodate a column with a separate ".key" when
specifying the column using its key name in the dictionary passed to
``set_``, as well as if the :attr:`_sqlite.Insert.excluded` or
:attr:`_postgresql.Insert.excluded` collection were used as the dictionary
directly.
Mike Bayer [Fri, 13 May 2022 19:43:53 +0000 (15:43 -0400)]
raise for same param name in expanding + non expanding
An informative error is raised if two individual :class:`.BindParameter`
objects share the same name, yet one is used within an "expanding" context
(typically an IN expression) and the other is not; mixing the same name in
these two different styles of usage is not supported and typically the
``expanding=True`` parameter should be set on the parameters that are to
receive list values outside of IN expressions (where ``expanding`` is set
by default).
Mike Bayer [Sat, 14 May 2022 14:25:53 +0000 (10:25 -0400)]
adjust log stacklevel for py3.11.0b1; enable greenlet
Fixed issue where support for logging "stacklevel" implemented in
:ticket:`7612` required adjustment to work with recently released Python
3.11.0b1, also repairs the unit tests which tested this feature.
Install greenlet from a py311 compat patch.
re: the stacklevel thing, this is going to be very inconvenient
if we have to keep hardcoding numbers everywhere for every
new python version
Mike Bayer [Mon, 9 May 2022 14:27:51 +0000 (10:27 -0400)]
dont use the label convention for memoized entities
Fixed issue where ORM results would apply incorrect key names to the
returned :class:`.Row` objects in the case where the set of columns to be
selected were changed, such as when using
:meth:`.Select.with_only_columns`.
Mike Bayer [Mon, 9 May 2022 02:49:33 +0000 (22:49 -0400)]
explicitly fetch inserted pk for values(pkcol=None)
Altered the compilation mechanics of the :class:`.Insert` construct such
that the "autoincrement primary key" column value will be fetched via
``cursor.lastrowid`` or RETURNING even if present in the parameter set or
within the :meth:`.Insert.values` method as a plain bound value, for
single-row INSERT statements on specific backends that are known to
generate autoincrementing values even when explicit NULL is passed. This
restores a behavior that was in the 1.3 series for both the use case of
separate parameter set as well as :meth:`.Insert.values`. In 1.4, the
parameter set behavior unintentionally changed to no longer do this, but
the :meth:`.Insert.values` method would still fetch autoincrement values up
until 1.4.21 where :ticket:`6770` changed the behavior yet again again
unintentionally as this use case was never covered.
The behavior is now defined as "working" to suit the case where databases
such as SQLite, MySQL and MariaDB will ignore an explicit NULL primary key
value and nonetheless invoke an autoincrement generator.
khashashin [Tue, 3 May 2022 20:00:59 +0000 (22:00 +0200)]
docs(types) Fix missing import from sqlalchemy (#7978)
* docs(types) Fix missing import from sqlalchemy
The sample code is missing the import of Enum from sqlalchemy, which might confuse the reader, since we are using another enum type from Python itself here. So it makes sense to clarify that here.
Mike Bayer [Tue, 3 May 2022 12:58:27 +0000 (08:58 -0400)]
bypass pyodbc default server version / set charset
Further adjustments to the MySQL PyODBC dialect to allow for complete
connectivity, which was previously still not working despite fixes in
:ticket:`7871`.
Mike Bayer [Sun, 1 May 2022 16:28:36 +0000 (12:28 -0400)]
use bindparam_type in BinaryElementImpl._post_coercion if available
Fixed an issue where using :func:`.bindparam` with no explicit data or type
given could be coerced into the incorrect type when used in expressions
such as when using :meth:`.ARRAY.comparator.any` and
:meth:`.ARRAY.comparator.all`.
Mike Bayer [Wed, 27 Apr 2022 19:43:02 +0000 (15:43 -0400)]
inline mypy config; files ignoring type errors for the moment
to simplify pyproject.toml change the remaining files
that aren't going to be typed on this first pass
(unless of course someone wants to type some of these)
to include # mypy: ignore-errors. for the moment, only a handful
of ORM modules are to have more type checking implemented.
It's important that ignore-errors is used and
not "# type: ignore", as in the latter case, mypy doesn't even
read the existing types in the file, which makes it impossible to
type any files that refer to those modules at all.
to simplify ongoing typing work use inline mypy config
for remaining files that are "done" for now, indicating the
level of type checking they currently have.
Mike Bayer [Wed, 20 Apr 2022 01:06:41 +0000 (21:06 -0400)]
pep484 ORM / SQL result support
after some experimentation it seems mypy is more amenable
to the generic types being fully integrated rather than
having separate spin-off types. so key structures
like Result, Row, Select become generic. For DML
Insert, Update, Delete, these are spun into type-specific
subclasses ReturningInsert, ReturningUpdate, ReturningDelete,
which is fine since the "row-ness" of these constructs
doesn't happen until returning() is called in any case.
a Tuple based model is then integrated so that these
objects can carry along information about their return
types. Overloads at the .execute() level carry through
the Tuple from the invoked object to the result.
To suit the issue of AliasedClass generating attributes
that are dynamic, experimented with a custom subclass
AsAliased, but then just settled on having aliased()
lie to the type checker and return `Type[_O]`, essentially.
will need some type-related accessors for with_polymorphic()
also.
Additionally, identified an issue in Update when used
"mysql style" against a join(), it basically doesn't work
if asked to UPDATE two tables on the same column name.
added an error message to the specific condition where
it happens with a very non-specific error message that we
hit a thing we can't do right now, suggest multi-table
update as a possible cause.
Stephen Finucane [Wed, 27 Apr 2022 12:56:17 +0000 (08:56 -0400)]
Add git-blame-ignore-revs file
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
This is helpful when using git-blame on files that 'black' heavily modified back in 2019'ish. While git doesn't specify this name, there seems to be a general convention around using this. See Django [1] and Twisted [2], for example.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
- [x] None of the above
- This is infra-related
Mike Bayer [Tue, 26 Apr 2022 19:32:30 +0000 (15:32 -0400)]
changelog edits for 1.4.36
for 2.0, we split 7953 into two separate changelog files,
one with the 2.0 fix to the logic itself, and the other under
1.4 indicating the warning being emitted.
we also add into 1.4 the file for the C extension change
made for 7875 so it renders into the 1.4 documentation.
Mike Bayer [Tue, 26 Apr 2022 19:02:37 +0000 (15:02 -0400)]
repair fetch_setting call in mysql pyodbc dialect
Fixed a regression in the untested MySQL PyODBC dialect caused by the fix
for :ticket:`7518` in version 1.4.32 where an argument was being propagated
incorrectly upon first connect, leading to a ``TypeError``.
Mike Bayer [Sun, 24 Apr 2022 20:19:16 +0000 (16:19 -0400)]
forwards port test cases and changelog for #7958 to 2.0
in 6f02d5edd88fe2475629438b0730181a2b00c5fe some cleanup
to ForeignKey repaired the use case of ForeignKey objects
referring to table name alone, by adding more robust
column resolution logic. This change also fixes an issue
where the "referred column" naming convention key uses the
resolved referred column earlier than usual when a
ForeignKey is setting up its constraint.
change message for 1.4:
Fixed bug where :class:`.ForeignKeyConstraint` naming conventions using the
``referred_column_0`` naming convention key would not work if the foreign
key constraint were set up as a :class:`.ForeignKey` object rather than an
explicit :class:`.ForeignKeyConstraint` object. As this change makes use of
a backport of some fixes from version 2.0, an additional little-known
feature that has likely been broken for many years is also fixed which is
that a :class:`.ForeignKey` object may refer to a referred table by name of
the table alone without using a column name, if the name of the referent
column is the same as that of the referred column.
The ``referred_column_0`` naming convention key was not previously not
tested with the :class:`.ForeignKey` object, only
:class:`.ForeignKeyConstraint`, and this bug reveals that the feature has
never worked correctly unless :class:`.ForeignKeyConstraint` is used for
all FK constraints. This bug traces back to the original introduction of
the feature introduced for :ticket:`3989`.
Mike Bayer [Fri, 22 Apr 2022 14:57:00 +0000 (10:57 -0400)]
properly type array element in any() / all()
Fixed bug in :class:`.ARRAY` datatype in combination with :class:`.Enum` on
PostgreSQL where using the ``.any()`` method to render SQL ANY(), given
members of the Python enumeration as arguments, would produce a type
adaptation failure on all drivers.
Mike Bayer [Thu, 21 Apr 2022 17:27:16 +0000 (13:27 -0400)]
fix result.columns() method
Fixed issue in :meth:`.Result.columns` method where calling upon
:meth:`.Result.columns` with a single index could in some cases,
particularly ORM result object cases, cause the :class:`.Result` to yield
scalar objects rather than :class:`.Row` objects, as though the
:meth:`.Result.scalars` method had been called. In SQLAlchemy 1.4, this
scenario emits a warning that the behavior will change in SQLAlchemy 2.0.
Alex Grönholm [Mon, 18 Apr 2022 17:07:19 +0000 (13:07 -0400)]
Implement UUID.python_type
Implemented :attr:`_postgresql.UUID.python_type` attribute for the
:class:`_postgresql.UUID` type object. The attribute will return either
``str`` or ``uuid.UUID`` based on the :paramref:`_postgresql.UUID.as_uuid`
parameter setting. Previously, this attribute was unimplemented. Pull
request courtesy Alex Grönholm.
Mike Bayer [Fri, 15 Apr 2022 15:05:36 +0000 (11:05 -0400)]
pep-484: ORM public API, constructors
for the moment, abandoning using @overload with
relationship() and mapped_column(). The overloads
are very difficult to get working at all, and
the overloads that were there all wouldn't pass on
mypy. various techniques of getting them to
"work", meaning having right hand side dictate
what's legal on the left, have mixed success
and wont give consistent results; additionally,
it's legal to have Optional / non-optional
independent of nullable in any case for columns.
relationship cases are less ambiguous but mypy
was not going along with things.
we have a comprehensive system of allowing
left side annotations to drive the right side,
in the absense of explicit settings on the right.
so type-centric SQLAlchemy will be left-side
driven just like dataclasses, and the various flags
and switches on the right side will just not be
needed very much.
in other matters, one surprise, forgot to remove string support
from orm.join(A, B, "somename") or do deprecations
for it in 1.4. This is a really not-directly-used
structure barely
mentioned in the docs for many years, the example
shows a relationship being used, not a string, so
we will just change it to raise the usual error here.
Mike Bayer [Wed, 13 Apr 2022 13:45:29 +0000 (09:45 -0400)]
pep484: schema API
implement strict typing for schema.py
this module has lots of public API, lots of old decisions
and very hard to follow construction sequences in many
cases, and is also where we get a lot of new feature requests,
so strict typing should help keep things clean.
among improvements here, fixed the pool .info getters
and also figured out how to get ColumnCollection and
related to be covariant so that we may set them up
as returning Column or ColumnClause without any conflicts.
DDL was affected, noting that superclasses of DDLElement
(_DDLCompiles, added recently) can now be passed into
"ddl_if" callables; reorganized ddl into ExecutableDDLElement
as a new name for DDLElement and _DDLCompiles renamed to
BaseDDLElement.
setting up strict also located an API use case that
is completely broken, which is connection.execute(some_default)
returns a scalar value. This case has been deprecated
and new paths have been set up so that connection.scalar()
may be used. This likely wasn't possible in previous
versions because scalar() would assume a CursorResult.
The scalar() change also impacts Session as we have explicit
support (since someone had reported it as a regression)
for session.execute(Sequence()) to work. They will get the
same deprecation message (which omits the word "Connection",
just uses ".execute()" and ".scalar()") and they can then
use Session.scalar() as well. Getting this to type
correctly while still supporting ORM use cases required
some refactoring, and I also set up a keyword only delimeter
for Session.execute() and related as execution_options /
bind_arguments should always be keyword only, applied these
changes to AsyncSession as well.
Additionally simpify Table __init__ now that we are Python
3 only, we can have positional plus explicit kwargs finally.
Simplify Column.__init__ as well again taking advantage
of kw only arguments.
Fill in most/all __init__ methods in sqltypes.py as
the constructor for types is most of the API. should
likely do this for dialect-specific types as well.
Apply _InfoType for all info attributes as should have been
done originally and update descriptor decorators.
Mike Bayer [Thu, 14 Apr 2022 16:01:16 +0000 (12:01 -0400)]
Ensure ORMInsert sets up bind state
Fixed regression where the change in #7861, released in version 1.4.33,
that brought the :class:`.Insert` construct to be partially recognized as
an ORM-enabled statement did not properly transfer the correct mapper /
mapped table state to the :class:`.Session`, causing the
:meth:`.Session.get_bind` method to fail for a :class:`.Session` that was
bound to engines and/or connections using the :paramref:`.Session.binds`
parameter.
Mike Bayer [Thu, 14 Apr 2022 13:59:11 +0000 (09:59 -0400)]
update Numeric/Float docstrings
These docs were very out of date re: cdecimal. Additionally,
as pointed out in #5252, the Numeric documentation is misleading;
SQLAlchemy's Numeric hierarchy resembles more of the Oracle
approach where precision and scale solely determine the kind of
datatype being worked with. Float is essentially Numeric with
different defaults.
Mike Bayer [Tue, 12 Apr 2022 17:52:31 +0000 (13:52 -0400)]
implement multi-element expression constructs
Improved the construction of SQL binary expressions to allow for very long
expressions against the same associative operator without special steps
needed in order to avoid high memory use and excess recursion depth. A
particular binary operation ``A op B`` can now be joined against another
element ``op C`` and the resulting structure will be "flattened" so that
the representation as well as SQL compilation does not require recursion.
To implement this more cleanly, the biggest change here is that
column-oriented lists of things are broken away from ClauseList
in a new class ExpressionClauseList, that also forms the basis
of BooleanClauseList. ClauseList is still used for the generic
"comma-separated list" of things such as Tuple and things like
ORDER BY, as well as in some API endpoints.
Also adds __slots__ to the TypeEngine-bound Comparator
classes. Still can't really do __slots__ on ClauseElement.
Mike Bayer [Mon, 4 Apr 2022 23:01:54 +0000 (19:01 -0400)]
read from cls.__dict__ so init_subclass works
Modified the :class:`.DeclarativeMeta` metaclass to pass ``cls.__dict__``
into the declarative scanning process to look for attributes, rather than
the separate dictionary passed to the type's ``__init__()`` method. This
allows user-defined base classes that add attributes within an
``__init_subclass__()`` to work as expected, as ``__init_subclass__()`` can
only affect the ``cls.__dict__`` itself and not the other dictionary. This
is technically a regression from 1.3 where ``__dict__`` was being used.
Additionally makes the reference between ClassManager and
the declarative configuration object a weak reference, so that it
can be discarded after mappers are set up.
Mike Bayer [Tue, 12 Apr 2022 13:41:55 +0000 (09:41 -0400)]
Don't pass vistor to immutables in cloned traverse
Saw someone using cloned_traverse to move columns around
(changing their .table) and not surprisingly having poor results.
As cloned traversal is to provide a hook for in-place mutation
of elements, it should not be given Immutable objects as these
should not be changed once they are structurally composed.
Mike Bayer [Sun, 10 Apr 2022 19:42:35 +0000 (15:42 -0400)]
pep-484: asyncio
in this patch the asyncio/events.py module, which
existed only to raise errors when trying to attach event
listeners, is removed, as we were already coding an asyncio-specific
workaround in upstream Pool / Session to raise this error,
just moved the error out to the target and did the same thing
for Engine.
We also add an async_sessionmaker class. The initial rationale
here is because sessionmaker() is hardcoded to Session subclasses,
and there's not a way to get the use case of
sessionmaker(class_=AsyncSession) to type correctly without changing
the sessionmaker() symbol itself to be a function and not a class,
which gets too complicated for what this is. Additionally,
_SessionClassMethods has only three methods on it, one of which
is not usable with asyncio (close_all()), the others
not generally used from the session class.
Mike Bayer [Tue, 5 Apr 2022 23:00:19 +0000 (19:00 -0400)]
use code generation for scoped_session
our decorator thing generates code in any case,
so point it at the file itself to generate real code
for the blocks rather than doing things dynamically.
this will allow typing tools to have no problem
whatsoever and we also reduce import time overhead.
file size will be a lot bigger though, shrugs.
syntax / dupe method / etc. checking will be accomplished
by our existing linting / typing / formatting tools.
As we are also using "from __future__ import annotations",
we also no longer have to apply quotes to generated
annotations.
Mike Bayer [Mon, 11 Apr 2022 14:54:42 +0000 (10:54 -0400)]
level up pep-484 on engine result
Have each result subclass be generic to the
kind of row/object it returns. rework things so that
a significant number of "type ignores" can go away
and also allow asyncio to more cleanly proxy
the result objects.
Mike Bayer [Tue, 12 Apr 2022 02:03:28 +0000 (22:03 -0400)]
repair ancient and incorrect comment
it referred towards _columntoproperty refering to
lists of MapperProperty. this comment goes all the
way to pre 0.1 being released. it's likely been
wrong for nearly all that time.
Mike Bayer [Sat, 9 Apr 2022 15:32:50 +0000 (11:32 -0400)]
simplify coverage config
I *think* we don't need to tell .coverage to look in lib,
it seems to be covering from whereever successfuly. not
sure, but want to run on jenkins to see how it comes out.
Mike Bayer [Sat, 9 Apr 2022 13:50:18 +0000 (09:50 -0400)]
add sane_rowcount to SimpleUpdateDeleteTest
For third party dialects, repaired a missing requirement for the
``SimpleUpdateDeleteTest`` suite test which was not checking for a working
"rowcount" function on the target dialect.
Mike Bayer [Mon, 28 Mar 2022 17:29:23 +0000 (13:29 -0400)]
implement iso date literals for all backends
Added modified ISO-8601 rendering (i.e. ISO-8601 with the T converted to a
space) when using ``literal_binds`` with the SQL compilers provided by the
PostgreSQL, MySQL, MariaDB, MSSQL, Oracle dialects. For Oracle, the ISO
format is wrapped inside of an appropriate TO_DATE() function call.
Previously this rendering was not implemented for dialect-specific
compilation.
Mike Bayer [Fri, 8 Apr 2022 13:28:47 +0000 (09:28 -0400)]
explicitly refer to Apple M1 for greenlet issue
As developers are now buying lots of Apple M1
machines, and AFAWK greenlet is still not able
to provide a pre-built wheel, we are going to get a lot
of devs trying to use asyncio on their Apple M1s, in
greater proportions compared to devs running containers
etc. on other various less popular CPU architectures.
Add a new FAQ section for installation, add new
red dragon to the very top of asyncio docs,
add new verbiage, all of which includes
"Apple M1" in bold text, to minimize the chance of anyone
missing this.
projects / thirdparty / sqlalchemy / sqlalchemy.git / log
