From: Mike Bayer Date: Sun, 5 Feb 2023 21:42:27 +0000 (-0500) Subject: update asyncio examples and add notes about writeonly X-Git-Tag: rel_2_0_2~10 X-Git-Url: http://git.ipfire.org/?a=commitdiff_plain;h=af24660a963c84998d102fb504742d05a9c15146;p=thirdparty%2Fsqlalchemy%2Fsqlalchemy.git update asyncio examples and add notes about writeonly Change-Id: I1233eb1a860b915fb265ec8bf177f1a0471cdbd1 --- diff --git a/doc/build/orm/extensions/asyncio.rst b/doc/build/orm/extensions/asyncio.rst index 94aa609145..322c5081a2 100644 --- a/doc/build/orm/extensions/asyncio.rst +++ b/doc/build/orm/extensions/asyncio.rst @@ -175,7 +175,7 @@ illustrates a complete example including mapper and session configuration:: id: Mapped[int] = mapped_column(primary_key=True) data: Mapped[str] create_date: Mapped[datetime.datetime] = mapped_column(server_default=func.now()) - bs: Mapped[List[B]] = relationship() + bs: Mapped[List[B]] = relationship(lazy="raise") class B(Base): @@ -192,7 +192,7 @@ illustrates a complete example including mapper and session configuration:: session.add_all( [ A(bs=[B(), B()], data="a1"), - A(bs=[B()], data="a2"), + A(bs=[], data="a2"), A(bs=[B(), B()], data="a3"), ] ) @@ -266,23 +266,44 @@ Preventing Implicit IO when Using AsyncSession ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Using traditional asyncio, the application needs to avoid any points at which -IO-on-attribute access may occur. Above, the following measures are taken to -prevent this: - -* The :func:`_orm.selectinload` eager loader is employed in order to eagerly +IO-on-attribute access may occur. Techniques that can be used to help +this are below, many of which are illustrated in the preceding example. + +* Collections can be replaced with **write only collections** that will never + emit IO implicitly, by using the :ref:`write_only_relationship` feature in + SQLAlchemy 2.0. Using this feature, collections are never read from, only + queried using explicit SQL calls. See the example + ``async_orm_writeonly.py`` in the :ref:`examples_asyncio` section for + an example of write-only collections used with asyncio. + + When using write only collections, the program's behavior is simple and easy + to predict regarding collections. However, the downside is that there is not + any built-in system for loading many of these collections all at once, which + instead would need to be performed manually. Therefore, many of the + bullets below address specific techniques when using traditional lazy-loaded + relationships with asyncio, which requires more care. + +* If using traditional ORM relationships which are subject to lazy loading, + relationships can be declared with ``lazy="raise"`` so that by + default they will not attempt to emit SQL. In order to load collections, + :term:`eager loading` must be used in all cases. + +* The most useful eager loading strategy is the + :func:`_orm.selectinload` eager loader, which is employed in the previous + example in order to eagerly load the ``A.bs`` collection within the scope of the ``await session.execute()`` call:: stmt = select(A).options(selectinload(A.bs)) - .. +* When constructing new objects, **collections are always assigned a default, + empty collection**, such as a list in the above example:: - If the default loader strategy of "lazyload" were left in place, the access - of the ``A.bs`` attribute would raise an asyncio exception. - There are a variety of ORM loader options available, which may be configured - at the default mapping level or used on a per-query basis, documented at - :ref:`loading_toplevel`. + A(bs=[], data="a2") + This allows the ``.bs`` collection on the above ``A`` object to be present and + readable when the ``A`` object is flushed; otherwise, when the ``A`` is + flushed, ``.bs`` would be unloaded and would raise an error on access. * The :class:`_asyncio.AsyncSession` is configured using :paramref:`_orm.Session.expire_on_commit` set to False, so that we may access @@ -308,30 +329,13 @@ prevent this: # expire_on_commit=False allows print(a1.data) -* The :paramref:`_schema.Column.server_default` value on the ``created_at`` - column will not be refreshed by default after an INSERT; instead, it is - normally - :ref:`expired so that it can be loaded when needed `. - Similar behavior applies to a column where the - :paramref:`_schema.Column.default` parameter is assigned to a SQL expression - object. To access this value with asyncio, it has to be refreshed within the - flush process, which is achieved by setting the - :paramref:`_orm.Mapper.eager_defaults` parameter on the mapping:: - - - class A(Base): - # ... - - # column with a server_default, or SQL expression default - create_date = mapped_column(DateTime, server_default=func.now()) - - # add this so that it can be accessed - __mapper_args__ = {"eager_defaults": True} - Other guidelines include: * Methods like :meth:`_asyncio.AsyncSession.expire` should be avoided in favor of - :meth:`_asyncio.AsyncSession.refresh` + :meth:`_asyncio.AsyncSession.refresh`; **if** expiration is absolutely needed. + Expiration should generally **not** be needed as + :paramref:`_orm.Session.expire_on_commit` + should normally be set to ``False`` when using asyncio. * Avoid using the ``all`` cascade option documented at :ref:`unitofwork_cascades` in favor of listing out the desired cascade features explicitly. The @@ -368,6 +372,14 @@ Other guidelines include: :ref:`migration_20_dynamic_loaders` - notes on migration to 2.0 style +* If using asyncio with a database that does not support RETURNING, such as + MySQL 8, server default values such as generated timestamps will not be + available on newly flushed objects unless the + :paramref:`_orm.Mapper.eager_defaults` option is used. In SQLAlchemy 2.0, + this behavior is applied automatically to backends like PostgreSQL, SQLite + and MariaDB which use RETURNING to fetch new values when rows are + INSERTed. + .. _session_run_sync: Running Synchronous Methods and Functions under asyncio diff --git a/examples/asyncio/async_orm.py b/examples/asyncio/async_orm.py index 4688911588..66501e5457 100644 --- a/examples/asyncio/async_orm.py +++ b/examples/asyncio/async_orm.py @@ -1,20 +1,22 @@ -"""Illustrates use of the sqlalchemy.ext.asyncio.AsyncSession object +"""Illustrates use of the ``sqlalchemy.ext.asyncio.AsyncSession`` object for asynchronous ORM use. """ +from __future__ import annotations import asyncio +import datetime +from typing import List +from typing import Optional -from sqlalchemy import Column -from sqlalchemy import DateTime from sqlalchemy import ForeignKey from sqlalchemy import func -from sqlalchemy import Integer -from sqlalchemy import String from sqlalchemy.ext.asyncio import async_sessionmaker from sqlalchemy.ext.asyncio import create_async_engine from sqlalchemy.future import select from sqlalchemy.orm import declarative_base +from sqlalchemy.orm import Mapped +from sqlalchemy.orm import mapped_column from sqlalchemy.orm import relationship from sqlalchemy.orm import selectinload @@ -24,22 +26,19 @@ Base = declarative_base() class A(Base): __tablename__ = "a" - id = Column(Integer, primary_key=True) - data = Column(String) - create_date = Column(DateTime, server_default=func.now()) - bs = relationship("B") - - # required in order to access columns with server defaults - # or SQL expression defaults, subsequent to a flush, without - # triggering an expired load - __mapper_args__ = {"eager_defaults": True} + id: Mapped[int] = mapped_column(primary_key=True) + data: Mapped[Optional[str]] + create_date: Mapped[datetime.datetime] = mapped_column( + server_default=func.now() + ) + bs: Mapped[List[B]] = relationship(lazy="raise") class B(Base): __tablename__ = "b" - id = Column(Integer, primary_key=True) - a_id = Column(ForeignKey("a.id")) - data = Column(String) + id: Mapped[int] = mapped_column(primary_key=True) + a_id: Mapped[int] = mapped_column(ForeignKey("a.id")) + data: Mapped[Optional[str]] async def async_main(): @@ -74,10 +73,10 @@ async def async_main(): # AsyncSession.execute() is used for 2.0 style ORM execution # (same as the synchronous API). - result = await session.execute(stmt) + result = await session.scalars(stmt) # result is a buffered Result object. - for a1 in result.scalars(): + for a1 in result: print(a1) print(f"created at: {a1.create_date}") for b1 in a1.bs: @@ -92,9 +91,9 @@ async def async_main(): for b1 in a1.bs: print(b1) - result = await session.execute(select(A).order_by(A.id)) + result = await session.scalars(select(A).order_by(A.id)) - a1 = result.scalars().first() + a1 = result.first() a1.data = "new data" diff --git a/examples/asyncio/async_orm_writeonly.py b/examples/asyncio/async_orm_writeonly.py new file mode 100644 index 0000000000..cdc4865249 --- /dev/null +++ b/examples/asyncio/async_orm_writeonly.py @@ -0,0 +1,102 @@ +"""Illustrates using **write only relationships** for simpler handling +of ORM collections under asyncio. + +""" +from __future__ import annotations + +import asyncio +import datetime +from typing import Optional + +from sqlalchemy import ForeignKey +from sqlalchemy import func +from sqlalchemy.ext.asyncio import async_sessionmaker +from sqlalchemy.ext.asyncio import create_async_engine +from sqlalchemy.future import select +from sqlalchemy.orm import declarative_base +from sqlalchemy.orm import Mapped +from sqlalchemy.orm import mapped_column +from sqlalchemy.orm import relationship +from sqlalchemy.orm import WriteOnlyMapped + +Base = declarative_base() + + +class A(Base): + __tablename__ = "a" + + id: Mapped[int] = mapped_column(primary_key=True) + data: Mapped[Optional[str]] + create_date: Mapped[datetime.datetime] = mapped_column( + server_default=func.now() + ) + + # collection relationships are declared with WriteOnlyMapped. There + # is no separate collection type + bs: WriteOnlyMapped[B] = relationship() + + +class B(Base): + __tablename__ = "b" + id: Mapped[int] = mapped_column(primary_key=True) + a_id: Mapped[int] = mapped_column(ForeignKey("a.id")) + data: Mapped[Optional[str]] + + +async def async_main(): + """Main program function.""" + + engine = create_async_engine( + "postgresql+asyncpg://scott:tiger@localhost/test", + echo=True, + ) + + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.drop_all) + async with engine.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + + async_session = async_sessionmaker(engine, expire_on_commit=False) + + async with async_session() as session: + async with session.begin(): + # WriteOnlyMapped may be populated using any iterable, + # e.g. lists, sets, etc. + session.add_all( + [ + A(bs=[B(), B()], data="a1"), + A(bs=[B()], data="a2"), + A(bs=[B(), B()], data="a3"), + ] + ) + + stmt = select(A) + + result = await session.scalars(stmt) + + for a1 in result: + print(a1) + print(f"created at: {a1.create_date}") + + # to iterate a collection, emit a SELECT statement + for b1 in await session.scalars(a1.bs.select()): + print(b1) + + result = await session.stream(stmt) + + async for a1 in result.scalars(): + print(a1) + + # similar using "streaming" (server side cursors) + async for b1 in (await session.stream(a1.bs.select())).scalars(): + print(b1) + + await session.commit() + result = await session.scalars(select(A).order_by(A.id)) + + a1 = result.first() + + a1.data = "new data" + + +asyncio.run(async_main())