Document implicit IO points in ORM

I purposely didn't spend much documentation writing
about implicit IO when I first pushed out the asyncio
extension because I wanted to get a sense on what kinds
of issues people had.   Now we know and the answer is
predictably "all of them".  List out all the known
implicit IO points and how to avoid them.  Also rename
the "adapting lazy loads" section, so that the title is less
suggestive that this is a necessary technique.

References: #5926

Change-Id: I3933b74bd37a5b06989531adbeade34347db679b

diff --git a/doc/build/orm/extensions/asyncio.rst b/doc/build/orm/extensions/asyncio.rst
index 43068288d27710a1aba33cd6bdae4a654c220c14..12e1cc12a5aa72b0e8680937cdc7267d07a9cce6 100644 (file)
--- a/doc/build/orm/extensions/asyncio.rst
+++ b/doc/build/orm/extensions/asyncio.rst
@@ -85,25 +85,69 @@ Synopsis - ORM
 ---------------
 
 Using :term:`2.0 style` querying, the :class:`_asyncio.AsyncSession` class
-provides full ORM functionality.   Within the default mode of use, special care
-must be taken to avoid :term:`lazy loading` of ORM relationships and column
-attributes, as below where the :func:`_orm.selectinload` eager loading strategy
-is used to ensure the ``A.bs`` on each ``A`` object is loaded::
+provides full ORM functionality. Within the default mode of use, special care
+must be taken to avoid :term:`lazy loading` or other expired-attribute access
+involving ORM relationships and column attributes; the next
+section :ref:`asyncio_orm_avoid_lazyloads` details this.   The example below
+illustrates a complete example including mapper and session configuration::
 
     import asyncio
 
-    from sqlalchemy.ext.asyncio import create_async_engine
+    from sqlalchemy import Column
+    from sqlalchemy import DateTime
+    from sqlalchemy import ForeignKey
+    from sqlalchemy import func
+    from sqlalchemy import Integer
+    from sqlalchemy import String
     from sqlalchemy.ext.asyncio import AsyncSession
+    from sqlalchemy.ext.asyncio import create_async_engine
+    from sqlalchemy.ext.declarative import declarative_base
+    from sqlalchemy.future import select
+    from sqlalchemy.orm import relationship
+    from sqlalchemy.orm import selectinload
+    from sqlalchemy.orm import sessionmaker
+
+    Base = declarative_base()
+
+
+    class A(Base):
+        __tablename__ = "a"
+
+        id = Column(Integer, primary_key=True)
+        data = Column(String)
+        create_date = Column(DateTime, server_default=func.now())
+        bs = relationship("B")
+
+        # required in order to access columns with server defaults
+        # or SQL expression defaults, subsequent to a flush, without
+        # triggering an expired load
+        __mapper_args__ = {"eager_defaults": True}
+
+
+    class B(Base):
+        __tablename__ = "b"
+        id = Column(Integer, primary_key=True)
+        a_id = Column(ForeignKey("a.id"))
+        data = Column(String)
+
 
     async def async_main():
         engine = create_async_engine(
-            "postgresql+asyncpg://scott:tiger@localhost/test", echo=True,
+            "postgresql+asyncpg://scott:tiger@localhost/test",
+            echo=True,
         )
+
         async with engine.begin() as conn:
             await conn.run_sync(Base.metadata.drop_all)
             await conn.run_sync(Base.metadata.create_all)
 
-        async with AsyncSession(engine) as session:
+        # expire_on_commit=False will prevent attributes from being expired
+        # after commit.
+        async_session = sessionmaker(
+            engine, expire_on_commit=False, class_=AsyncSession
+        )
+
+        async with async_session() as session:
             async with session.begin():
                 session.add_all(
                     [
@@ -119,6 +163,7 @@ is used to ensure the ``A.bs`` on each ``A`` object is loaded::
 
             for a1 in result.scalars():
                 print(a1)
+                print(f"created at: {a1.create_date}")
                 for b1 in a1.bs:
                     print(b1)
 
@@ -130,31 +175,111 @@ is used to ensure the ``A.bs`` on each ``A`` object is loaded::
 
             await session.commit()
 
+            # access attribute subsequent to commit; this is what
+            # expire_on_commit=False allows
+            print(a1.data)
+
+
     asyncio.run(async_main())
 
-Above, the :func:`_orm.selectinload` eager loader is employed in order
-to eagerly load the ``A.bs`` collection within the scope of the
-``await session.execute()`` call.   If the default loader strategy of
-"lazyload" were left in place, the access of the ``A.bs`` attribute would
-raise an asyncio exception.  Using traditional asyncio, the application
-needs to avoid any points at which IO-on-attribute access may occur.
-This also includes that methods such as :meth:`_orm.Session.expire` should be
-avoided in favor of :meth:`_asyncio.AsyncSession.refresh`, and that
-appropriate loader options should be employed for :func:`_orm.deferred`
-columns as well as for :func:`_orm.relationship` constructs.
-The full list of available loaders is documented in the section
-:doc:`/orm/loading_relationships`.
-
-In the example above the :class:`_asyncio.AsyncSession` is instantiated with an
-:class:`_asyncio.AsyncEngine` associated with a particular database URL.
-It is then used in a Python asynchronous context manager (i.e. ``async with:`` statement)
-so that it is automatically closed at the end of the block; this is equivalent
-to calling the :meth:`_asyncio.AsyncSession.close` method.
+In the example above, the :class:`_asyncio.AsyncSession` is instantiated using
+the optional :class:`_orm.sessionmaker` helper, and associated with an
+:class:`_asyncio.AsyncEngine` against particular database URL. It is
+then used in a Python asynchronous context manager (i.e. ``async with:``
+statement) so that it is automatically closed at the end of the block; this is
+equivalent to calling the :meth:`_asyncio.AsyncSession.close` method.
+
+.. _asyncio_orm_avoid_lazyloads:
+
+Preventing Implicit IO when Using AsyncSession
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Using traditional asyncio, the application needs to avoid any points at which
+IO-on-attribute access may occur. Above, the following measures are taken to
+prevent this:
+
+* The :func:`_orm.selectinload` eager loader is employed in order to eagerly
+  load the ``A.bs`` collection within the scope of the
+  ``await session.execute()`` call::
+
+      stmt = select(A).options(selectinload(A.bs))
+
+  ..
+
+  If the default loader strategy of "lazyload" were left in place, the access
+  of the ``A.bs`` attribute would raise an asyncio exception.
+  There are a variety of ORM loader options available, which may be configured
+  at the default mapping level or used on a per-query basis, documented at
+  :ref:`loading_toplevel`.
+
+
+* The :class:`_asyncio.AsyncSession` is configured using
+  :paramref:`_orm.Session.expire_on_commit` set to False, so that we may access
+  attributes on an object subsequent to a call to
+  :meth:`_asyncio.AsyncSession.commit`, as in the line at the end where we
+  access an attribute::
+
+      # create AsyncSession with expire_on_commit=False
+      async_session = AsyncSession(engine, expire_on_commit=False)
+
+      # sessionmaker version
+      async_session = sessionmaker(
+          engine, expire_on_commit=False, class_=AsyncSession
+      )
+
+      async with async_session() as session:
+
+          result = await session.execute(select(A).order_by(A.id))
+
+          a1 = result.scalars().first()
+
+          # commit would normally expire all attributes
+          await session.commit()
+
+          # access attribute subsequent to commit; this is what
+          # expire_on_commit=False allows
+          print(a1.data)
+
+* The :paramref:`_schema.Column.server_default` value on the ``created_at``
+  column will not be refreshed by default after an INSERT; instead, it is
+  normally
+  :ref:`expired so that it can be loaded when needed <orm_server_defaults>`.
+  Similar behavior applies to a column where the
+  :paramref:`_schema.Column.default` parameter is assigned to a SQL expression
+  object. To access this value with asyncio, it has to be refreshed within the
+  flush process, which is achieved by setting the
+  :paramref:`_orm.mapper.eager_defaults` parameter on the mapping::
+
+
+    class A(Base):
+        # ...
+
+        # column with a server_default, or SQL expression default
+        create_date = Column(DateTime, server_default=func.now())
+
+        # add this so that it can be accessed
+        __mapper_args__ = {"eager_defaults": True}
+
+Other guidelines include:
+
+* Methods like :meth:`_asyncio.AsyncSession.expire` should be avoided in favor of
+  :meth:`_asyncio.AsyncSession.refresh`
+
+* Appropriate loader options should be employed for :func:`_orm.deferred`
+  columns, if used at all, in addition to that of :func:`_orm.relationship`
+  constructs as noted above.  See :ref:`deferred` for background on
+  deferred column loading.
+
+* The "dynamic" relationship loader strategy described at
+  :ref:`dynamic_relationship` is not compatible with the asyncio approach and
+  cannot be used, unless invoked within the
+  :meth:`_asyncio.AsyncSession.run_sync` method described at
+  :ref:`session_run_sync`.
 
 .. _session_run_sync:
 
-Adapting ORM Lazy loads to asyncio
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Running Synchronous Methods and Functions under asyncio
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 .. deepalchemy::  This approach is essentially exposing publicly the
    mechanism by which SQLAlchemy is able to provide the asyncio interface

diff --git a/examples/asyncio/async_orm.py b/examples/asyncio/async_orm.py
index 52df8bd2fc4d4a264788e8e3867a0e53def935b0..d3355791c36b622e83cc4caef41796a5aaefddd3 100644 (file)
--- a/examples/asyncio/async_orm.py
+++ b/examples/asyncio/async_orm.py
@@ -6,7 +6,9 @@ for asynchronous ORM use.
 import asyncio
 
 from sqlalchemy import Column
+from sqlalchemy import DateTime
 from sqlalchemy import ForeignKey
+from sqlalchemy import func
 from sqlalchemy import Integer
 from sqlalchemy import String
 from sqlalchemy.ext.asyncio import AsyncSession
@@ -15,6 +17,7 @@ from sqlalchemy.ext.declarative import declarative_base
 from sqlalchemy.future import select
 from sqlalchemy.orm import relationship
 from sqlalchemy.orm import selectinload
+from sqlalchemy.orm import sessionmaker
 
 Base = declarative_base()
 
@@ -24,8 +27,14 @@ class A(Base):
 
     id = Column(Integer, primary_key=True)
     data = Column(String)
+    create_date = Column(DateTime, server_default=func.now())
     bs = relationship("B")
 
+    # required in order to access columns with server defaults
+    # or SQL expression defaults, subsequent to a flush, without
+    # triggering an expired load
+    __mapper_args__ = {"eager_defaults": True}
+
 
 class B(Base):
     __tablename__ = "b"
@@ -46,7 +55,13 @@ async def async_main():
         await conn.run_sync(Base.metadata.drop_all)
         await conn.run_sync(Base.metadata.create_all)
 
-    async with AsyncSession(engine) as session:
+    # expire_on_commit=False will prevent attributes from being expired
+    # after commit.
+    async_session = sessionmaker(
+        engine, expire_on_commit=False, class_=AsyncSession
+    )
+
+    async with async_session() as session:
         async with session.begin():
             session.add_all(
                 [
@@ -66,6 +81,7 @@ async def async_main():
         # result is a buffered Result object.
         for a1 in result.scalars():
             print(a1)
+            print(f"created at: {a1.create_date}")
             for b1 in a1.bs:
                 print(b1)