From: Mike Bayer Date: Sat, 29 Aug 2020 22:24:18 +0000 (-0400) Subject: Update connection docs for migrating off of nesting X-Git-Tag: rel_1_3_20~32 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=479e2300da86526fa1a4e87dcccd3f6e2422fbb4;p=thirdparty%2Fsqlalchemy%2Fsqlalchemy.git Update connection docs for migrating off of nesting Change-Id: I3a81140f00a4a9945121bfb8ec4c0e3953b4085f --- diff --git a/doc/build/core/connections.rst b/doc/build/core/connections.rst index 942260d6a5..74f2b13a3a 100644 --- a/doc/build/core/connections.rst +++ b/doc/build/core/connections.rst @@ -125,9 +125,12 @@ as a best practice. Nesting of Transaction Blocks ----------------------------- -.. note:: The "transaction nesting" feature of SQLAlchemy is a legacy feature - that will be deprecated in an upcoming release. New usage paradigms will - eliminate the need for it to be present. +.. deprecated:: 1.4 The "transaction nesting" feature of SQLAlchemy is a legacy feature + that will be deprecated in the 1.4 release and no longer part of the 2.0 + series of SQLAlchemy. The pattern has proven to be a little too awkward + and complicated, unless an application makes more of a first-class framework + around the behavior. See the following subsection + :ref:`connections_avoid_nesting`. The :class:`.Transaction` object also handles "nested" behavior by keeping track of the outermost begin/commit pair. In this example, two functions both @@ -162,8 +165,145 @@ which "guarantee" that a transaction will be used if one was not already available, but will automatically participate in an enclosing transaction if one exists. -.. index:: - single: thread safety; transactions +.. _connections_avoid_nesting: + +Arbitrary Transaction Nesting as an Antipattern +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +With many years of experience, the above "nesting" pattern has not proven to +be very popular, and where it has been observed in large projects such +as Openstack, it tends to be complicated. + +The most ideal way to organize an application would have a single, or at +least very few, points at which the "beginning" and "commit" of all +database transactions is demarcated. This is also the general +idea discussed in terms of the ORM at :ref:`session_faq_whentocreate`. To +adapt the example from the previous section to this practice looks like:: + + + # method_a calls method_b + def method_a(connection): + method_b(connection) + + # method_b uses the connection and assumes the transaction + # is external + def method_b(connection): + connection.execute(text("insert into mytable values ('bat', 'lala')")) + connection.execute(mytable.insert(), {"col1": "bat", "col2": "lala"}) + + # open a Connection inside of a transaction and call method_a + with engine.begin() as conn: + method_a(conn) + +That is, ``method_a()`` and ``method_b()`` do not deal with the details +of the transaction at all; the transactional scope of the connection is +defined **externally** to the functions that have a SQL dialogue with the +connection. + +It may be observed that the above code has fewer lines, and less indentation +which tends to correlate with lower :term:`cyclomatic complexity`. The +above code is organized such that ``method_a()`` and ``method_b()`` are always +invoked from a point at which a transaction is begun. The previous +version of the example features a ``method_a()`` and a ``method_b()`` that are +trying to be agnostic of this fact, which suggests they are prepared for +at least twice as many potential codepaths through them. + +.. _connections_subtransactions: + + +Migrating from the "nesting" pattern +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +As SQLAlchemy's intrinsic-nested pattern is considered legacy, an application +that for either legacy or novel reasons still seeks to have a context that +automatically frames transactions should seek to maintain this functionality +through the use of a custom Python context manager. A similar example is also +provided in terms of the ORM in the "seealso" section below. + +To provide backwards compatibility for applications that make use of this +pattern, the following context manager or a similar implementation based on +a decorator may be used:: + + import contextlib + + @contextlib.contextmanager + def transaction(connection): + if not connection.in_transaction(): + with connection.begin(): + yield connection + else: + yield connection + +The above contextmanager would be used as:: + + # method_a starts a transaction and calls method_b + def method_a(connection): + with transaction(connection): # open a transaction + method_b(connection) + + # method_b either starts a transaction, or uses the one already + # present + def method_b(connection): + with transaction(connection): # open a transaction + connection.execute(text("insert into mytable values ('bat', 'lala')")) + connection.execute(mytable.insert(), {"col1": "bat", "col2": "lala"}) + + # open a Connection and call method_a + with engine.connect() as conn: + method_a(conn) + +A similar approach may be taken such that connectivity is established +on demand as well; the below approach features a single-use context manager +that accesses an enclosing state in order to test if connectivity is already +present:: + + import contextlib + + def connectivity(engine): + connection = None + + @contextlib.contextmanager + def connect(): + nonlocal connection + + if connection is None: + connection = engine.connect() + with connection: + with connection.begin(): + yield connection + else: + yield connection + + return connect + +Using the above would look like:: + + # method_a passes along connectivity context, at the same time + # it chooses to establish a connection by calling "with" + def method_a(connectivity): + with connectivity(): + method_b(connectivity) + + # method_b also wants to use a connection from the context, so it + # also calls "with:", but also it actually uses the connection. + def method_b(connectivity): + with connectivity() as connection: + connection.execute(text("insert into mytable values ('bat', 'lala')")) + connection.execute(mytable.insert(), {"col1": "bat", "col2": "lala"}) + + # create a new connection/transaction context object and call + # method_a + method_a(connectivity(engine)) + +The above context manager acts not only as a "transaction" context but also +as a context that manages having an open connection against a particular +:class:`_engine.Engine`. When using the ORM :class:`_orm.Session`, this +connectivty management is provided by the :class:`_orm.Session` itself. +An overview of ORM connectivity patterns is at :ref:`unitofwork_transaction`. + +.. seealso:: + + :ref:`session_subtransactions_migrating` - ORM version .. _autocommit: diff --git a/doc/build/orm/session_transaction.rst b/doc/build/orm/session_transaction.rst index 505995c32f..9afbe134a4 100644 --- a/doc/build/orm/session_transaction.rst +++ b/doc/build/orm/session_transaction.rst @@ -28,19 +28,22 @@ which is acquired via the :meth:`_engine.Engine.contextual_connect` method. If for an example of this), it is added to the transactional state directly. -For each :class:`_engine.Connection`, the :class:`.Session` also maintains a :class:`.Transaction` object, -which is acquired by calling :meth:`_engine.Connection.begin` on each :class:`_engine.Connection`, -or if the :class:`.Session` -object has been established using the flag ``twophase=True``, a :class:`.TwoPhaseTransaction` -object acquired via :meth:`_engine.Connection.begin_twophase`. These transactions are all committed or -rolled back corresponding to the invocation of the -:meth:`.Session.commit` and :meth:`.Session.rollback` methods. A commit operation will -also call the :meth:`.TwoPhaseTransaction.prepare` method on all transactions if applicable. - -When the transactional state is completed after a rollback or commit, the :class:`.Session` -:term:`releases` all :class:`.Transaction` and :class:`_engine.Connection` resources, -and goes back to the "begin" state, which -will again invoke new :class:`_engine.Connection` and :class:`.Transaction` objects as new +For each :class:`_engine.Connection`, the :class:`.Session` also maintains a +:class:`.Transaction` object, which is acquired by calling +:meth:`_engine.Connection.begin` on each :class:`_engine.Connection`, or if the +:class:`.Session` object has been established using the flag ``twophase=True``, +a :class:`.TwoPhaseTransaction` object acquired via +:meth:`_engine.Connection.begin_twophase`. These transactions are all +committed or rolled back corresponding to the invocation of the +:meth:`.Session.commit` and :meth:`.Session.rollback` methods. A commit +operation will also call the :meth:`.TwoPhaseTransaction.prepare` method on +all transactions if applicable. + +When the transactional state is completed after a rollback or commit, the +:class:`.Session` +:term:`releases` all :class:`.Transaction` and :class:`_engine.Connection` +resources, and goes back to the "begin" state, which will again invoke new +:class:`_engine.Connection` and :class:`.Transaction` objects as new requests to emit SQL statements are received. The example below illustrates this lifecycle:: @@ -143,6 +146,17 @@ things like unique constraint exceptions:: Autocommit Mode --------------- +.. deprecated:: 1.4 + + "autocommit" mode is a **legacy mode of use** and should not be considered + for new projects. The feature will be deprecated in SQLAlchemy 1.4 and + removed in version 2.0; both versions provide a more refined + "autobegin" approach that allows the :meth:`.Session.begin` method + to be used normally. If autocommit mode is used, it is strongly + advised that the application at least ensure that transaction scope is made + present via the :meth:`.Session.begin` method, rather than using the + session in pure autocommit mode. + The examples of session lifecycle at :ref:`unitofwork_transaction` refer to a :class:`.Session` that runs in its default mode of ``autocommit=False``. In this mode, the :class:`.Session` begins new transactions automatically @@ -161,22 +175,6 @@ returned by :meth:`.Session.query`. For a flush operation, the :class:`.Session starts a new transaction for the duration of the flush, and commits it when complete. -.. warning:: - - "autocommit" mode is a **legacy mode of use** and should not be - considered for new projects. If autocommit mode is used, it is strongly - advised that the application at least ensure that transaction scope - is made present via the :meth:`.Session.begin` method, rather than - using the session in pure autocommit mode. An upcoming release of - SQLAlchemy will include a new mode of usage that provides this pattern - as a first class feature. - - If the :meth:`.Session.begin` method is not used, and operations are allowed - to proceed using ad-hoc connections with immediate autocommit, then the - application probably should set ``autoflush=False, expire_on_commit=False``, - since these features are intended to be used only within the context - of a database transaction. - Modern usage of "autocommit mode" tends to be for framework integrations that wish to control specifically when the "begin" state occurs. A session which is configured with ``autocommit=True`` may be placed into the "begin" state using @@ -214,18 +212,26 @@ compatible with the ``with`` statement:: Using Subtransactions with Autocommit ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -A subtransaction indicates usage of the :meth:`.Session.begin` method in conjunction with -the ``subtransactions=True`` flag. This produces a non-transactional, delimiting construct that -allows nesting of calls to :meth:`~.Session.begin` and :meth:`~.Session.commit`. -Its purpose is to allow the construction of code that can function within a transaction -both independently of any external code that starts a transaction, -as well as within a block that has already demarcated a transaction. +.. deprecated:: 1.4 The :paramref:`.Session.begin.subtransactions` + flag will be deprecated in SQLAlchemy 1.4 and removed in SQLAlchemy 2.0. + For background on migrating away from the "subtransactions" pattern + see the next section :ref:`session_subtransactions_migrating`. + +A subtransaction indicates usage of the :meth:`.Session.begin` method in +conjunction with the :paramref:`.Session.begin.subtransactions` flag set to +``True``. This produces a +non-transactional, delimiting construct that allows nesting of calls to +:meth:`~.Session.begin` and :meth:`~.Session.commit`. Its purpose is to allow +the construction of code that can function within a transaction both +independently of any external code that starts a transaction, as well as within +a block that has already demarcated a transaction. ``subtransactions=True`` is generally only useful in conjunction with -autocommit, and is equivalent to the pattern described at :ref:`connections_nested_transactions`, -where any number of functions can call :meth:`_engine.Connection.begin` and :meth:`.Transaction.commit` -as though they are the initiator of the transaction, but in fact may be participating -in an already ongoing transaction:: +autocommit, and is equivalent to the pattern described at +:ref:`connections_nested_transactions`, where any number of functions can call +:meth:`_engine.Connection.begin` and :meth:`.Transaction.commit` as though they +are the initiator of the transaction, but in fact may be participating in an +already ongoing transaction:: # method_a starts a transaction and calls method_b def method_a(session): @@ -255,11 +261,98 @@ in an already ongoing transaction:: method_a(session) session.close() -Subtransactions are used by the :meth:`.Session.flush` process to ensure that the -flush operation takes place within a transaction, regardless of autocommit. When -autocommit is disabled, it is still useful in that it forces the :class:`.Session` -into a "pending rollback" state, as a failed flush cannot be resumed in mid-operation, -where the end user still maintains the "scope" of the transaction overall. +Subtransactions are used by the :meth:`.Session.flush` process to ensure that +the flush operation takes place within a transaction, regardless of autocommit. +When autocommit is disabled, it is still useful in that it forces the +:class:`.Session` into a "pending rollback" state, as a failed flush cannot be +resumed in mid-operation, where the end user still maintains the "scope" of the +transaction overall. + +.. _session_subtransactions_migrating: + +Migrating from the "subtransaction" pattern +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The "subtransaction" pattern will be deprecated in SQLAlchemy 1.4 and removed +in version 2.0 as a public API. This pattern has been shown to be confusing in +real world applications, and it is preferable for an application to ensure that +the top-most level of database operations are performed with a single +begin/commit pair. + +To provide backwards compatibility for applications that make use of this +pattern, the following context manager or a similar implementation based on +a decorator may be used. It relies on autocommit mode within SQLAlchemy +1.3 but not in SQLAlchemy 1.4:: + + + import contextlib + + @contextlib.contextmanager + def transaction(session): + assert session.autocommit, ( + "this pattern expects the session to be in autocommit mode. " + "This assertion can be removed for SQLAlchemy 1.4." + ) + if not session.transaction: + with session.begin(): + yield + else: + yield + + +The above context manager may be used in the same way the +"subtransaction" flag works, such as in the following example:: + + + # method_a starts a transaction and calls method_b + def method_a(session): + with transaction(session): + method_b(session) + + # method_b also starts a transaction, but when + # called from method_a participates in the ongoing + # transaction. + def method_b(session): + with transaction(session): + session.add(SomeObject('bat', 'lala')) + + Session = sessionmaker(engine, autocommit=True) + + # create a Session and call method_a + session = Session() + try: + method_a(session) + finally: + session.close() + +To compare towards the preferred idiomatic pattern, the begin block should +be at the outermost level. This removes the need for individual functions +or methods to be concerned with the details of transaction demarcation:: + + def method_a(session): + method_b(session) + + def method_b(session): + session.add(SomeObject('bat', 'lala')) + + Session = sessionmaker(engine) + + # create a Session and call method_a + session = Session() + try: + # Session "begins" the transaction automatically, so the + # .transaction attribute may be used as a context manager. + with session.transaction: + method_a(session) + finally: + session.close() + +SQLAlchemy 1.4 will feature an improved API for the above transactional +patterns. + +.. seealso:: + + :ref:`connections_subtransactions` - similar pattern based on Core only .. _session_twophase: