docs: add first cut of pipeline mode docs

diff --git a/docs/advanced/index.rst b/docs/advanced/index.rst
index a554ae2393733a71cab446b39dd1c7663541d711..6920bd74f059d87a3b68b36ac31e632c6a7b651a 100644 (file)
--- a/docs/advanced/index.rst
+++ b/docs/advanced/index.rst
@@ -18,3 +18,4 @@ usages.
     cursors
     adapt
     prepare
+    pipeline

diff --git a/docs/advanced/pipeline.rst b/docs/advanced/pipeline.rst
new file mode 100644 (file)
index 0000000..6a50f4c
--- /dev/null
+++ b/docs/advanced/pipeline.rst
@@ -0,0 +1,73 @@
+.. currentmodule:: psycopg
+
+.. _pipeline-mode:
+
+Pipeline mode support
+=====================
+
+.. versionadded:: 3.1
+
+The *pipeline mode* allows PostgreSQL client applications to send a query
+without having to read the result of the previously sent query. Taking
+advantage of the pipeline mode, a client will wait less for the server, since
+multiple queries/results can be sent/received in a single network transaction.
+Pipeline mode can provide a significant performance boost to the application.
+
+The server executes statements, and returns results, in the order the client
+sends them. The server will begin executing the commands in the pipeline
+immediately, not waiting for the end of the pipeline. Note that results are
+buffered on the server side; the server flushes that buffer when a
+*synchronization point* is established. If any statement encounters an error,
+the server aborts the current transaction and does not execute any subsequent
+command in the queue until the next synchronization point; a
+`~errors.PipelineAborted` exception is raised for each such command (including
+a `~Connection.rollback()`). Query processing resumes after the
+synchronization point.
+
+Pipeline mode is most useful when the server is distant, i.e., network latency
+(“ping time”) is high, and also when many small operations are being performed
+in rapid succession. There is usually less benefit in using pipelined commands
+when each query takes many multiples of the client/server round-trip time to
+execute. A 100-statement operation run on a server 300 ms round-trip-time away
+would take 30 seconds in network latency alone without pipelining; with
+pipelining it may spend as little as 0.3 s waiting for results from the
+server.
+
+The pipeline mode is available on any currently supported PostgreSQL version
+but, in order to make use of it, the client must use a libpq from PostgreSQL
+14 or higher. You can use `Pipeline.is_supported()` to make sure your client
+has the right library.
+
+.. seealso:: The `PostgreSQL pipeline mode documentation`__ contains many
+    details around when it is most useful to use the pipeline mode and about
+    errors management and interaction with transactions.
+
+    .. __: https://www.postgresql.org/docs/14/libpq-pipeline-mode.html
+
+Psycopg supports the pipeline mode via the `Connection.pipeline()` method. The
+method is a context manager: at the end of the ``with`` block, the connection
+resumes the normal operation mode.
+
+Within the pipeline block, you can use one or more cursors to execute several
+operations, using `~Cursor.execute()` and `~Cursor.executemany()`. Unlike in
+normal mode, Psycopg will not wait for the server to receive the result of
+each query, which will be received in batches when a synchronization point is
+established.
+
+Psycopg can establish a synchronization points:
+
+- using the `Pipeline.sync()` method;
+- at the end of a `!Pipeline` block;
+- using a fetch method such as `Cursor.fetchone()`.
+
+The server might perform a sync on its own initiative, for instance when the
+query buffer is full.
+
+When a sync is performed, all the pending results are sent back to the cursors
+which executed them. If a cursor had run more than one query, it will receive
+more than one result; results after the first will be available, in their
+execution order, using `~Cursor.nextset()`.
+
+.. note::
+    Starting from Psycopg 3.1, `Cursor.executemany()` is optimised to make use
+    of pipeline mode.

diff --git a/docs/api/connections.rst b/docs/api/connections.rst
index 666464473db90afa6061bc0f16f35bba84cfb2cd..8679271c4f5ddf199c3c0181698ddbd5b01048cc 100644 (file)
--- a/docs/api/connections.rst
+++ b/docs/api/connections.rst
@@ -160,6 +160,25 @@ The `!Connection` class
         See :ref:`query-parameters` for all the details about executing
         queries.
 
+    .. automethod:: pipeline
+
+        The method is a context manager: you should call it using::
+
+            with conn.pipeline() as p:
+                ...
+
+        At the end of the block, a synchronization point is established and
+        the connection returns in normal mode.
+
+        You can call the method recursively from within a pipeline block.
+        Innermost blocks will establish a synchronization point on exit, but
+        pipeline mode will be kept until the outermost block exits.
+
+        See :ref:`pipeline-mode` for details.
+
+        .. versionadded:: 3.1
+
+
     .. rubric:: Transaction management methods
 
     For details see :ref:`transactions`.
@@ -266,8 +285,6 @@ The `!Connection` class
 
     .. automethod:: fileno
 
-    .. automethod:: pipeline
-
 
     .. _tpc-methods:
 
@@ -431,6 +448,16 @@ The `!AsyncConnection` class
     .. autoattribute:: row_factory
 
     .. automethod:: execute
+
+    .. automethod:: pipeline
+
+        .. note::
+
+            It must be called as::
+
+                async with conn.pipeline() as p:
+                    ...
+
     .. automethod:: commit
     .. automethod:: rollback
 
@@ -449,15 +476,6 @@ The `!AsyncConnection` class
     .. automethod:: set_read_only
     .. automethod:: set_deferrable
 
-    .. automethod:: pipeline
-
-        .. note::
-
-            It must be called as::
-
-                async with conn.pipeline():
-                    ...
-
     .. automethod:: tpc_prepare
     .. automethod:: tpc_commit
     .. automethod:: tpc_rollback

diff --git a/docs/api/cursors.rst b/docs/api/cursors.rst
index 282436c880e642a9bac97a8ed2ab4412a015db12..d7c75bb69c0ce71e4886f3207c1d9156982354b0 100644 (file)
--- a/docs/api/cursors.rst
+++ b/docs/api/cursors.rst
@@ -102,7 +102,9 @@ The `!Cursor` class
 
         .. versionchanged:: 3.1
 
-            added ``returning`` parameter to receive query results.
+            - Added ``returning`` parameter to receive query results.
+            - Preformance optimised by making use of the pipeline mode, when
+              using libpq 14 or newer.
 
     .. automethod:: copy
 

diff --git a/docs/api/errors.rst b/docs/api/errors.rst
index ca567be263418ed55da365e1d555a6c22c8364e6..cd1f3c6b3d435104231fd8f3498f79a07cc02623 100644 (file)
--- a/docs/api/errors.rst
+++ b/docs/api/errors.rst
@@ -83,6 +83,7 @@ In addition to the standard DB-API errors, Psycopg defines a few more specific
 ones.
 
 .. autoexception:: ConnectionTimeout()
+.. autoexception:: PipelineAborted()
 
 
 

diff --git a/docs/api/objects.rst b/docs/api/objects.rst
index 419606c24e690d2898db5fae2a16e6afe5d289ff..14a4b08f25cca479ecb5d5805fc13f1489192457 100644 (file)
--- a/docs/api/objects.rst
+++ b/docs/api/objects.rst
@@ -193,6 +193,26 @@ Notifications
         The PID of the backend process which sent the notification.
 
 
+Pipeline-related objects
+------------------------
+
+See :ref:`pipeline-mode` for details.
+
+.. autoclass:: Pipeline
+
+    This objects is returned by `Connection.pipeline()`.
+
+    .. automethod:: sync
+    .. automethod:: is_supported
+
+
+.. autoclass:: AsyncPipeline
+
+    This objects is returned by `AsyncConnection.pipeline()`.
+
+    .. automethod:: sync
+
+
 Transaction-related objects
 ---------------------------
 

diff --git a/docs/news.rst b/docs/news.rst
index 08ad5151fd62103f6a31190b5f64bfba42a05e04..ac33b9a3400d50dbbc4c0d2d9389e39f84795387 100644 (file)
--- a/docs/news.rst
+++ b/docs/news.rst
@@ -13,6 +13,7 @@ Future releases
 Psycopg 3.1 (unreleased)
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
+- Add :ref:`Pipeline mode <pipeline-mode>` (:ticket:`#74`).
 - Add :ref:`Two-Phase Commit <two-phase-commit>` support (:ticket:`#72`).
 - Add :ref:`adapt-enum` (:ticket:`#274`).
 - Add ``returning`` parameter to `~Cursor.executemany()` to retrieve query

diff --git a/psycopg/psycopg/_pipeline.py b/psycopg/psycopg/_pipeline.py
index f9d967c1db4e2032c53bf60365f08cb13d09b0ca..80e47acb87889371747befee739c442dc7d19fc5 100644 (file)
--- a/psycopg/psycopg/_pipeline.py
+++ b/psycopg/psycopg/_pipeline.py
@@ -66,7 +66,7 @@ class BasePipeline:
 
     @staticmethod
     def is_supported() -> bool:
-        """Return `True` if the psycopg libpq wrapper suports pipeline mode."""
+        """Return `!True` if the psycopg libpq wrapper suports pipeline mode."""
         if BasePipeline._is_supported is None:
             # Support only depends on the libpq functions available in the pq
             # wrapper, not on the database version.

diff --git a/psycopg/psycopg/connection.py b/psycopg/psycopg/connection.py
index fc92c0512f3a2cf31da2a03ea2c7031a03554995..8d01d86306f12de9cf3815c4c5981789b5444873 100644 (file)
--- a/psycopg/psycopg/connection.py
+++ b/psycopg/psycopg/connection.py
@@ -883,7 +883,7 @@ class Connection(BaseConnection[Row]):
 
     @contextmanager
     def pipeline(self) -> Iterator[Pipeline]:
-        """Context manager to switch the connection into pipeline mode."""
+        """Switch the connection into pipeline mode."""
         with self.lock:
             pipeline = self._pipeline
             if pipeline is None: