From: Daniele Varrazzo Date: Thu, 6 Jan 2022 02:39:42 +0000 (+0100) Subject: Add null-pools documentation X-Git-Tag: pool-3.1~21^2~2 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=56fff0b1355ce7182a791deef8a43faf44bdcbea;p=thirdparty%2Fpsycopg.git Add null-pools documentation --- diff --git a/docs/advanced/pool.rst b/docs/advanced/pool.rst index 73bc267ca..f8569574e 100644 --- a/docs/advanced/pool.rst +++ b/docs/advanced/pool.rst @@ -188,6 +188,52 @@ to tune the configuration parameters. The size of the pool can also be changed at runtime using the `~ConnectionPool.resize()` method. +.. _null-pool: + +Null connection pools +--------------------- + +.. versionadded:: 3.1 + +Sometimes you may want leave the choice of using or not using a connection +pool as a configuration parameter of your application. For instance, you might +want to use a pool if you are deploying a "large instance" of your application +and can dedicate it a handful of connections; conversely you might not want to +use it if you deploy the application in several instances, behind a load +balancer, and/or using an external connection pool process such as PgBouncer. + +Switching between using or not using a pool requires some code change, because +the `ConnectionPool` API is different from the normal `~psycopg.connect()` +function and because the pool can perform additional connection configuration +(in the *configure* parameter) that, if the pool is removed, should be +performed in some different code path of your application. + +The `!psycopg_pool` 3.1 package introduces the `NullConnectionPool` class. +This class has the same interface, and largely the same behaviour, of the +`!ConnectionPool`, but doesn't create any connection beforehand. When a +connection is returned, unless there are other clients already waiting, it +is closed immediately and not kept in the pool state. + +A null pool is not only a configuration convenience, but can also be used to +regulate the access to the server by a client program. If *max_size* is set to +a value greater than 0, the pool will make sure that no more than *max_size* +connections are created at any given time. If more clients ask for further +connections, they will be queued and served a connection as soon as a previous +client has finished using it, like for the basic pool. Other mechanisms to +throttle client requests (such as *timeout* or *max_waiting*) are respected +too. + +.. note:: + + Queued clients will be handed an already established connection, as soon + as a previous client has finished using it (and after the pool has + returned it to idle state and called *reset()* on it, if necessary). + +Because normally (i.e. unless queued) every client will be served a new +connection, the time to obtain the connection is paid by the waiting client; +background workers are not normally involved in obtaining new connections. + + Connection quality ------------------ diff --git a/docs/api/pool.rst b/docs/api/pool.rst index 7c446b00b..c6234cf00 100644 --- a/docs/api/pool.rst +++ b/docs/api/pool.rst @@ -12,11 +12,22 @@ working connection and allowing an arbitrary large number of concurrent threads or tasks to use a controlled amount of resources on the server. See :ref:`connection-pools` for more details and usage pattern. -This module implement two connection pools: `ConnectionPool` is a -synchronous connection pool yielding `~psycopg.Connection` objects and can be -used by multithread applications. `AsyncConnectionPool` has a similar -interface, but with `asyncio` functions replacing blocking functions, and -yields `~psycopg.AsyncConnection` instances. +This package exposes a few connection pool classes: + +- `ConnectionPool` is a synchronous connection pool yielding + `~psycopg.Connection` objects and can be used by multithread applications. + +- `AsyncConnectionPool` has an interface similar to `!ConnectionPool`, but + with `asyncio` functions replacing blocking functions, and yields + `~psycopg.AsyncConnection` instances. + +- `NullConnectionPool` is a `!ConnectionPool` subclass exposing the same + interface of its parent, but not keeping any unused connection in its state. + See :ref:`null-pool` for details about related use cases. + +- `AsyncNullConnectionPool` has the same behaviour of the + `!NullConnectionPool`, but with the same async interface of the + `!AsyncConnectionPool`. .. note:: The `!psycopg_pool` package is distributed separately from the main `psycopg` package: use ``pip install psycopg[pool]``, or ``pip install @@ -271,3 +282,50 @@ listed here. .. automethod:: check .. automethod:: getconn .. automethod:: putconn + + +Null connection pools +--------------------- + +.. versionadded:: 3.1 + +The `NullConnectionPool` is a `ConnectionPool` subclass which doesn't create +connections preemptively and doesn't keep unused connections in its state. See +:ref:`null-pool` for further details. + +The interface of the object is entirely compatible with its parent class. Its +behaviour is similar, with the following differences: + +.. autoclass:: NullConnectionPool + + All the other constructor parameters are the same as in `ConnectionPool`. + + :param min_size: Always 0, cannot be changed. + :type min_size: `!int`, default: 0 + + :param max_size: If None or 0, create a new connection at every request, + without a maximum. If greater than 0, don't create more + than *max_size* connections and queue the waiting clients. + :type max_size: `!int`, default: None + + :param reset: It is only called when there are waiting clients in the + queue, before giving them a connection already nopen. If no + client is waiting, the connection is closed and discarded + without a fuss. + :type reset: `Callable[[Connection], None]` + + :param max_idle: Ignored, as null pools don't leave idle connections + sitting around. + + .. automethod:: wait + .. automethod:: resize + .. automethod:: check + + +The `AsyncNullConnectionPool` is, similarly, an `AsyncConnectionPool` subclass +with the same behaviour of the `NullConnectionPool`. + +.. autoclass:: AsyncNullConnectionPool + + The interface is the same of its parent class `AsyncConnectionPool`. The + behaviour is different in the same way described for `NullConnectionPool`. diff --git a/docs/news_pool.rst b/docs/news_pool.rst index 0ac87770f..69508d574 100644 --- a/docs/news_pool.rst +++ b/docs/news_pool.rst @@ -13,7 +13,8 @@ Future releases psycopg_pool 3.1.0 (unreleased) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- Add `ConnectionPool.open()` and `!open` parameter to the pool init +- Add :ref:`null-pool` (:ticket:`#148`). +- Add `ConnectionPool.open()` and *open* parameter to the pool init (:ticket:`#151`). diff --git a/psycopg_pool/psycopg_pool/null_pool.py b/psycopg_pool/psycopg_pool/null_pool.py index 58823cb2e..479799b28 100644 --- a/psycopg_pool/psycopg_pool/null_pool.py +++ b/psycopg_pool/psycopg_pool/null_pool.py @@ -56,7 +56,8 @@ class NullConnectionPool(_BaseNullConnectionPool, ConnectionPool): database works as expected. However the connection will not be stored in the pool. - Raise `PoolTimeout` if not ready within *timeout* sec. + Close the pool, and raise `PoolTimeout`, if not ready within *timeout* + sec. """ self._check_open_getconn() @@ -153,6 +154,10 @@ class NullConnectionPool(_BaseNullConnectionPool, ConnectionPool): self._return_connection(conn) def resize(self, min_size: int, max_size: Optional[int] = None) -> None: + """Change the size of the pool during runtime. + + Only *max_size* can be changed; *min_size* must remain 0. + """ min_size, max_size = self._check_size(min_size, max_size) logger.info(