From: Daniele Varrazzo Date: Sun, 24 Jul 2022 03:01:25 +0000 (+0100) Subject: docs(copy): add documentation for COPY Writer objects X-Git-Tag: 3.1~44^2 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=022910045f0dc88519956d48cae64cd77323f205;p=thirdparty%2Fpsycopg.git docs(copy): add documentation for COPY Writer objects --- diff --git a/docs/api/copy.rst b/docs/api/copy.rst new file mode 100644 index 000000000..0f12c743d --- /dev/null +++ b/docs/api/copy.rst @@ -0,0 +1,117 @@ +.. currentmodule:: psycopg + +COPY-related objects +==================== + +The main objects (`Copy`, `AsyncCopy`) present the main interface to exchange +data during a COPY operations. These objects are normally obtained by the +methods `Cursor.copy()` and `AsyncCursor.copy()`; however, they can be also +created directly, for instance to write to a destination which is not a +database (e.g. using a `~psycopg.copy.FileWriter`). + +See :ref:`copy` for details. + + +Main Copy objects +----------------- + +.. autoclass:: Copy() + + The object is normally returned by ``with`` `Cursor.copy()`. + + .. automethod:: write_row + + The data in the tuple will be converted as configured on the cursor; + see :ref:`adaptation` for details. + + .. automethod:: write + .. automethod:: read + + Instead of using `!read()` you can iterate on the `!Copy` object to + read its data row by row, using ``for row in copy: ...``. + + .. automethod:: rows + + Equivalent of iterating on `read_row()` until it returns `!None` + + .. automethod:: read_row + .. automethod:: set_types + + +.. autoclass:: AsyncCopy() + + The object is normally returned by ``async with`` `AsyncCursor.copy()`. + Its methods are similar to the ones of the `Copy` object but offering an + `asyncio` interface (`await`, `async for`, `async with`). + + .. automethod:: write_row + .. automethod:: write + .. automethod:: read + + Instead of using `!read()` you can iterate on the `!AsyncCopy` object + to read its data row by row, using ``async for row in copy: ...``. + + .. automethod:: rows + + Use it as `async for record in copy.rows():` ... + + .. automethod:: read_row + + +.. _copy-writers: + +Writer objects +-------------- + +.. currentmodule:: psycopg.copy + +.. versionadded:: 3.1 + +Copy writers are helper objects to specify where to write COPY-formatted data. +By default, data is written to the database (using the `LibpqWriter`). It is +possible to write copy-data for offline use by using a `FileWriter`, or to +customize further writing by implementing your own `Writer` or `AsyncWriter` +subclass. + +Writers instances can be used passing them to the cursor +`~psycopg.Cursor.copy()` method or to the `~psycopg.Copy` constructor, as the +``writer`` argument. + +.. autoclass:: Writer + + This is an abstract base class: subclasses are required to implement their + `write()` method. + + .. automethod:: write + .. automethod:: finish + + +.. autoclass:: LibpqWriter + + This is the writer used by default if none is specified. + + +.. autoclass:: FileWriter + + This writer should be used without executing a :sql:`COPY` operation on + the database. For example, if `records` is a list of tuples containing + data to save in COPY format to a file (e.g. for later import), it can be + used as: + + .. code:: python + + with open("target-file.pgcopy", "wb") as f: + with Copy(cur, writer=FileWriter(f)) as copy: + for record in records + copy.write_row(record) + + +.. autoclass:: AsyncWriter + + This class methods have the same semantics of the ones of `Writer`, but + offer an async interface. + + .. automethod:: write + .. automethod:: finish + +.. autoclass:: AsyncLibpqWriter diff --git a/docs/api/index.rst b/docs/api/index.rst index a53fed6e0..b99550d89 100644 --- a/docs/api/index.rst +++ b/docs/api/index.rst @@ -14,6 +14,7 @@ This sections is a reference for all the public objects exposed by the module connections cursors + copy objects sql rows diff --git a/docs/api/objects.rst b/docs/api/objects.rst index 620905581..f6bb5cfe9 100644 --- a/docs/api/objects.rst +++ b/docs/api/objects.rst @@ -129,54 +129,6 @@ The description `Column` object .. autoattribute:: scale -COPY-related objects --------------------- - -.. autoclass:: Copy() - - The object is normally returned by ``with`` `Cursor.copy()`. - - See :ref:`copy` for details. - - .. automethod:: write_row - - The data in the tuple will be converted as configured on the cursor; - see :ref:`adaptation` for details. - - .. automethod:: write - .. automethod:: read - - Instead of using `!read()` you can iterate on the `!Copy` object to - read its data row by row, using ``for row in copy: ...``. - - .. automethod:: rows - - Equivalent of iterating on `read_row()` until it returns `!None` - - .. automethod:: read_row - .. automethod:: set_types - - -.. autoclass:: AsyncCopy() - - The object is normally returned by ``async with`` `AsyncCursor.copy()`. - Its methods are similar to the ones of the `Copy` object but offering an - `asyncio` interface (`await`, `async for`, `async with`). - - .. automethod:: write_row - .. automethod:: write - .. automethod:: read - - Instead of using `!read()` you can iterate on the `!AsyncCopy` object - to read its data row by row, using ``async for row in copy: ...``. - - .. automethod:: rows - - Use it as `async for record in copy.rows():` ... - - .. automethod:: read_row - - Notifications ------------- diff --git a/docs/news.rst b/docs/news.rst index 74eea147d..7952c74a6 100644 --- a/docs/news.rst +++ b/docs/news.rst @@ -24,6 +24,7 @@ Psycopg 3.1 (unreleased) - `~Cursor.executemany()` performance improved by using batch mode internally (:ticket:`#145`). - Add parameters to `~Cursor.copy()`. +- Add :ref:`COPY Writer objects `. - Resolve domain names asynchronously in `AsyncConnection.connect()` (:ticket:`#259`). - Add `pq.PGconn.trace()` and related trace functions (:ticket:`#167`). diff --git a/psycopg/psycopg/copy.py b/psycopg/psycopg/copy.py index 3c54aafec..bec670b7a 100644 --- a/psycopg/psycopg/copy.py +++ b/psycopg/psycopg/copy.py @@ -203,7 +203,20 @@ class BaseCopy(Generic[ConnectionType]): class Copy(BaseCopy["Connection[Any]"]): - """Manage a :sql:`COPY` operation.""" + """Manage a :sql:`COPY` operation. + + :param cursor: the cursor where the operation is performed. + :param binary: if `!True`, write binary format. + :param writer: the object to write to destination. If not specified, write + to the `!cursor` connection. + + Choosing `!binary` is not necessary if the cursor has executed a + :sql:`COPY` operation, because the operation result describes the format + too. The parameter is useful when a `!Copy` object is created manually and + no operation is performed on the cursor, such as when using ``writer=``\\ + `~psycopg.copy.FileWriter`. + + """ __module__ = "psycopg" @@ -326,6 +339,8 @@ class Writer(ABC): def finish(self, exc: Optional[BaseException] = None) -> None: """ Called when write operations are finished. + + If operations finished with an error, it will be passed to ``exc``. """ pass @@ -438,7 +453,8 @@ class FileWriter(Writer): """ A `Writer` to write copy data to a file-like object. - The file must be open for writing in binary mode. + :param file: the file where to write copy data. It muse be open for writing + in binary mode. """ def __init__(self, file: IO[bytes]): @@ -525,20 +541,14 @@ class AsyncCopy(BaseCopy["AsyncConnection[Any]"]): class AsyncWriter(ABC): """ - A class to write copy data somewhere. + A class to write copy data somewhere (for async connections). """ @abstractmethod async def write(self, data: Buffer) -> None: - """ - Write some data to destination. - """ ... async def finish(self, exc: Optional[BaseException] = None) -> None: - """ - Called when write operations are finished. - """ pass