From: Daniele Varrazzo Date: Sat, 28 Aug 2021 14:35:39 +0000 (+0200) Subject: Improve rows documentation X-Git-Tag: 3.0.beta1~18 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=386b41cfe9e5193aee9eeae6f341a7f01b8ecd93;p=thirdparty%2Fpsycopg.git Improve rows documentation Add api section, move from examples to theory instead of the other way around. --- diff --git a/docs/advanced/index.rst b/docs/advanced/index.rst index 3606224a3..f8d6f8317 100644 --- a/docs/advanced/index.rst +++ b/docs/advanced/index.rst @@ -12,8 +12,8 @@ usages. :caption: Contents: async - cursors rows pool + cursors adapt prepare diff --git a/docs/advanced/rows.rst b/docs/advanced/rows.rst index 5b7a9837d..b5bdd0cdf 100644 --- a/docs/advanced/rows.rst +++ b/docs/advanced/rows.rst @@ -7,36 +7,76 @@ Row factories ============= -Cursor's `fetch*` methods return tuples of column values by default. This can -be changed to adapt the needs of the programmer by using custom *row -factories*. +Cursor's `fetch*` methods, by default, return the records received from the +database as tuples. This can be changed to better suit the needs of the +programmer by using custom *row factories*. -A row factory (formally implemented by the `~psycopg.rows.RowFactory` -protocol) is a callable that accepts a `Cursor` object and returns another -callable (formally the `~psycopg.rows.RowMaker` protocol) accepting a -`values` tuple and returning a row in the desired form. +The module `psycopg.rows` exposes several row factories ready to be used. For +instance, if you want to return your records as dictionaries, you can use +`~psycopg.rows.dict_row`:: -.. autoclass:: psycopg.rows.RowMaker() + >>> from psycopg.rows import dict_row - .. method:: __call__(values: Sequence[Any]) -> Row + >>> conn = psycopg.connect(DSN, row_factory=dict_row) - Convert a sequence of values from the database to a finished object. + >>> conn.execute("select 'John Doe' as name, 33 as age").fetchone() + {'name': 'John Doe', 'age': 33} +The `!row_factory` parameter is supported by the `~Connection.connect()` +method and the `~Connection.cursor()` method. Later usage of `!row_factory` +overrides a previous one. It is also possible to change the +`Connection.row_factory` or `Cursor.row_factory` attributes to change what +they return:: -.. autoclass:: psycopg.rows.RowFactory() + >>> cur = conn.cursor(row_factory=dict_row) + >>> cur.execute("select 'John Doe' as name, 33 as age").fetchone() + {'name': 'John Doe', 'age': 33} - .. method:: __call__(cursor: Cursor[Row]) -> RowMaker[Row] + >>> from psycopg.rows import namedtuple_row + >>> cur.row_factory = namedtuple_row + >>> cur.execute("select 'John Doe' as name, 33 as age").fetchone() + Row(name='John Doe', age=33) - Inspect the result on a cursor and return a `RowMaker` to convert rows. +If you want to return objects of your choice you can use a row factory +*generator*, for instance `~psycopg.rows.class_row` or +`~psycopg.rows.args_row`, or you can :ref:`write your own row factory +`:: -.. autoclass:: psycopg.rows.AsyncRowFactory() + >>> from dataclasses import dataclass -.. autoclass:: psycopg.rows.BaseRowFactory() + >>> @dataclass + ... class Person: + ... name: str + ... age: int + ... weight: Optional[int] = None + >>> from psycopg.rows import class_row + >>> cur = conn.cursor(row_factory=class_row(Person)) + >>> cur.execute("select 'John Doe' as name, 33 as age").fetchone() + Person(name='John Doe', age=33, weight=None) -Note that it's easy to implement an object implementing both `!RowFactory` and -`!AsyncRowFactory`: usually, everything you need to implement a row factory is -to access `~Cursor.description`, which is provided by both the cursor flavours. + +.. index:: + single: Row Maker + single: Row Factory + +.. _row-factory-create: + +Creating new row factories +-------------------------- + +A *row factory* is a callable that accepts a `Cursor` object and returns +another callable, a *row maker*, which takes a raw data (as a sequence of +values) and returns the desired object. + +The role of the row factory is to inspect a query result (it is called after a +query is executed and properties such as `~Cursor.description` and +`~Cursor.pgresult` are available on the cursor) and to prepare a callable +which is efficient to call repeatedly (because, for instance, the names of the +columns are extracted, sanitised, and stored in local variables). + +Formally, these objects are represented by the `~psycopg.rows.RowFactory` and +`~psycopg.rows.RowMaker` protocols. `~RowFactory` objects can be implemented as a class, for instance: @@ -75,58 +115,19 @@ These can then be used by specifying a `row_factory` argument in person = cur.fetchone() print(f"{person['first_name']} {person['last_name']}") -Later usages of `row_factory` override earlier definitions; for instance, -the `row_factory` specified at `Connection.connect()` can be overridden by -passing another value at `Connection.cursor()`. - - -Available row factories ------------------------ - -The module `psycopg.rows` provides the implementation for a few row factories: - -.. currentmodule:: psycopg.rows - -.. autofunction:: tuple_row -.. autofunction:: dict_row -.. autofunction:: namedtuple_row -.. autofunction:: class_row -.. autofunction:: args_row -.. autofunction:: kwargs_row - - This is not a row factory, but rather a factory of row factories. - Specifying ``row_factory=class_row(MyClass)`` will create connections and - cursors returning `!MyClass` objects on fetch. - - Example:: - - from dataclasses import dataclass - import psycopg - from psycopg.rows import class_row - - @dataclass - class Person: - first_name: str - last_name: str - age: int = None - - conn = psycopg.connect() - cur = conn.cursor(row_factory=class_row(Person)) - - cur.execute("select 'John' as first_name, 'Smith' as last_name").fetchone() - # Person(first_name='John', last_name='Smith', age=None) +.. _row-factory-static: Use with a static analyzer -------------------------- -The `~psycopg.Connection` and `~psycopg.Cursor` classes are `generic -types`__: the parameter `!Row` is passed by the ``row_factory`` argument (of -the `~Connection.connect()` and the `~Connection.cursor()` method) and it -controls what type of record is returned by the fetch methods of the cursors. -The default `tuple_row()` returns a generic tuple as return type (`Tuple[Any, -...]`). This information can be used for type checking using a static analyzer -such as mypy_. +The `~psycopg.Connection` and `~psycopg.Cursor` classes are `generic types`__: +the parameter `!Row` is passed by the ``row_factory`` argument (of the +`~Connection.connect()` and the `~Connection.cursor()` method) and it controls +what type of record is returned by the fetch methods of the cursors. The +default `~psycopg.rows.tuple_row` returns a generic tuple as return type +(`Tuple[Any, ...]`). This information can be used for type checking using a +static analyzer such as mypy_. .. _mypy: https://mypy.readthedocs.io/ .. __: https://mypy.readthedocs.io/en/stable/generics.html @@ -154,7 +155,7 @@ such as mypy_. Example: returning records as Pydantic models ---------------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Using Pydantic_ it is possible to enforce static typing at runtime. Using a Pydantic model factory the code can be checked statically using mypy and @@ -165,7 +166,7 @@ compatible with the model. The following example can be checked with ``mypy --strict`` without reporting any issue. Pydantic will also raise a runtime error in case the -`!PersonFactory` is used with a query that returns incompatible data. +`!Person` is used with a query that returns incompatible data. .. code:: python diff --git a/docs/api/index.rst b/docs/api/index.rst index 6ea9b6010..b466f55ca 100644 --- a/docs/api/index.rst +++ b/docs/api/index.rst @@ -15,9 +15,10 @@ This sections is a reference for all the public objects exposed by the connections cursors sql + rows errors - pool adapt types abc pq + pool diff --git a/docs/api/rows.rst b/docs/api/rows.rst new file mode 100644 index 000000000..6720dee86 --- /dev/null +++ b/docs/api/rows.rst @@ -0,0 +1,74 @@ +.. _psycopg.rows: + +`rows` -- row factory implementations +===================================== + +.. module:: psycopg.rows + +The module exposes a few generic `~psycopg.RowFactory` implementation, which +can be used to retrieve data from the database in more complex structures than +the basic tuples. + +Check out :ref:`row-factories` for information about how to use these objects. + +.. autofunction:: tuple_row +.. autofunction:: dict_row +.. autofunction:: namedtuple_row +.. autofunction:: class_row + + This is not a row factory, but rather a factory of row factories. + Specifying ``row_factory=class_row(MyClass)`` will create connections and + cursors returning `!MyClass` objects on fetch. + + Example:: + + from dataclasses import dataclass + import psycopg + from psycopg.rows import class_row + + @dataclass + class Person: + first_name: str + last_name: str + age: int = None + + conn = psycopg.connect() + cur = conn.cursor(row_factory=class_row(Person)) + + cur.execute("select 'John' as first_name, 'Smith' as last_name").fetchone() + # Person(first_name='John', last_name='Smith', age=None) + +.. autofunction:: args_row +.. autofunction:: kwargs_row + + +Formal rows protocols +--------------------- + +These objects can be used to describe your own rows adapter for static typing +checks, such as mypy_. + +.. _mypy: https://mypy.readthedocs.io/ + + +.. autoclass:: psycopg.rows.RowMaker() + + .. method:: __call__(values: Sequence[Any]) -> Row + + Convert a sequence of values from the database to a finished object. + + +.. autoclass:: psycopg.rows.RowFactory() + + .. method:: __call__(cursor: Cursor[Row]) -> RowMaker[Row] + + Inspect the result on a cursor and return a `RowMaker` to convert rows. + +.. autoclass:: psycopg.rows.AsyncRowFactory() + +.. autoclass:: psycopg.rows.BaseRowFactory() + +Note that it's easy to implement an object implementing both `!RowFactory` and +`!AsyncRowFactory`: usually, everything you need to implement a row factory is +to access the cursor's `~psycopg.Cursor.description`, which is provided by +both the cursor flavours. diff --git a/psycopg/psycopg/rows.py b/psycopg/psycopg/rows.py index 927c2919f..4ca752638 100644 --- a/psycopg/psycopg/rows.py +++ b/psycopg/psycopg/rows.py @@ -92,18 +92,24 @@ database. """ -def tuple_row(cursor: "BaseCursor[Any, TupleRow]") -> RowMaker[TupleRow]: +def tuple_row(cursor: "BaseCursor[Any, TupleRow]") -> "RowMaker[TupleRow]": r"""Row factory to represent rows as simple tuples. - This is the default factory. + This is the default factory, used when `~psycopg.Connection.connect()` or + `~psycopg.Connection.cursor()` are called withouth a `!row_factory` + parameter. + """ # Implementation detail: make sure this is the tuple type itself, not an # equivalent function, because the C code fast-paths on it. return tuple -def dict_row(cursor: "BaseCursor[Any, DictRow]") -> RowMaker[DictRow]: - """Row factory to represent rows as dictionaries.""" +def dict_row(cursor: "BaseCursor[Any, DictRow]") -> "RowMaker[DictRow]": + """Row factory to represent rows as dictionaries. + + The dictionary keys are taken from the column names of the returned columns. + """ desc = cursor.description if desc is None: return no_result @@ -118,8 +124,12 @@ def dict_row(cursor: "BaseCursor[Any, DictRow]") -> RowMaker[DictRow]: def namedtuple_row( cursor: "BaseCursor[Any, NamedTuple]", -) -> RowMaker[NamedTuple]: - """Row factory to represent rows as `~collections.namedtuple`.""" +) -> "RowMaker[NamedTuple]": + """Row factory to represent rows as `~collections.namedtuple`. + + The field names are taken from the column names of the returned columns, + with some mangling to deal with invalid names. + """ desc = cursor.description if desc is None: return no_result @@ -157,7 +167,7 @@ def class_row(cls: Type[T]) -> BaseRowFactory[T]: :rtype: `!Callable[[Cursor],` `RowMaker`\[~T]] """ - def class_row_(cur: "BaseCursor[Any, T]") -> RowMaker[T]: + def class_row_(cur: "BaseCursor[Any, T]") -> "RowMaker[T]": desc = cur.description if desc is None: return no_result @@ -179,7 +189,7 @@ def args_row(func: Callable[..., T]) -> BaseRowFactory[T]: returned by the query as positional arguments. """ - def args_row_(cur: "BaseCursor[Any, T]") -> RowMaker[T]: + def args_row_(cur: "BaseCursor[Any, T]") -> "RowMaker[T]": def args_row__(values: Sequence[Any]) -> T: return func(*values) @@ -195,7 +205,7 @@ def kwargs_row(func: Callable[..., T]) -> BaseRowFactory[T]: returned by the query as keyword arguments. """ - def kwargs_row_(cur: "BaseCursor[Any, T]") -> RowMaker[T]: + def kwargs_row_(cur: "BaseCursor[Any, T]") -> "RowMaker[T]": desc = cur.description if desc is None: return no_result