From: Daniele Varrazzo Date: Thu, 26 Aug 2021 01:13:56 +0000 (+0200) Subject: More types adaptation docs X-Git-Tag: 3.0.beta1~34 X-Git-Url: http://git.ipfire.org/gitweb.cgi?a=commitdiff_plain;h=a19b3df4b706fc073002a9fd833b6b536a974d77;p=thirdparty%2Fpsycopg.git More types adaptation docs --- diff --git a/docs/api/types.rst b/docs/api/types.rst index b38dc4e04..bf792bb82 100644 --- a/docs/api/types.rst +++ b/docs/api/types.rst @@ -62,8 +62,7 @@ can extend the behaviour of the adapters: if you create a loader for Registering the `TypeInfo` in a context allows the adapters of that context to look up type information: for instance it allows to recognise automatically arrays of that type and load them from the - database as a list of the base type (how the base type is converted to - Python is demanded to a `Loader`. + database as a list of the base type. .. autoclass:: TypesRegistry @@ -98,14 +97,27 @@ specialised adapters configurations. correct oid for your :sql:`inetrange` type. -Objects wrappers +.. _numeric-wrappers: + +Numeric wrappers ---------------- +.. autoclass:: psycopg.types.numeric.Int2 +.. autoclass:: psycopg.types.numeric.Int4 +.. autoclass:: psycopg.types.numeric.Int8 +.. autoclass:: psycopg.types.numeric.Oid +.. autoclass:: psycopg.types.numeric.Float4 +.. autoclass:: psycopg.types.numeric.Float8 + +These wrappers can be used to force to dump Python numeric values to a certain +PostgreSQL type. This is rarely needed, usually the automatic rules do the +right thing. One case when they are needed is :ref:`copy-binary`. + + .. admonition:: TODO Document the various objects wrappers - - Int2, Int4, Int8, ... - Range diff --git a/docs/basic/adapt.rst b/docs/basic/adapt.rst index 39d7368e8..88ea7c911 100644 --- a/docs/basic/adapt.rst +++ b/docs/basic/adapt.rst @@ -32,9 +32,10 @@ TODO: complete table | `!float` | | :sql:`real` | :ref:`adapt-numbers` | | | | :sql:`double` | | +--------------------+-------------------------+ | - | | `!int` | | :sql:`smallint` | | - | | | | :sql:`integer` | | + | `!int` | | :sql:`smallint` | | + | | | :sql:`integer` | | | | | :sql:`bigint` | | + | | | :sql:`numeric` | | +--------------------+-------------------------+ | | `~decimal.Decimal` | :sql:`numeric` | | +--------------------+-------------------------+--------------------------+ @@ -103,37 +104,37 @@ Python `bool` values `!True` and `!False` are converted to the equivalent Numbers adaptation ------------------ -Python `int` values are converted to PostgreSQL :sql:`bigint` (a.k.a. -:sql:`int8`). Note that this could create some problems: - -- Python `!int` is unbounded. If you are inserting numbers larger than 2^63 - (so your target table is `numeric`, or you'll get an overflow on - arrival...) you should convert them to `~decimal.Decimal`. - -- Certain PostgreSQL functions and operators, such a :sql:`date + int` - expect an :sql:`integer` (aka :sql:`int4`): passing them a :sql:`bigint` - may cause an error:: - - cur.execute("SELECT current_date + %s", [1]) - # UndefinedFunction: operator does not exist: date + bigint +.. seealso:: - In this case you should add an :sql:`::int` cast to your query or use the - `~psycopg.types.numeric.Int4` wrapper:: + - `PostgreSQL numeric types + `__ - cur.execute("SELECT current_date + %s::int", [1]) +- Python `int` values can be converted to PostgreSQL :sql:`smallint`, + :sql:`integer`, :sql:`bigint`, or :sql:`numeric`, according to their numeric + value. Psycopg will choose the smallest data type available, because + PostgreSQL can automatically cast a type up (e.g. passing a `smallint` where + PostgreSQL expect an `integer` is gladly accepted) but will not cast down + automatically (e.g. if a function has an :sql:`integer` argument, passing it + a :sql:`bigint` value will fail, even if the value is 1). - cur.execute("SELECT current_date + %s", [Int4(1)]) +- Python `float` values are converted to PostgreSQL :sql:`float8`. - .. admonition:: TODO +- Python `~decimal.Decimal` values are converted to PostgreSQL :sql:`numeric`. - document Int* wrappers +On the way back, smaller types (:sql:`int2`, :sql:`int4`, :sql:`flaot4`) are +promoted to the larger Python counterpart. -Python `float` values are converted to PostgreSQL :sql:`float8`. +If you need a more precise control of how `!int` or `!float` are converted, +The `psycopg.types.numeric` module contains a few :ref:`wrappers +` which can be used to convince Psycopg to cast the values +to a specific PostgreSQL type:: -Python `~decimal.Decimal` values are converted to PostgreSQL :sql:`numeric`. + >>> conn.execute("select pg_typeof(%s), pg_typeof(%s)", (42, Int4(42))).fetchone() + ('smallint', 'integer') -On the way back, smaller types (:sql:`int2`, :sql:`int4`, :sql:`flaot4`) are -promoted to the larger Python counterpart. +These wrappers are rarely needed, because PostgreSQL cast rules and Psycopg +choices usually do the right thing. One use case where they are useful is +:ref:`copy-binary`. .. note:: @@ -142,13 +143,6 @@ promoted to the larger Python counterpart. an adapter to :ref:`cast PostgreSQL numeric to Python float `. This of course may imply a loss of precision. -.. seealso:: - - - `PostgreSQL numeric types - `__ - - `Musings about numeric adaptation choices - `__ - .. index:: pair: Strings; Adaptation @@ -160,16 +154,21 @@ promoted to the larger Python counterpart. Strings adaptation ------------------ -Python `str` is converted to PostgreSQL string syntax, and PostgreSQL types +.. seealso:: + + - `PostgreSQL character types + `__ + +Python `str` are converted to PostgreSQL string syntax, and PostgreSQL types such as :sql:`text` and :sql:`varchar` are converted back to Python `!str`: .. code:: python conn = psycopg.connect() conn.execute( - "INSERT INTO strtest (id, data) VALUES (%s, %s)", + "INSERT INTO menu (id, entry) VALUES (%s, %s)", (1, "Crème Brûlée at 4.99€")) - conn.execute("SELECT data FROM strtest WHERE id = 1").fetchone()[0] + conn.execute("SELECT entry FROM menu WHERE id = 1").fetchone()[0] 'Crème Brûlée at 4.99€' PostgreSQL databases `have an encoding`__, and `the session has an encoding`__ @@ -191,12 +190,12 @@ encoding/decoding errors: # The Latin-9 encoding can manage some European accented letters # and the Euro symbol conn.client_encoding = 'latin9' - conn.execute("SELECT data FROM strtest WHERE id = 1").fetchone()[0] + conn.execute("SELECT entry FROM menu WHERE id = 1").fetchone()[0] 'Crème Brûlée at 4.99€' # The Latin-1 encoding doesn't have a representation for the Euro symbol conn.client_encoding = 'latin1' - conn.execute("SELECT data FROM strtest WHERE id = 1").fetchone()[0] + conn.execute("SELECT entry FROM menu WHERE id = 1").fetchone()[0] # Traceback (most recent call last) # ... # UntranslatableCharacter: character with byte sequence 0xe2 0x82 0xac @@ -210,7 +209,7 @@ coming from the database, which will be returned as `bytes`: .. code:: python conn.client_encoding = "ascii" - conn.execute("SELECT data FROM strtest WHERE id = 1").fetchone()[0] + conn.execute("SELECT entry FROM menu WHERE id = 1").fetchone()[0] b'Cr\xc3\xa8me Br\xc3\xbbl\xc3\xa9e at 4.99\xe2\x82\xac' Alternatively you can cast the unknown encoding data to :sql:`bytea` to @@ -250,6 +249,40 @@ time and more bandwidth. See :ref:`binary-data` for details. .. __: https://www.postgresql.org/docs/current/datatype-binary.html +.. _adapt-date: + +Date/time types adaptation +-------------------------- + +.. seealso:: + + - `PostgreSQL date/time types + `__ + +- Python `~datetime.date` objects are converted to PostgreSQL :sql:`date`. +- Python `~datetime.datetime` objects are converted to PostgreSQL + :sql:`timestamp` (if they don't have a `!tzinfo` set) or :sql:`timestamptz` + (if they do). +- Python `~datetime.time` objects are converted to PostgreSQL :sql:`time` + (if they don't have a `!tzinfo` set) or :sql:`timetz` (if they do). +- Python `~datetime.timedelta` objects are converted to PostgreSQL + :sql:`interval`. + +PostgreSQL :sql:`timestamptz` values are returned with a timezone set to the +`connection TimeZone setting`__, which is available as a Python +`~zoneinfo.ZoneInfo` object in the `!Connection.info`.\ `~ConnectionInfo.timezone` +attribute:: + + >>> cnn.info.timezone + zoneinfo.ZoneInfo(key='Europe/London') + + >>> cnn.execute("select '2048-07-08 12:00'::timestamptz").fetchone()[0] + datetime.datetime(2048, 7, 8, 12, 0, tzinfo=zoneinfo.ZoneInfo(key='Europe/London')) + + +.. __: https://www.postgresql.org/docs/current/runtime-config-client.html#GUC-TIMEZONE + + .. _adapt-json: JSON adaptation @@ -262,7 +295,7 @@ types`__, allowing to customise the load and dump function used. Because several Python objects could be considered JSON (dicts, lists, scalars, even date/time if using a dumps function customised to use them), -Psycopg requires you to wrap what you want to dump as JSON into a wrapper: +Psycopg requires you to wrap the object to dump as JSON into a wrapper: either `psycopg.types.json.Json` or `~psycopg.types.json.Jsonb`. .. code:: python @@ -318,7 +351,6 @@ take precedence over what specified by `!set_json_dumps()`. # will insert: {'uuid': '0a40799d-3980-4c65-8315-2956b18ab0e1'} -.. _adapt-date: .. _adapt-list: .. _adapt-composite: .. _adapt-hstore: diff --git a/docs/basic/copy.rst b/docs/basic/copy.rst index 7f0c52ed2..89af1a6f7 100644 --- a/docs/basic/copy.rst +++ b/docs/basic/copy.rst @@ -39,6 +39,8 @@ have to commit the pending changes and you can still roll them back. See :ref:`transactions` for details. +.. _copy-in-row: + Writing data row-by-row ----------------------- @@ -64,18 +66,26 @@ In order to read or write from `!Copy` row-by-row you must not specify :sql:`COPY` options such as :sql:`FORMAT CSV`, :sql:`DELIMITER`, :sql:`NULL`: please leave these details alone, thank you :) + +.. _copy-binary: + +Binary copy +----------- + Binary copy is supported by specifying :sql:`FORMAT BINARY` in the :sql:`COPY` statement. In order to load binary data, all the types passed to the database must have a binary dumper registered (see see :ref:`binary-data`). Note that PostgreSQL is particularly finicky when loading data in binary mode -and will apply *no cast rule*. This means that e.g. passing a Python `!int` -object to an :sql:`integer` column (aka :sql:`int4`) will likely fail, because -the default `!int` `~adapt.Dumper` will use the :sql:`bigint` aka :sql:`int8` -format. You can work around the problem by registering the right binary dumper -on the cursor or using the right data wrapper (see :ref:`adaptation`). +and will apply *no cast rule*. This means that e.g. passing the value 100 to +an `integer` column will fail because Psycopg will pass it as a `smallint` +value. You can work around the problem by registering the right binary +`~adapt.Dumper` on the cursor (see :ref:`adaptation`) or using the right data +wrapper (e.g. `~psycopg.types.numeric.Int4`). +.. _copy-out-row: + Reading data row-by-row ----------------------- @@ -105,6 +115,8 @@ you have to specify them yourselves. print(row) # (10, datetime.date(2046, 12, 24)) +.. _copy-block: + Copying block-by-block ---------------------- @@ -136,6 +148,8 @@ produce a stream of `!bytes`: f.write(data) +.. _copy-async: + Asynchronous copy support ------------------------- diff --git a/psycopg/psycopg/_wrappers.py b/psycopg/psycopg/_wrappers.py index 0064872ab..9e75a0c22 100644 --- a/psycopg/psycopg/_wrappers.py +++ b/psycopg/psycopg/_wrappers.py @@ -12,6 +12,9 @@ _MODULE = "psycopg.types.numeric" class Int2(int): + """ + Force dumping a Python `!int` as a PostgreSQL :sql:`smallint/int2`. + """ __module__ = _MODULE __slots__ = () @@ -27,6 +30,9 @@ class Int2(int): class Int4(int): + """ + Force dumping a Python `!int` as a PostgreSQL :sql:`integer/int4`. + """ __module__ = _MODULE __slots__ = () @@ -42,6 +48,9 @@ class Int4(int): class Int8(int): + """ + Force dumping a Python `!int` as a PostgreSQL :sql:`bigint/int8`. + """ __module__ = _MODULE __slots__ = () @@ -57,6 +66,9 @@ class Int8(int): class IntNumeric(int): + """ + Force dumping a Python `!int` as a PostgreSQL :sql:`numeric/decimal`. + """ __module__ = _MODULE __slots__ = () @@ -72,6 +84,9 @@ class IntNumeric(int): class Float4(float): + """ + Force dumping a Python `!float` as a PostgreSQL :sql:`float4/real`. + """ __module__ = _MODULE __slots__ = () @@ -87,6 +102,9 @@ class Float4(float): class Float8(float): + """ + Force dumping a Python `!float` as a PostgreSQL :sql:`float8/double precision`. + """ __module__ = _MODULE __slots__ = () @@ -102,6 +120,9 @@ class Float8(float): class Oid(int): + """ + Force dumping a Python `!int` as a PostgreSQL :sql:`oid`. + """ __module__ = _MODULE __slots__ = ()