From: Daniele Varrazzo Date: Mon, 23 Nov 2020 07:14:17 +0000 (+0000) Subject: Added some documentation on the adaptation system X-Git-Tag: 3.0.dev0~324 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=2755f489ed07c21fe51f6c02b5b83d7c055971d2;p=thirdparty%2Fpsycopg.git Added some documentation on the adaptation system --- diff --git a/docs/adaptation.rst b/docs/adaptation.rst index 78a67ce86..ad73e37dc 100644 --- a/docs/adaptation.rst +++ b/docs/adaptation.rst @@ -1,6 +1,160 @@ .. _adaptation: -Adaptation of data between Python and PostgreSQL -================================================ +.. module:: psycopg3.adapt -TODO + +``psycopg3.adapt`` -- Data adaptation configuration +=================================================== + +The adaptation system is at the core of psycopg3 and allows to customise the +way Python objects are converted to PostgreSQL when a query is performed and +how PostgreSQL values are converted to Python objects when query results are +returned. + +.. note:: + For a high-level view of the conversion of types between Python and + PostgreSQL please look at :ref:`query-parameters`. Using the objects + described in this page is useful if you intend to *customise* the + adaptation rules. + +The `Dumper` is the base object to perform conversion from a Python object to +a `!bytes` string understood by PostgreSQL. The string returned *shouldn't be +quoted*: the value will be passed to the database using functions such as +:pq:`PQexecParams()` so quoting and quotes escaping is not necessary. + +The `Loader` is the base object to perform the opposite operation: to read a +`!bytes` string from PostgreSQL and create a Python object. + +`!Dumper` and `!Loader` are abstract classes: concrete classes must implement +the `~Dumper.dump()` and `~Loader.load()` method. `!psycopg3` provides +implementation for several builtin Python and PostgreSQL types. + + +.. rubric:: Dumpers and loaders configuration + +Dumpers and loaders can be registered on different scopes: globally, per +`~psycopg3.Connection`, per `~psycopg3.Cursor`, so that adaptation rules can +be customised for specific needs within the same application: in order to do +so you can use the *context* parameter of `~Dumper.register()` and similar +methods. + +Dumpers and loaders might need to handle data in text and binary format, +according to how they are registered (e.g. with `~Dumper.register()` or +`~Dumper.register_binary()`). For most types the format is different so there +will have to be two different classes. + + +.. rubric:: Dumpers and loaders life cycle + +Registering dumpers and loaders will instruct `!psycopg3` to use them +in the queries to follow, in the context where they have been registered. + +When a query is performed, a `Transformer` object will be used to instantiate +dumpers and loaders as requested and to dispatch the values to convert +to the right instance: + +- The `!Trasformer` will look up the most specific adapter: one registered on + the `~psycopg3.Cursor` if available, then one registered on the + `~psycopg3.Connection`, finally a global one. + +- For every Python type passed as query argument there will be a `!Dumper` + instantiated. All the objects of the same type will use the same loader. + +- For every OID returned by a query there will be a `!Loader` instantiated. + All the values with the same OID will be converted by the same loader. + +- Recursive types (e.g. Python lists, PostgreSQL arrays and composite types) + will use the same adaptation rules. + +As a consequence it is possible to perform certain choices only once per query +(e.g. looking up the connection encoding) and then call a fast-path operation +for each value to convert. + +Querying will fail if a Python object for which there isn't a `!Dumper` +registered (for the right `~psycopg3.pq.Format`) is used as query parameter. +If the query returns a data type whose OID doesn't have a `!Loader`, the +value will be returned as a string (or bytes string for binary types). + + +Objects involved in types adaptation +------------------------------------ + +.. autoclass:: Dumper(src, context=None) + + :param src: The type that will be managed by this dumper. + :type src: type + :param context: The context where the transformation is performed. If not + specified the conversion might be inaccurate, for instance it will not + be possible to know the connection encoding or the server date format. + :type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer` + + .. automethod:: dump + + The format returned by dump shouldn't contain quotes or escaped + values. + + .. automethod:: quote + + By default will return the `dump()` value quoted and sanitised, so + that the result can be used to build a SQL string. For instance, the + method will be used by `~psycopg3.sql.Literal` to convert a value + client-side. + + This method only makes sense for text dumpers; the result of calling + it on a binary dumper is undefined. It might scratch your car, or burn + your cake. Don't tell me I didn't warn you. + + .. autoattribute:: oid + :annotation: int + + .. automethod:: register(src, context=None) + + :param src: The type to manage. + :type src: `!type` or `!str` + :param context: Where the dumper should be used. If `!None` the dumper + will be used globally. + :type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer` + + If *src* is specified as string it will be lazy-loaded, so that it + will be possible to register it without importing it before. In this + case it should be the fully qualified name of the object (e.g. + ``"uuid.UUID"``). + + .. automethod:: register_binary(src, context=None) + + In order to convert a value in binary you can use a ``%b`` placeholder + in the query instead of ``%s``. + + Parameters as the same as in `register()`. + + +.. autoclass:: Loader(oid, context=None) + + :param oid: The type that will be managed by this dumper. + :type oid: int + :param context: The context where the transformation is performed. If not + specified the conversion might be inaccurate, for instance it will not + be possible to know the connection encoding or the server date format. + :type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer` + + .. automethod:: load + + .. automethod:: register(oid, context=None) + + :param oid: The PostgreSQL OID to manage. + :type oid: `!int` + :param context: Where the loader should be used. If `!None` the loader + will be used globally. + :type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer` + + .. automethod:: register_binary(oid, context=None) + + Parameters as the same as in `register()`. + + +.. autoclass:: Transformer(context=None) + + :param context: The context where the transformer should operate. + :type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer` + + TODO: finalise the interface of this object diff --git a/docs/index.rst b/docs/index.rst index 6127b29f0..db7305616 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -21,9 +21,9 @@ the COPY support. install usage - adaptation connection cursor + adaptation sql errors pq diff --git a/psycopg3/psycopg3/_transform.py b/psycopg3/psycopg3/_transform.py index 0ac2b0893..713fe2227 100644 --- a/psycopg3/psycopg3/_transform.py +++ b/psycopg3/psycopg3/_transform.py @@ -28,7 +28,7 @@ class Transformer: The life cycle of the object is the query, so it is assumed that stuff like the server version or connection encoding will not change. It can have its - state so adapting several values of the same type can use optimisations. + state so adapting several values of the same type can be optimised. """ def __init__(self, context: AdaptContext = None): diff --git a/psycopg3/psycopg3/adapt.py b/psycopg3/psycopg3/adapt.py index 7055232d3..ddb763a4d 100644 --- a/psycopg3/psycopg3/adapt.py +++ b/psycopg3/psycopg3/adapt.py @@ -18,6 +18,10 @@ TEXT_OID = builtins["text"].oid class Dumper: + """ + Convert Python object of the type *src* to PostgreSQL representation. + """ + globals: DumpersMap = {} connection: Optional[BaseConnection] @@ -27,9 +31,11 @@ class Dumper: self.connection = _connection_from_context(context) def dump(self, obj: Any) -> bytes: + """Convert the object *obj* to PostgreSQL representation.""" raise NotImplementedError() def quote(self, obj: Any) -> bytes: + """Convert the object *obj* to escaped representation.""" value = self.dump(obj) if self.connection: @@ -41,6 +47,7 @@ class Dumper: @property def oid(self) -> int: + """The oid to pass to the server, if known.""" return 0 @classmethod @@ -50,6 +57,9 @@ class Dumper: context: AdaptContext = None, format: Format = Format.TEXT, ) -> None: + """ + Configure *context* to use this dumper to convert object of type *src*. + """ if not isinstance(src, (str, type)): raise TypeError( f"dumpers should be registered on classes, got {src} instead" @@ -62,6 +72,9 @@ class Dumper: def register_binary( cls, src: Union[type, str], context: AdaptContext = None ) -> None: + """ + Configure *context* to use this dumper for binary format conversion. + """ cls.register(src, context, format=Format.BINARY) @classmethod @@ -84,6 +97,10 @@ class Dumper: class Loader: + """ + Convert PostgreSQL objects with OID *oid* to Python objects. + """ + globals: LoadersMap = {} connection: Optional[BaseConnection] @@ -93,6 +110,7 @@ class Loader: self.connection = _connection_from_context(context) def load(self, data: bytes) -> Any: + """Convert a PostgreSQL value to a Python object.""" raise NotImplementedError() @classmethod @@ -102,6 +120,9 @@ class Loader: context: AdaptContext = None, format: Format = Format.TEXT, ) -> None: + """ + Configure *context* to use this loader to convert values with OID *oid*. + """ if not isinstance(oid, int): raise TypeError( f"loaders should be registered on oid, got {oid} instead" @@ -112,6 +133,9 @@ class Loader: @classmethod def register_binary(cls, oid: int, context: AdaptContext = None) -> None: + """ + Configure *context* to use this loader to convert binary values. + """ cls.register(oid, context, format=Format.BINARY) @classmethod