so you can use the *context* parameter of `Dumper.register()` and
`Loader.register()`.
+When a `!Connection` is created, it inherits the global adapters
+configuration; when a `!Cursor` is created it inherits its `!Connection`
+configuration.
+
.. note::
`!register()` is a class method on the base class, so if you
subclass `!Dumper` or `!Loader` you should call the ``.register()`` on the
class you created.
-If the data type has also a distinct binary format which you may want to use
-from psycopg (as documented in :ref:`binary-data`) you may want to implement
-binary loaders and dumpers, whose only difference from text dumper is that
-they must be registered using ``format=Format.BINARY`` in `Dumper.register()`
-and `Loader.register()`.
-
.. admonition:: TODO
- Example: infinity date customisation
Registering dumpers and loaders will instruct `!psycopg3` to use them
in the queries to follow, in the context where they have been registered.
-When a query is performed, a `Transformer` object will be used to instantiate
-dumpers and loaders as requested and to dispatch the values to convert
-to the right instance:
+When a query is performed on a `!Cursor`, a `Transformer` object is created
+as a local context to manage conversions during the query, instantiating the
+required dumpers and loaders and dispatching the values to convert to the
+right instance.
-- The `!Trasformer` will look up the most specific adapter: one registered on
- the `~psycopg3.Cursor` if available, then one registered on the
- `~psycopg3.Connection`, finally a global one.
+- The `!Trasformer` copies the adapters configuration from the `!Cursor`, thus
+ inheriting all the changes made to the global configuration, the current
+ `!Connection`, the `!Cursor`.
-- For every Python type passed as query argument there will be a `!Dumper`
- instantiated. All the objects of the same type will use the same loader.
+- For every Python type passed as query argument, the `!Transformer` will
+ instantiate a `!Dumper`. All the objects of the same type will be converted
+ by the same dumper.
-- For every OID returned by a query there will be a `!Loader` instantiated.
- All the values with the same OID will be converted by the same loader.
+- For every OID returned by the query, the `!Transformer` will instantiate a
+ `!Loader`. All the values with the same OID will be converted by the same
+ loader.
- Recursive types (e.g. Python lists, PostgreSQL arrays and composite types)
will use the same adaptation rules.
move to API section
+
+.. autoclass:: Format
+ :members:
+
+
.. autoclass:: Dumper(src, context=None)
This is an abstract base class: subclasses *must* implement the `dump()`
- method. They *may* implement `oid` (as attribute or property) in order to
+ method and specify the `format`.
+ They *may* implement `oid` (as attribute or property) in order to
override the oid type oid; if not PostgreSQL will try to infer the type
from the context, but this may fail in some contexts and may require a
cast.
be possible to know the connection encoding or the server date format.
:type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer`
+ .. attribute:: format
+ :type: Format
+
+ The format this class dumps, `~Format.TEXT` or `~Format.BINARY`.
+ This is a class attribute.
+
+
.. automethod:: dump
The format returned by dump shouldn't contain quotes or escaped
Document how to find type OIDs in a database.
- .. automethod:: register(src, context=None, format=Format.TEXT)
+ .. automethod:: register(src, context=None)
You should call this method on the `Dumper` subclass you create,
passing the Python type you want to dump as *src*.
:param context: Where the dumper should be used. If `!None` the dumper
will be used globally.
:type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer`
- :param format: Register the dumper for text or binary adaptation
- :type format: `~psycopg3.pq.Format`
If *src* is specified as string it will be lazy-loaded, so that it
will be possible to register it without importing it before. In this
.. autoclass:: Loader(oid, context=None)
This is an abstract base class: subclasses *must* implement the `load()`
- method.
+ method and specify a `format`.
:param oid: The type that will be managed by this dumper.
:type oid: int
be possible to know the connection encoding or the server date format.
:type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer`
+ .. attribute:: format
+ :type: Format
+
+ The format this class can load, `~Format.TEXT` or `~Format.BINARY`.
+ This is a class attribute.
+
.. automethod:: load
- .. automethod:: register(oid, context=None, format=Format.TEXT)
+ .. automethod:: register(oid, context=None)
You should call this method on the `Loader` subclass you create,
passing the OID of the type you want to load as *oid* parameter.
:param context: Where the loader should be used. If `!None` the loader
will be used globally.
:type context: `~psycopg3.Connection`, `~psycopg3.Cursor`, or `Transformer`
- :param format: Register the loader for text or binary adaptation
- :type format: `~psycopg3.pq.Format`
.. autoclass:: Transformer(context=None)
.. seealso:: :pq:`PQtransactionStatus` for a description of these states.
-.. autoclass:: Format
- :members:
-
-
.. autoclass:: ExecStatus
:members:
self._connection = None
# mapping class, fmt -> Dumper instance
- self._dumpers_cache: Dict[Tuple[type, Format], "Dumper"] = {}
+ self._dumpers_cache: Tuple[Dict[type, "Dumper"], Dict[type, "Dumper"]]
+ self._dumpers_cache = ({}, {})
# mapping oid, fmt -> Loader instance
- self._loaders_cache: Dict[Tuple[int, Format], "Loader"] = {}
+ self._loaders_cache: Tuple[Dict[int, "Loader"], Dict[int, "Loader"]]
+ self._loaders_cache = ({}, {})
# sequence of load functions from value to python
# the length of the result columns
# Fast path: return a Dumper class already instantiated from the same type
cls = type(obj)
try:
- return self._dumpers_cache[cls, format]
+ return self._dumpers_cache[format][cls]
except KeyError:
pass
- # We haven't seen this type in this query yet. Look for an adapter
- # in the current context (which was grown from more generic ones).
- # Also look for superclasses: if you can adapt a type you should be
- # able to adapt its subtypes, otherwise Liskov is sad.
- dmap = self._adapters._dumpers
- for scls in cls.__mro__:
- dumper_class = dmap.get((scls, format))
- if not dumper_class:
- continue
-
- dumper = self._dumpers_cache[cls, format] = dumper_class(cls, self)
- return dumper
-
- # If the adapter is not found, look for its name as a string
- for scls in cls.__mro__:
- fqn = f"{cls.__module__}.{scls.__qualname__}"
- dumper_class = dmap.get((fqn, format))
- if dumper_class is None:
- continue
-
- dmap[cls, format] = dumper_class
- dumper = self._dumpers_cache[cls, format] = dumper_class(cls, self)
- return dumper
+ dumper_class = self._adapters.get_dumper(cls, format)
+ if dumper_class:
+ d = self._dumpers_cache[format][cls] = dumper_class(cls, self)
+ return d
raise e.ProgrammingError(
- f"cannot adapt type {type(obj).__name__}"
- f" to format {Format(format).name}"
+ f"cannot adapt type {cls.__name__} to format {Format(format).name}"
)
def load_rows(self, row0: int, row1: int) -> Sequence[Tuple[Any, ...]]:
)
def get_loader(self, oid: int, format: Format) -> "Loader":
- key = (oid, format)
try:
- return self._loaders_cache[key]
+ return self._loaders_cache[format][oid]
except KeyError:
pass
- loader_cls = self._adapters._loaders.get(key)
+ loader_cls = self._adapters.get_loader(oid, format)
if not loader_cls:
- loader_cls = self._adapters._loaders[INVALID_OID, format]
- loader = self._loaders_cache[key] = loader_cls(oid, self)
+ loader_cls = self._adapters.get_loader(INVALID_OID, format)
+ if not loader_cls:
+ raise e.InterfaceError("unknown oid loader not found")
+ loader = self._loaders_cache[format][oid] = loader_cls(oid, self)
return loader
# Copyright (C) 2020 The Psycopg Team
from abc import ABC, abstractmethod
-from typing import Any, Callable, Optional, Type, TYPE_CHECKING, Union
-
+from typing import Any, Dict, Callable, List, Optional, Type, Union
+from typing import TYPE_CHECKING
from . import pq
from . import proto
from .pq import Format as Format
from .oids import builtins, TEXT_OID
-from .proto import DumpersMap, DumperType, LoadersMap, LoaderType, AdaptContext
+from .proto import AdaptContext
if TYPE_CHECKING:
from .connection import BaseConnection
class Dumper(ABC):
"""
- Convert Python object of the type *src* to PostgreSQL representation.
+ Convert Python object of the type *cls* to PostgreSQL representation.
"""
format: Format
# the subclass overrides it in init.
_oid: int = 0
- def __init__(self, src: type, context: Optional[AdaptContext] = None):
- self.src = src
+ def __init__(self, cls: type, context: Optional[AdaptContext] = None):
+ self.cls = cls
self.connection = context.connection if context else None
self.oid = self._oid
@classmethod
def register(
- cls, src: Union[type, str], context: Optional[AdaptContext] = None
+ this_cls, cls: Union[type, str], context: Optional[AdaptContext] = None
) -> None:
"""
- Configure *context* to use this dumper to convert object of type *src*.
+ Configure *context* to use this dumper to convert object of type *cls*.
"""
adapters = context.adapters if context else global_adapters
- adapters.register_dumper(src, cls)
+ adapters.register_dumper(cls, this_cls)
@classmethod
def builtin(
cls, *types: Union[type, str]
- ) -> Callable[[DumperType], DumperType]:
+ ) -> Callable[[Type["Dumper"]], Type["Dumper"]]:
"""
Decorator to mark a dumper class as default for a builtin type.
"""
- def builtin_(dumper: DumperType) -> DumperType:
- for src in types:
- dumper.register(src)
+ def builtin_(dumper: Type["Dumper"]) -> Type["Dumper"]:
+ for cls in types:
+ dumper.register(cls)
return dumper
return builtin_
@classmethod
def builtin(
cls, *types: Union[int, str]
- ) -> Callable[[LoaderType], LoaderType]:
+ ) -> Callable[[Type["Loader"]], Type["Loader"]]:
"""
Decorator to mark a loader class as default for a builtin type.
"""
- def builtin_(loader: LoaderType) -> LoaderType:
- for src in types:
- if isinstance(src, str):
- src = builtins[src].oid
- loader.register(src)
+ def builtin_(loader: Type["Loader"]) -> Type["Loader"]:
+ for cls in types:
+ if isinstance(cls, str):
+ cls = builtins[cls].oid
+ loader.register(cls)
return loader
return builtin_
is cheap: a copy is made only on customisation.
"""
- _dumpers: DumpersMap
- _loaders: LoadersMap
+ _dumpers: List[Dict[Union[type, str], Type["Dumper"]]]
+ _loaders: List[Dict[int, Type["Loader"]]]
def __init__(self, extend: Optional["AdaptersMap"] = None):
if extend:
- self._dumpers = extend._dumpers
- self._own_dumpers = False
- self._loaders = extend._loaders
- self._own_loaders = False
+ self._dumpers = extend._dumpers[:]
+ self._own_dumpers = [False, False]
+ self._loaders = extend._loaders[:]
+ self._own_loaders = [False, False]
else:
- self._dumpers = {}
- self._own_dumpers = True
- self._loaders = {}
- self._own_loaders = True
+ self._dumpers = [{}, {}]
+ self._own_dumpers = [True, True]
+ self._loaders = [{}, {}]
+ self._own_loaders = [True, True]
def register_dumper(
- self, src: Union[type, str], dumper: Type[Dumper]
+ self, cls: Union[type, str], dumper: Type[Dumper]
) -> None:
"""
- Configure the context to use *dumper* to convert object of type *src*.
+ Configure the context to use *dumper* to convert object of type *cls*.
"""
- if not isinstance(src, (str, type)):
+ if not isinstance(cls, (str, type)):
raise TypeError(
- f"dumpers should be registered on classes, got {src} instead"
+ f"dumpers should be registered on classes, got {cls} instead"
)
- if not self._own_dumpers:
- self._dumpers = self._dumpers.copy()
- self._own_dumpers = True
+ fmt = dumper.format
+ if not self._own_dumpers[fmt]:
+ self._dumpers[fmt] = self._dumpers[fmt].copy()
+ self._own_dumpers[fmt] = True
- self._dumpers[src, dumper.format] = dumper
+ self._dumpers[fmt][cls] = dumper
def register_loader(self, oid: int, loader: Type[Loader]) -> None:
"""
f"loaders should be registered on oid, got {oid} instead"
)
- if not self._own_loaders:
- self._loaders = self._loaders.copy()
- self._own_loaders = True
+ fmt = loader.format
+ if not self._own_loaders[fmt]:
+ self._loaders[fmt] = self._loaders[fmt].copy()
+ self._own_loaders[fmt] = True
+
+ self._loaders[fmt][oid] = loader
+
+ def get_dumper(self, cls: type, format: Format) -> Optional[Type[Dumper]]:
+ """
+ Return the dumper class for the given type and format.
+
+ Return None if not found.
+ """
+ dumpers = self._dumpers[format]
+
+ # Look for the right class, including looking at superclasses
+ for scls in cls.__mro__:
+ if scls in dumpers:
+ return dumpers[scls]
+
+ # If the adapter is not found, look for its name as a string
+ for scls in cls.__mro__:
+ fqn = scls.__module__ + "." + scls.__qualname__
+ if fqn in dumpers:
+ # Replace the class name with the class itself
+ d = dumpers[scls] = dumpers.pop(fqn)
+ return d
+
+ return None
- self._loaders[oid, loader.format] = loader
+ def get_loader(self, oid: int, format: Format) -> Optional[Type[Loader]]:
+ """
+ Return the loader class for the given oid and format.
+
+ Return None if not found.
+ """
+ return self._loaders[format].get(oid)
global_adapters = AdaptersMap()
class Format(IntEnum):
"""
- The format of a query argument or return value.
+ Enum representing the format of a query argument or return value.
"""
- __module__ = "psycopg3.pq"
+ __module__ = "psycopg3.adapt"
TEXT = 0
"""Text parameter."""
# Copyright (C) 2020 The Psycopg Team
-from typing import Any, Callable, Dict, Generator, Mapping
-from typing import Optional, Sequence, Tuple, Type, TypeVar, Union
+from typing import Any, Callable, Generator, Mapping
+from typing import Optional, Sequence, Tuple, TypeVar, Union
from typing import TYPE_CHECKING
from typing_extensions import Protocol
# Adaptation types
DumpFunc = Callable[[Any], bytes]
-DumperType = Type["Dumper"]
-DumpersMap = Dict[Tuple[Union[type, str], Format], DumperType]
-
LoadFunc = Callable[[bytes], Any]
-LoaderType = Type["Loader"]
-LoadersMap = Dict[Tuple[int, Format], LoaderType]
# TODO: Loader, Dumper should probably become protocols
# as there are both C and a Python implementation
_oid = TEXT_ARRAY_OID
- def __init__(self, src: type, context: Optional[AdaptContext] = None):
- super().__init__(src, context)
+ def __init__(self, cls: type, context: Optional[AdaptContext] = None):
+ super().__init__(cls, context)
self._tx = Transformer(context)
def _get_array_oid(self, base_oid: int) -> int:
oid = 0
def calc_dims(L: List[Any]) -> None:
- if isinstance(L, self.src):
+ if isinstance(L, self.cls):
if not L:
raise e.DataError("lists cannot contain empty lists")
dims.append(len(L))
data.append(b"\xff\xff\xff\xff")
else:
for item in L:
- if not isinstance(item, self.src):
+ if not isinstance(item, self.cls):
raise e.DataError(
"nested lists have inconsistent depths"
)
cdef readonly object connection
cdef readonly object adapters
- # TODO: the dumpers should have the same tweaking of the loaders
- cdef dict _adapters_loaders
- cdef dict _adapters_dumpers
- cdef dict _dumpers_cache
+ cdef dict _text_dumpers
+ cdef dict _binary_dumpers
cdef dict _text_loaders
cdef dict _binary_loaders
cdef pq.PGresult _pgresult
self.adapters = global_adapters
self.connection = None
- self._adapters_loaders = self.adapters._loaders
- self._adapters_dumpers = self.adapters._dumpers
-
- # mapping class, fmt -> Dumper instance
- self._dumpers_cache: Dict[Tuple[type, Format], "Dumper"] = {}
+ # mapping class -> Dumper instance (text, binary)
+ self._text_dumpers = {}
+ self._binary_dumpers = {}
# mapping oid -> Loader instance (text, binary)
self._text_loaders = {}
def get_dumper(self, obj: Any, format: Format) -> "Dumper":
# Fast path: return a Dumper class already instantiated from the same type
+ cdef dict cache
+ cdef PyObject *ptr
+
cls = type(obj)
- key = (cls, format)
- cdef PyObject *ptr = PyDict_GetItem(self._dumpers_cache, key)
+ cache = self._binary_dumpers if format else self._text_dumpers
+ ptr = PyDict_GetItem(cache, cls)
if ptr != NULL:
return <object>ptr
- # We haven't seen this type in this query yet. Look for an adapter
- # in the current context (which was grown from more generic ones).
- # Also look for superclasses: if you can adapt a type you should be
- # able to adapt its subtypes, otherwise Liskov is sad.
- cdef dict dmap = self.adapters._dumpers
- for scls in cls.__mro__:
- dumper_class = dmap.get((scls, format))
- if not dumper_class:
- continue
-
- self._dumpers_cache[key] = dumper = dumper_class(cls, self)
- return dumper
-
- # If the adapter is not found, look for its name as a string
- for scls in cls.__mro__:
- fqn = f"{cls.__module__}.{scls.__qualname__}"
- dumper_class = dmap.get((fqn, format))
- if dumper_class is None:
- continue
-
- dmap[key] = dumper_class
- self._dumpers_cache[key] = dumper = dumper_class(cls, self)
- return dumper
+ dumper_class = self.adapters.get_dumper(cls, format)
+ if dumper_class:
+ d = dumper_class(cls, self)
+ cache[cls] = d
+ return d
raise e.ProgrammingError(
- f"cannot adapt type {type(obj).__name__}"
- f" to format {Format(format).name}"
+ f"cannot adapt type {cls.__name__} to format {Format(format).name}"
)
def load_rows(self, int row0, int row1) -> Sequence[Tuple[Any, ...]]:
cdef dict cache
cdef PyObject *ptr
- if format == 0:
- cache = self._text_loaders
- else:
- cache = self._binary_loaders
-
+ cache = self._binary_loaders if format else self._text_loaders
ptr = PyDict_GetItem(cache, oid)
if ptr != NULL:
return <object>ptr
- key = (oid, format)
- ptr = PyDict_GetItem(self._adapters_loaders, (oid, format))
- if ptr != NULL:
- loader_cls = <object>ptr
- else:
- ptr = PyDict_GetItem(
- self.adapters._loaders, (oids.INVALID_OID, format))
- if ptr == NULL:
- raise e.InterfaceError(f"unknown oid loader not found")
- loader_cls = <object>ptr
+ loader_cls = self.adapters.get_loader(oid, format)
+ if not loader_cls:
+ loader_cls = self.adapters.get_loader(oids.INVALID_OID, format)
+ if not loader_cls:
+ raise e.InterfaceError("unknown oid loader not found")
+
loader = loader_cls(oid, self)
PyDict_SetItem(cache, oid, loader)
return loader
assert isinstance(res[0].baz, float)
-def test_register_scope(conn):
+def test_register_scope(conn, testcomp):
info = CompositeInfo.fetch(conn, "testcomp")
info.register()
for fmt in (Format.TEXT, Format.BINARY):
for oid in (info.oid, info.array_oid):
- assert global_adapters._loaders.pop((oid, fmt))
+ assert global_adapters._loaders[fmt].pop(oid)
cur = conn.cursor()
info.register(cur)
for fmt in (Format.TEXT, Format.BINARY):
for oid in (info.oid, info.array_oid):
- key = oid, fmt
- assert key not in global_adapters._loaders
- assert key not in conn.adapters._loaders
- assert key in cur.adapters._loaders
+ assert oid not in global_adapters._loaders[fmt]
+ assert oid not in conn.adapters._loaders[fmt]
+ assert oid in cur.adapters._loaders[fmt]
info.register(conn)
for fmt in (Format.TEXT, Format.BINARY):
for oid in (info.oid, info.array_oid):
- key = oid, fmt
- assert key not in global_adapters._loaders
- assert key in conn.adapters._loaders
+ assert oid not in global_adapters._loaders[fmt]
+ assert oid in conn.adapters._loaders[fmt]
projects / thirdparty / psycopg.git / commitdiff
