The dumper chosen in auto format is the last one registered, if both the
text and binary ones are available. For all the builtin types the
preferred format is binary, except for strings and for json.
Dumping strings as text by default was handled by a special case; now
they are just handled by registering the text dumper after the binary
one. Json is now text by default, because it's marginally more
efficient.
This will make it easier to define custom row factories without having
to use BaseCursor[Any, <rowtype>] as the connection type is not
meaningful in this context.
We expose this name instead of BaseCursor in top-level package
accordingly.
Now `connect()` returns a `Connection[Tuple]`, whereas
`connect(row_factory=something)` return the type of what row factory
produces. The implementation of this is somewhat brittle, but that's
mypy for you: @dlax (thank you!) noticed that defining `**kwargs:
Union[str, int]` helped to disambiguate the row_factory param. I guess
we will make a best-effort to maintain this "interface". Everything is
to be documented.
We use a type variable 'RowConn' for Connection that is distinct from
'Row' that is used on Cursor side in order to reflect the possibility to
have distinct row factories on connection and cursor sides.
In order to avoid "propagation" of the Row type variable of Connection
classes, we use Any everywhere that variable is not used (namely,
everywhere outside the connection module).
The typing_example.py almost works: connect(row_factory=...) returns a
typed Connection, but only connect() still returns a Connection[Any].
Avoid repeating the default row_factory in Connection methods
We set the default row_factory argument value to None in connect()
methods and use row_factory class attribute when unspecified, thus
avoiding repetition of the 'tuple_row' value.
This example is checked by running mypy on it with both Python and C
implementation.
By having cursor and record values' type declared before assignment, we
make sure that expected types are consistent (otherwise mypy would raise
"Incompatible types in assignment" errors).
Example check_row_factory_connection() is still incomplete because
Connection is not generic on Row; it thus contain many Any types.
with conn.cursor(row_factory=my_row_factory) as cur:
cur.execute("select 1")
reveal_type(cur)
# Revealed type is 'psycopg3.cursor.Cursor[R`-1]'
r = cur.fetchone()
reveal_type(r)
# Revealed type is 'Union[R`-1, None]'
The definition of RowMaker and RowFactory protocols needs two distinct
type variable because the former is covariant on Row (using 'Row_co'
type variable) and the latter is invariant on Row.
In Cursor.__init__(), row_factory argument is now required as we remove
its default value 'tuple_row'; this is helpful in order to keep Cursor
definition generic on Row, which would be more difficult when specifying
a concrete RowFactory by default binding Row to Tuple.
The Connection is not (yet) generic on Row, so we use RowFactory[Any].
Still, in cursor() methods, we get a fully typed Cursor value when a
row_factory argument is passed. We add two overloaded variants of these
cursor() methods depending on whether row_factory is passed or not (in
the former case, we return a Cursor[Row], in the latter case, a
Cursor[Any]).
A noticeable improvement is that we no longer need to explicitly declare
or ignore types in Transformer's load_row() and load_rows() as this is
not correctly inferred. Similarly, type annotations are not needed
anymore in callers of these methods (Cursor's fetch*() methods).
In TypeInfo's fetch*() method, we can drop superfluous type annotations.
Move 'make_row' attribute from Transformer to Cursor
Having this attribute defined in Transformer protocol will be
problematic when making the RowMaker protocol generic on Row because
we'll then have to also make Transformer generic on Row and
"propagating" the type variable will produce a lot of churn in the code
base.
Also, the variance (covariant) of Row in RowMaker will conflict with that
in Transformer (invariant).
On the other hand, keeping the RowMaker value and its underlying
RowFactory together/attached to the same object (now Cursor) seems
safer for consistency.
Denis Laxalde [Thu, 25 Feb 2021 07:24:29 +0000 (08:24 +0100)]
Remove constraints on Row TypeVar
It's not clear why there is both Tuple[Any, ...] and Any as possible
types for Row and since Any is one possibility it seems that anything is
possible so those constraints appear to be useless.
We can now remove a few '# type: ignore[no-any-return]' in
*Cursor.fetchone(). On the other hand, mypy wants us to declare the type
of 'records' values coming from transformer's load_rows():
error: Need type annotation for 'records' (hint: "records: List[<type>] = ...") [var-annotated]
Now that we are tapping into ctypes to access to the libpq dynamically
the Python value is handy for the cython layer too (see PGconn.hostaddr
implementation).
Allow reading needs_password, used_password on broken connections
It makes sense (especially the former: it is used to detect if a
password was needed). However they are not that easy to use because a
failed connection attempt will throw an exception. For this reason they
are not added to ConnectionInfo.
Alias Task to asyncio.Future for python version < 3.8
This resolves the following mypy error on Python 3.7:
psycopg3/psycopg3/pool/async_pool.py:72: error: Argument 1 to "append" of "list" has incompatible type "Future[None]"; expected "Task[None]" [arg-type]
Daniele Varrazzo [Thu, 11 Mar 2021 20:11:48 +0000 (21:11 +0100)]
Grow the pool size one connection at time
Not doing so saturates the worker processes which may not be able to do
other maintenance, including returning connections to the pool. It also
helps to limit the amount of growth in a sudden spike.
o