]>
git.ipfire.org Git - thirdparty/psycopg.git/log
Daniele Varrazzo [Thu, 11 Feb 2021 02:07:03 +0000 (03:07 +0100)]
Add documentation for the server-side cursors
Daniele Varrazzo [Wed, 10 Feb 2021 17:50:03 +0000 (18:50 +0100)]
Make sue ServerCursor.execute() can be used more than once
Daniele Varrazzo [Wed, 10 Feb 2021 17:49:06 +0000 (18:49 +0100)]
Don't clobber error leaving a ServerCursor block with a broken connection
Daniele Varrazzo [Wed, 10 Feb 2021 15:28:05 +0000 (16:28 +0100)]
Add name to the ServerCursor repr()
Daniele Varrazzo [Wed, 10 Feb 2021 14:56:37 +0000 (15:56 +0100)]
NamedCursor renamed to ServerCursor
In psycopg2 they are not a separate class as they are in psycopg3.
They still need a mandatory name, but having a name is not their
defining trait.
Daniele Varrazzo [Wed, 10 Feb 2021 14:30:27 +0000 (15:30 +0100)]
Add test for all the documented NamedCursor interface
Daniele Varrazzo [Wed, 10 Feb 2021 13:41:32 +0000 (14:41 +0100)]
Fix docs after AsyncConnection.cursor() made non-async
Daniele Varrazzo [Wed, 10 Feb 2021 02:31:14 +0000 (03:31 +0100)]
AsyncConnection.cursor() is no more async
Even after implementing server-side cursors there is not really a reason
for it to be.
Daniele Varrazzo [Wed, 10 Feb 2021 02:04:00 +0000 (03:04 +0100)]
Close correctly named cursors in corner cases
Daniele Varrazzo [Wed, 10 Feb 2021 01:47:45 +0000 (02:47 +0100)]
Make sure you can use a named cursor to "steal" a portal
Daniele Varrazzo [Wed, 10 Feb 2021 01:18:51 +0000 (02:18 +0100)]
Add docs for Cursor.scroll()
Daniele Varrazzo [Wed, 10 Feb 2021 01:05:31 +0000 (02:05 +0100)]
Add scrollable tests
Daniele Varrazzo [Wed, 10 Feb 2021 00:49:41 +0000 (01:49 +0100)]
Add named cursor scroll
Daniele Varrazzo [Wed, 10 Feb 2021 00:26:08 +0000 (01:26 +0100)]
Add cursor.scroll()
Daniele Varrazzo [Tue, 9 Feb 2021 22:53:54 +0000 (23:53 +0100)]
Fix rownumber during iteration in named cursors
Daniele Varrazzo [Tue, 9 Feb 2021 20:04:00 +0000 (21:04 +0100)]
Add fetch methods to named cursors
Daniele Varrazzo [Tue, 9 Feb 2021 17:47:41 +0000 (18:47 +0100)]
Add Cursor.rownumber attribute
Daniele Varrazzo [Tue, 9 Feb 2021 14:06:06 +0000 (15:06 +0100)]
Delete the Postgres cursor when closing a named cursor
Daniele Varrazzo [Tue, 9 Feb 2021 03:22:01 +0000 (04:22 +0100)]
Added classes for named cursors
Only execute implemented, with a describe roundtrip to get the
portal description.
Daniele Varrazzo [Tue, 9 Feb 2021 00:52:50 +0000 (01:52 +0100)]
Added pgconn.send_describe_prepared, send_describe_portal
Daniele Varrazzo [Tue, 9 Feb 2021 00:17:58 +0000 (01:17 +0100)]
You don't want a psycoGP, no
Daniele Varrazzo [Mon, 8 Feb 2021 22:56:22 +0000 (23:56 +0100)]
Added set_jsonb_loads/dumps functions to customise JSON adaptation
Dropped dumps function from the Json/Jsonb wrappers.
Added JSON documentation.
Daniele Varrazzo [Mon, 8 Feb 2021 01:42:31 +0000 (02:42 +0100)]
Add some docs on TypeInfo and adaptation docs corrections
Daniele Varrazzo [Mon, 8 Feb 2021 01:37:57 +0000 (02:37 +0100)]
Expose TypeInfo objects from the `psycopg3.types` package
Daniele Varrazzo [Sat, 6 Feb 2021 03:14:43 +0000 (04:14 +0100)]
Register arrays using the TypeInfo object
TypeInfo and subclasses implementations moved together to a dedicated module,
sharing more uniform interface and implementation.
Daniele Varrazzo [Fri, 5 Feb 2021 12:42:14 +0000 (13:42 +0100)]
Allow mypy to run from root dir
Daniele Varrazzo [Fri, 5 Feb 2021 10:52:30 +0000 (11:52 +0100)]
Added @healthchecks, @ferpection, @abegerho to the backers file
Thank you very much :)
Daniele Varrazzo [Fri, 5 Feb 2021 02:45:05 +0000 (03:45 +0100)]
Detect the range oid from the subtype for non-builtins too
Daniele Varrazzo [Fri, 5 Feb 2021 02:13:23 +0000 (03:13 +0100)]
Add AdaptersMap.types map
Use the map from the adapters when it's needed to look up for a type
info.
Probably there are other places where to make use of it.
Daniele Varrazzo [Thu, 4 Feb 2021 03:17:35 +0000 (04:17 +0100)]
Add py.typed marker file to installed package
Daniele Varrazzo [Tue, 26 Jan 2021 13:22:41 +0000 (14:22 +0100)]
Proud to have @yougov backing us!
Daniele Varrazzo [Tue, 26 Jan 2021 13:16:50 +0000 (14:16 +0100)]
Fixed comment
Daniele Varrazzo [Sat, 23 Jan 2021 01:49:14 +0000 (02:49 +0100)]
Fixed dumping of integer ranges
Daniele Varrazzo [Fri, 22 Jan 2021 21:30:08 +0000 (22:30 +0100)]
Dump time and timestamp naive and tz-aware with the respective oids
Using the tz version of the data types is ok enough for most uses,
because Postgres can downgrade the type to naive, but ranges don't have
an implicit cast and the exact type of the range subtype is necessary.
Daniele Varrazzo [Thu, 21 Jan 2021 03:05:33 +0000 (04:05 +0100)]
Have a single Range class, not one per subtype
There are two problems here, so this doesn't work. One, which will be
solved, is to have a registry of types attached to an adaptation
context: a TODO in already several places, so this will be solved.
Before that we don't really have a way to find back e.g. the oid of a
custom range on strings starting from Range('a', 'b').
The second problem, more serious, is that Postgres doesn't cast
int4range <-> int8range and tsrange <-> tstzrange. The latter pair can
be solved with a two steps dumper choosing between tz aware and not. The
first I don't have in mind how to solve it: Given Range(1,2) I wouldn't
know if int4 or int8 should be used, and Postgres doesn't seem very
forgiving. Probably we should go unknown.
Daniele Varrazzo [Fri, 22 Jan 2021 18:04:14 +0000 (19:04 +0100)]
Added docs for AsyncCursor.stream()
Daniele Varrazzo [Fri, 22 Jan 2021 17:39:52 +0000 (18:39 +0100)]
Mypy version bumped to 0.800
Daniele Varrazzo [Fri, 22 Jan 2021 03:05:12 +0000 (04:05 +0100)]
Added Cursor.stream() to support fetch in single-row mode
Daniele Varrazzo [Thu, 21 Jan 2021 03:04:09 +0000 (04:04 +0100)]
Added a few docstrings on types registry
Daniele Varrazzo [Thu, 21 Jan 2021 03:02:03 +0000 (04:02 +0100)]
Run black on the builtin oids
Daniele Varrazzo [Thu, 21 Jan 2021 02:57:08 +0000 (03:57 +0100)]
Array dumper ligtened up
Don't fetch a string dumper on init, dropped sub_oid attribute.
Daniele Varrazzo [Wed, 20 Jan 2021 17:25:05 +0000 (18:25 +0100)]
Added two-steps dumpers implementation in C too
Daniele Varrazzo [Wed, 20 Jan 2021 12:23:06 +0000 (13:23 +0100)]
Don't require a Transformer roundtrip to choose the specific int dumper
They are static object, they require no customisation with the
connection. I don't expect anyone wanting to subclass them; if they
do they will deal with the whole 5 objects.
Daniele Varrazzo [Wed, 20 Jan 2021 01:56:52 +0000 (02:56 +0100)]
Added test to verify queries failing if int is dumped as int8 or numeric
I wanted these so badly.
Daniele Varrazzo [Wed, 20 Jan 2021 01:46:45 +0000 (02:46 +0100)]
Adapt the dumper used to the value of the objects
Added a second dispatch to allow a dumper to upgrade to a specialised
version. Currently used to dump int to the smallest Postgres type
holding that value and to dump lists of object into typed arrays.
This change allows to write queries more naturally as no ``::int`` cast
should be needed anymore e.g. in date + int or jsonb ->> int.
Only Python implementation; C version to be implemented yet.
Daniele Varrazzo [Tue, 19 Jan 2021 17:29:12 +0000 (18:29 +0100)]
Use prepared statements machinery for executemany
Don't use the anonymous prepared statement anymore, as it doesn't allow
to manage different types.
Daniele Varrazzo [Sun, 17 Jan 2021 06:49:38 +0000 (07:49 +0100)]
Copyright year bumped
egg-info dirs ignored from the root gitignore file to help ag to skpt
them.
Command used:
ag -l copyright | xargs sed -i "s/\(.*copyright[^0-9]*\)\([0-9]\+\)\(-[0-9]\+\)\?\(.*Psycopg.*\)/\1\2-$(date +%Y)\4/I"
Daniele Varrazzo [Sun, 17 Jan 2021 06:19:40 +0000 (07:19 +0100)]
More uniform names for internal objects
Daniele Varrazzo [Sat, 16 Jan 2021 02:51:19 +0000 (03:51 +0100)]
Added tests to detect leaks on copy from
Daniele Varrazzo [Sat, 16 Jan 2021 02:35:20 +0000 (03:35 +0100)]
Added leak test for async cursor
Leak and correctness tests separated.
Daniele Varrazzo [Sat, 16 Jan 2021 02:34:32 +0000 (03:34 +0100)]
Added random tests for data copy
Daniele Varrazzo [Sat, 16 Jan 2021 02:06:56 +0000 (03:06 +0100)]
Declare all loaders to receive an object supporting the buffer interface
There isn't in mypy such an object, so just use `Union[bytes, bytearray,
memoryview]`
Avoid a memory copy passing data from the Transformer to the loaders.
I had started exercising this code path by adding copy tests, but after
dropping the extra copy the same path is now exercised by any select, so
I've stopped doing that.
Daniele Varrazzo [Fri, 15 Jan 2021 23:35:56 +0000 (00:35 +0100)]
Allow passing a list of names to copy.set_types()
Also improved TypeRegistry by recognising a type name even if includes
the array marker '[]' and added get_oid() function.
No test for this object as it's considered internal and leveraged by
several other tests, but I'm sure I will regret it.
Daniele Varrazzo [Fri, 15 Jan 2021 22:23:35 +0000 (23:23 +0100)]
Don't clobber the exception on rollback error in connection exit
Daniele Varrazzo [Fri, 15 Jan 2021 17:26:04 +0000 (18:26 +0100)]
Better integration between C copy functions and C transformer
Daniele Varrazzo [Fri, 15 Jan 2021 17:09:45 +0000 (18:09 +0100)]
Added copy Format helpers to handle text/binary differences
Daniele Varrazzo [Fri, 15 Jan 2021 13:55:29 +0000 (14:55 +0100)]
Added parallel worker to COPY FROM
Have a background writer to push data down the connection while the main
thread is CPU-bound to encode data.
Use a thread to implement the sync worker, a task to implement the
asyncio one.
Daniele Varrazzo [Fri, 15 Jan 2021 01:54:50 +0000 (02:54 +0100)]
End-of-copy logic moved from exit to finish gen
Daniele Varrazzo [Thu, 14 Jan 2021 23:31:21 +0000 (00:31 +0100)]
Added some docs about binary parameters/values.
Daniele Varrazzo [Thu, 14 Jan 2021 22:40:03 +0000 (23:40 +0100)]
Dropped format argument to cursor: use binary= instead
This should make pretty unneeded to import the Format object in client
code.
Daniele Varrazzo [Thu, 14 Jan 2021 21:31:15 +0000 (22:31 +0100)]
Define MAXINT8LEN as a constant
Daniele Varrazzo [Thu, 14 Jan 2021 19:45:00 +0000 (20:45 +0100)]
Dropped intermediate dumpers cache in C transformer too
Daniele Varrazzo [Thu, 14 Jan 2021 17:41:46 +0000 (18:41 +0100)]
Dropped intermediate C Transformer cache creating loaders
Keep in the instance cache the RowLoader instance.
Daniele Varrazzo [Thu, 14 Jan 2021 16:40:05 +0000 (17:40 +0100)]
Less work to create a C Transformer
Daniele Varrazzo [Thu, 14 Jan 2021 16:39:15 +0000 (17:39 +0100)]
Don't create a new transformer if the query is repeated on a cursor
Daniele Varrazzo [Tue, 12 Jan 2021 22:47:39 +0000 (23:47 +0100)]
Added reference leak test for other fetch methods
Daniele Varrazzo [Thu, 14 Jan 2021 15:04:19 +0000 (16:04 +0100)]
Fixed fake string creation: never create \x00 byte
Daniele Varrazzo [Wed, 13 Jan 2021 14:46:49 +0000 (15:46 +0100)]
Choose automatically text or binary format using %s placeholders
A format can be enforced using %b or %t placeholders.
Use binary formats dumpers if available, otherwise use text dumpers.
Python str are an exception: prefer text dumper, because it will dump
with format 0, so Postgres can cast the value to any format.
Special-case empty lists to be always dumped as textual unknown '{}' so
that Postgres can cast them into whatever array it fancies.
Daniele Varrazzo [Wed, 13 Jan 2021 14:28:06 +0000 (15:28 +0100)]
Transformer.dump_sequence returns a list of formats too
Daniele Varrazzo [Wed, 13 Jan 2021 13:55:59 +0000 (14:55 +0100)]
Added xfail to a binary empty array test
Fixing it requires reconsidering how to deal with parameters formats.
Also fixed xpass to make sure to spot the tests to fix after fixing the
above.
Daniele Varrazzo [Wed, 13 Jan 2021 13:38:00 +0000 (14:38 +0100)]
Dropped explicit support for Postgres 9.6
People can still use it, but they should likely disable stored procedure
on a connection and avoid to use executemany, on top of adding plenty of
cast.
Daniele Varrazzo [Wed, 13 Jan 2021 00:02:13 +0000 (01:02 +0100)]
Don't return None as text in Python 9.6
This makes to require a cast on pretty much any placeholder that might
receive a None, which is really asking too much.
Exploratory commit to test what can be done with PG 9.6. The result
shows that in PG 9.6 it's pretty much impossible to use prepared
statements: neither for executemany nor for preparation. Tests failing:
- tests/test_cursor.py::test_executemany_null_first
- tests/test_cursor_async.py::test_executemany_null_first
- tests/test_prepared.py::test_different_types
- tests/test_prepared_async.py::test_different_types
all with: psycopg3.errors.IndeterminateDatatype: could not determine
data type of parameter $x
Daniele Varrazzo [Tue, 12 Jan 2021 22:18:01 +0000 (23:18 +0100)]
Added random data generator and test for memory leaks
Only implemented for binary types, as they are less than text...
Daniele Varrazzo [Tue, 12 Jan 2021 21:47:57 +0000 (22:47 +0100)]
Use cdll instead of pydll in ctypes pq wrapper
cdll releases the GIL. Even for callback functions it doesn't seem a
problem.
Daniele Varrazzo [Tue, 12 Jan 2021 21:47:19 +0000 (22:47 +0100)]
Added [] in the representation of array columns
Daniele Varrazzo [Tue, 12 Jan 2021 21:44:28 +0000 (22:44 +0100)]
Added dumpers row
Speed up repeated dumps on the same query (e.g. in executemany, in copy,
for composite types).
Things are a bit iffy around empty list dumping. As a tradeoff, prefer
that empty list don't behave too differently from non-empty list, but
accept that text and binary work differently (because it doesn't seem
possible to pass an array as unknown in binary, whereas in text '{}'
works ok).
Daniele Varrazzo [Mon, 11 Jan 2021 22:09:19 +0000 (23:09 +0100)]
Set sub-dumper on array dumper on creation in Transformer
Fixes binary dump of empty array and other unforeseen problems such as
the same array dumper dumping lists with two different elements.
Daniele Varrazzo [Mon, 11 Jan 2021 15:04:11 +0000 (16:04 +0100)]
Added test to reproduce array bug found with random data
Daniele Varrazzo [Sun, 10 Jan 2021 18:27:22 +0000 (19:27 +0100)]
Add a reference on the ViewBuffer to the object owning the memory
Daniele Varrazzo [Sun, 10 Jan 2021 04:15:27 +0000 (05:15 +0100)]
Added randomisation of tests run
Fixed a test dealing with an unpredictable libpq behaviour
Daniele Varrazzo [Tue, 12 Jan 2021 15:06:23 +0000 (16:06 +0100)]
Finish correctly a COPY TO operation reading copy.rows()
Close #23
Daniele Varrazzo [Tue, 12 Jan 2021 15:08:17 +0000 (16:08 +0100)]
Added note to remember to document multiple statements + binary
Daniele Varrazzo [Tue, 12 Jan 2021 14:43:12 +0000 (15:43 +0100)]
Reporting pq import error improved
List all the reasons why the implementations failed to import, e.g.
```
Attempts made:
- error importing 'c' wrapper: No module named 'psycopg3_c'
- error importing 'binary' wrapper: No module named 'psycopg3_binary'
- error importing 'python' wrapper: module 'ctypes' has no attribute 'pydll'
```
Daniele Varrazzo [Tue, 12 Jan 2021 01:00:59 +0000 (02:00 +0100)]
Fixed raising unicode encode errors in C
Daniele Varrazzo [Mon, 11 Jan 2021 14:54:31 +0000 (15:54 +0100)]
Added freelist to PGresult
Daniele Varrazzo [Mon, 11 Jan 2021 14:53:26 +0000 (15:53 +0100)]
Added repr and slots to Json wrappers
Daniele Varrazzo [Mon, 11 Jan 2021 14:52:55 +0000 (15:52 +0100)]
Added comment to remember what the array structure is
Daniele Varrazzo [Sun, 10 Jan 2021 19:10:05 +0000 (20:10 +0100)]
Don't use Python enums in C PGconn.status
Leftover from previous cleaning.
Daniele Varrazzo [Tue, 12 Jan 2021 15:09:48 +0000 (16:09 +0100)]
Make cursor.query/params available after closing too
Useful to debug what went wrong with a statement after a cursor context
is exited.
Daniele Varrazzo [Sun, 10 Jan 2021 03:53:24 +0000 (04:53 +0100)]
Avoid local import raising exception in text encoder
Daniele Varrazzo [Sun, 10 Jan 2021 03:46:26 +0000 (04:46 +0100)]
Added C implementation of binary copy load
Return value of the copy read function redefined as memoryview to avoid
unneeded data copying.
Daniele Varrazzo [Sun, 10 Jan 2021 01:56:10 +0000 (02:56 +0100)]
Added C implementation of text copy load
Daniele Varrazzo [Sat, 9 Jan 2021 03:45:08 +0000 (04:45 +0100)]
Added reading row-by-row from Copy
Daniele Varrazzo [Sat, 9 Jan 2021 03:46:09 +0000 (04:46 +0100)]
Added result status in PGresult and Cursor repr
Daniele Varrazzo [Fri, 8 Jan 2021 03:57:22 +0000 (04:57 +0100)]
Further C code streamlining
Daniele Varrazzo [Fri, 8 Jan 2021 02:54:36 +0000 (03:54 +0100)]
Some refcount care in tight loops
Daniele Varrazzo [Fri, 8 Jan 2021 01:57:35 +0000 (02:57 +0100)]
Added test to verify that the C adapters are registered correctly
Daniele Varrazzo [Fri, 8 Jan 2021 01:24:33 +0000 (02:24 +0100)]
Return type of fetch methods improved
Daniele Varrazzo [Fri, 8 Jan 2021 01:03:53 +0000 (02:03 +0100)]
Fixed executemany
D'oh
Daniele Varrazzo [Fri, 8 Jan 2021 00:55:26 +0000 (01:55 +0100)]
Added Transformer.dump_sequence()