]>
git.ipfire.org Git - thirdparty/psycopg.git/log
Daniele Varrazzo [Sun, 18 Dec 2022 19:14:16 +0000 (19:14 +0000)]
test: drop --allow-fail and --no-collect-ok options
They were needed to deal with problem caused by tox. Not needed anymore.
Daniele Varrazzo [Sun, 18 Dec 2022 15:23:26 +0000 (15:23 +0000)]
ci: enable subprocess-based tests in ci
Daniele Varrazzo [Sun, 18 Dec 2022 02:54:08 +0000 (02:54 +0000)]
test: skip test on windows, C implementation, in ci
No idea why ctypes doesn't work, and I really don't care.
Daniele Varrazzo [Thu, 8 Dec 2022 14:44:36 +0000 (14:44 +0000)]
ci: drop tox from tests
Daniele Varrazzo [Fri, 16 Dec 2022 15:29:48 +0000 (15:29 +0000)]
Merge branch 'fix-438'
Daniele Varrazzo [Fri, 16 Dec 2022 03:21:38 +0000 (03:21 +0000)]
fix(pool): make sure that check() fills an empty pool
Close #438.
Daniele Varrazzo [Fri, 16 Dec 2022 15:29:21 +0000 (15:29 +0000)]
Merge branch 'test-crdb-v22.2'
Daniele Varrazzo [Fri, 16 Dec 2022 03:32:42 +0000 (03:32 +0000)]
test(crdb): re-introduce a mix of Python and libpq versions
Rafi Shamim [Mon, 12 Dec 2022 21:47:33 +0000 (16:47 -0500)]
Run tests against CockroachDB v22.2
v22.2 was released recently, so we can test it. This upgrade revealed a
regression in CRDB that we must workaround
(https://github.com/cockroachdb/cockroach/issues/93739).
This also removes testing for v21.2, since it is no longer supported.
Daniele Varrazzo [Thu, 15 Dec 2022 15:54:25 +0000 (15:54 +0000)]
ops: add script to bump package version
Daniele Varrazzo [Thu, 15 Dec 2022 15:49:31 +0000 (15:49 +0000)]
test: skip a flakey test on release
For 3.1.6 release, three package jobs failed on this test. One of them,
a ppc manylinux, took more than 30 minutes to rerun. Not pleasurable.
Daniele Varrazzo [Thu, 15 Dec 2022 15:36:20 +0000 (15:36 +0000)]
chore: bump psycopg package version to 3.1.7.dev1
Daniele Varrazzo [Thu, 15 Dec 2022 11:12:37 +0000 (11:12 +0000)]
chore: bump version number for 3.1.6 release
Daniele Varrazzo [Thu, 15 Dec 2022 11:11:22 +0000 (11:11 +0000)]
Merge branch 'fix-460'
Daniele Varrazzo [Thu, 15 Dec 2022 11:03:44 +0000 (11:03 +0000)]
fix(copy): don't create a row maker on copy
A COPY_OUT result has columns, but no names for the columns. This case
must be handled in cur.description (see #235) but we don't need to
handle it in copy. If we did handle it in copy, we would need a column
name fallback, which we forgot to handle, hence the problem in #460.
Close #460.
Daniele Varrazzo [Thu, 15 Dec 2022 09:39:59 +0000 (09:39 +0000)]
chore(binary): upgrade packaged libraries to libpq 15.1, openssl 1.1.1s
Daniele Varrazzo [Thu, 15 Dec 2022 09:42:17 +0000 (09:42 +0000)]
chore: bump to dev version 3.1.6.dev0
Daniele Varrazzo [Tue, 13 Dec 2022 14:23:56 +0000 (14:23 +0000)]
ci: run tests on Python 3.11 on macOS
Daniele Varrazzo [Tue, 13 Dec 2022 14:20:37 +0000 (14:20 +0000)]
chore: bump version number for 3.1.5 release
Daniele Varrazzo [Tue, 13 Dec 2022 12:49:23 +0000 (12:49 +0000)]
perf(rows): reduce row_factories overhead
Drop the call to cur.description, which is very heavy. Use the low-level
PGresult to get the fields names.
Close #457.
Daniele Varrazzo [Tue, 13 Dec 2022 05:19:05 +0000 (05:19 +0000)]
perf: drop call to pgconn_encoding per query
Cursor._encoding converted to property. It is only used in error
messages and infrequent operations.
Use uniformly the fallback value of utf-8 in *_encoding() functions,
especially when the connection is closed.
Daniele Varrazzo [Tue, 13 Dec 2022 04:04:04 +0000 (04:04 +0000)]
test: fix resource leaking in tests
It results in spurious warnings on ``gc_collect()`` in some innocent test
downstream::
ResourceWarning: unclosed <socket.socket fd=14,
family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>
Daniele Varrazzo [Tue, 13 Dec 2022 03:05:39 +0000 (03:05 +0000)]
test: ignore error looking for leaks
The leaks counter frequently reports a leak between run 1 and 2 of the
tests. It turns out to be a Decimal Context whose items() method reports
no content. So the problem is related to Python, not psycopg, and it is
not an unbound leak.
Can't reproduce it in CI, but it happens on my laptop, since switching
to Ubuntu 22.04 with Python 3.10.
Daniele Varrazzo [Tue, 13 Dec 2022 04:25:46 +0000 (04:25 +0000)]
fix: don't re-export TypeAlias from _compat module
Pyright special-cases that type and requires to import it from typing or
typing_extensions only (see https://github.com/microsoft/pyright/issues/4197).
Drop conditional dependency on typing_extension and always import
TypeAlias from there.
Fix #439.
Daniele Varrazzo [Mon, 12 Dec 2022 14:41:54 +0000 (14:41 +0000)]
docs: fix ticket number in news file
Daniele Varrazzo [Mon, 12 Dec 2022 14:40:52 +0000 (14:40 +0000)]
Merge branch 'wait-select'
Daniele Varrazzo [Mon, 12 Dec 2022 14:39:48 +0000 (14:39 +0000)]
docs: report select speedup in news file
Close #411.
Daniele Varrazzo [Sun, 11 Dec 2022 20:25:12 +0000 (20:25 +0000)]
perf(c): add fast-call Cython function to PQflush wrapper
Daniele Varrazzo [Sun, 11 Dec 2022 19:48:32 +0000 (19:48 +0000)]
perf(c): small optimization tweaks to wait_c
Daniele Varrazzo [Tue, 1 Nov 2022 23:38:50 +0000 (00:38 +0100)]
test: skip high fd number test on Windows
Windows is pretty simple in regards of the wait function, because it only
implement select, and it doesn't have the same limit of the Unix select.
However this test sometimes fails in weird ways: on the pgconn.socket >
1024 assert, for instance. So, let's just skip it.
Daniele Varrazzo [Tue, 1 Nov 2022 22:39:22 +0000 (23:39 +0100)]
test: allow choosing a wait function using the PSYCOPG_WAIT_FUNC env var
We won't make this public yet, possibly we never will (I think we are
the people responsible for this choice), but it might be useful for
testing.
Daniele Varrazzo [Sat, 29 Oct 2022 23:37:01 +0000 (01:37 +0200)]
perf(c): use poll() instead of select() for waiting, where available
Daniele Varrazzo [Sat, 29 Oct 2022 10:27:10 +0000 (12:27 +0200)]
fix(c): guard select() from high number file descriptors
The Python `select.select()` wrapper raises ValueError. Without a guard,
the C `wait_select()` segfaults. The C way to say you have been naughty.
On Windows this seems not only not needed, but it actually stops from creating
any connection at all.
Daniele Varrazzo [Wed, 26 Oct 2022 01:02:14 +0000 (03:02 +0200)]
fix(c): allow wait_select() to be interrupted by signals
Daniele Varrazzo [Tue, 25 Oct 2022 21:14:58 +0000 (23:14 +0200)]
fix(c): fix import on Windows
Daniele Varrazzo [Tue, 25 Oct 2022 20:31:11 +0000 (22:31 +0200)]
perf(c): add C version of select waiting function
'Wait' and 'Ready' enums moved into the '_enums' module to avoid a
circular import.
Daniele Varrazzo [Tue, 25 Oct 2022 16:13:15 +0000 (18:13 +0200)]
perf: use wait_select() as even if kqueue is the default
There is a bit less overhead using this simpler function than using
wait_selector.
Daniele Varrazzo [Wed, 19 Oct 2022 16:33:07 +0000 (17:33 +0100)]
test: add test on wait_select
Refactor tests to make a better use of pytest parametrization and skip
conditions.
Daniele Varrazzo [Wed, 19 Oct 2022 15:38:20 +0000 (16:38 +0100)]
perf: use wait_select on Windows
On windows, the default selector strategy is select. As a consequence
using the higher level selector object comes with a performance penalty.
Daniele Varrazzo [Mon, 17 Oct 2022 02:12:24 +0000 (03:12 +0100)]
feat: add wait_select() function
A few tests show that using select() from Python has a lower overhead.
See #414.
Use this function wherever defined and epoll() is the default select
strategy instead. This is mostly a draft to figure out where this
strategy can be used with success.
Daniele Varrazzo [Mon, 17 Oct 2022 09:42:33 +0000 (10:42 +0100)]
test: add simple test to compare performances in refactoring
This test is inspired to the problem open in #411, and is pretty much a
simple operation in a tight loop. It is used, at the moment, to compare
different ways to test I/O completion strategies with simple and fast
queries under no concurrency.
Daniele Varrazzo [Wed, 7 Dec 2022 20:33:46 +0000 (20:33 +0000)]
chore: declare Copy.read to return a buffer object
We still return memoryview, but leave the possibility open to return
something different, such as bytes.
Daniele Varrazzo [Thu, 8 Dec 2022 14:11:00 +0000 (14:11 +0000)]
fix: add C Float4 dumper
It would be used when looked up by oid. At the moment there is only a
Python version for it.
Daniele Varrazzo [Thu, 8 Dec 2022 16:20:04 +0000 (16:20 +0000)]
lint(c): avoid types warning building on Windows
Daniele Varrazzo [Sat, 10 Dec 2022 13:08:09 +0000 (13:08 +0000)]
Merge branch 'array-speedup'
Daniele Varrazzo [Tue, 6 Dec 2022 12:40:27 +0000 (13:40 +0100)]
docs: add a note about further optimizing text array loading
Daniele Varrazzo [Tue, 6 Dec 2022 12:23:15 +0000 (13:23 +0100)]
perf(c/array): cache the scratch area in the dumper
Daniele Varrazzo [Tue, 6 Dec 2022 11:50:03 +0000 (12:50 +0100)]
perf(c/array): cache the element loader in the array loader
Daniele Varrazzo [Tue, 6 Dec 2022 07:03:58 +0000 (08:03 +0100)]
docs: report array speedup in the news
Daniele Varrazzo [Sun, 27 Nov 2022 14:20:10 +0000 (15:20 +0100)]
perf(c/array): add C implementation of array loader
With this change we get to load an array without the transformer calling
any Python code (if the element loader is a CLoader as well).
We don't need to subclass the binary loader anymore because all the
required info is now taken from the loaded data. As a consequence, we
don't need a BaseArrayLoader anymore.
Daniele Varrazzo [Sun, 27 Nov 2022 11:12:50 +0000 (12:12 +0100)]
perf(c/array): reuse buffer to unescape array tokens
Save a malloc and setting up an exception per element, replaced by
realloc when needed and a single exception per array.
Daniele Varrazzo [Fri, 25 Nov 2022 14:48:05 +0000 (15:48 +0100)]
fix(c/array): null-terminate strings in text array loading
Daniele Varrazzo [Wed, 9 Nov 2022 14:52:55 +0000 (14:52 +0000)]
perf(c): optimise access to C item loaders in array loading
Daniele Varrazzo [Fri, 21 Oct 2022 15:44:17 +0000 (17:44 +0200)]
perf(c/array): build binary array recursively instead of rearranging them
Daniele Varrazzo [Thu, 20 Oct 2022 23:13:45 +0000 (01:13 +0200)]
perf(c/array): optimize reassembling multidimensional arrays as nested lists
Daniele Varrazzo [Thu, 20 Oct 2022 21:58:08 +0000 (23:58 +0200)]
refactor(c/array): types hygiene around endianness conversions
Daniele Varrazzo [Thu, 20 Oct 2022 21:41:03 +0000 (23:41 +0200)]
fix: get element oid from the array itself in binary format
Getting it from the type was causing problems in CRDB because they seem
to have a different interpretation of what Int2 and Int4 are. Refactor
the binary parser to take a transformer, instead of a loader function,
in order to look up the loader from the oid parsed from the array.
Also refactor the text array parser to take the whole loader instead of
the load function only, which will be useful in order to optimize the
use of C loaders.
Daniele Varrazzo [Wed, 14 Sep 2022 11:33:00 +0000 (12:33 +0100)]
perf: add C implementation of array load
This implementation can be still improved: we always call the Python
loader to decode the single elements; we can try and call the C loader
if available.
Daniele Varrazzo [Sat, 10 Sep 2022 17:53:20 +0000 (18:53 +0100)]
perf(array): introduce mockup of optimized array loading functions
Daniele Varrazzo [Wed, 31 Aug 2022 09:40:44 +0000 (10:40 +0100)]
fix: add prod() implementation for Python 3.7
Daniele Varrazzo [Tue, 30 Aug 2022 11:34:27 +0000 (12:34 +0100)]
perf(array) faster algorithm to load nested binary arrays
Avoid using generators in the algorithm. This also makes the function
easier to port to C.
Daniele Varrazzo [Tue, 30 Aug 2022 09:36:46 +0000 (10:36 +0100)]
refactor(array): make load function static
This will allow replacing them with a fast version as with the helper
functions in the copy module.
Daniele Varrazzo [Sat, 10 Dec 2022 13:01:23 +0000 (13:01 +0000)]
Merge branch 'fix-452'
Daniele Varrazzo [Thu, 8 Dec 2022 14:35:55 +0000 (14:35 +0000)]
fix: add lookup of dumpers by varchar, name oid
Fix #452
Daniele Varrazzo [Thu, 8 Dec 2022 14:00:37 +0000 (14:00 +0000)]
test: add test to verify missing oid lookups in string formats
Daniele Varrazzo [Wed, 7 Dec 2022 22:28:08 +0000 (22:28 +0000)]
docs: drop tox from readme
I don't really use in for development, so it's better not suggesting it.
Daniele Varrazzo [Wed, 7 Dec 2022 22:14:20 +0000 (22:14 +0000)]
ci: force the use of tox 3
Work around https://github.com/tox-dev/tox/issues/2619
Daniele Varrazzo [Thu, 1 Dec 2022 12:45:01 +0000 (13:45 +0100)]
ci: fix tests failing because of missing repository public key
Fix the error appearing in CI reporting
The following signatures couldn't be verified because the public key
is not available: NO_PUBKEY
7FCC7D46ACCC4CF8
on the jobs using libpq: latest.
Daniele Varrazzo [Wed, 9 Nov 2022 15:25:42 +0000 (15:25 +0000)]
fix: return rownumber=None if the result has no row to fetch
Close #437.
Daniele Varrazzo [Thu, 3 Nov 2022 23:16:10 +0000 (00:16 +0100)]
chore: bump mypy to 0.990
Daniele Varrazzo [Tue, 8 Nov 2022 12:13:47 +0000 (13:13 +0100)]
chore: fix version number, accidentally bumped back
Daniele Varrazzo [Fri, 4 Nov 2022 22:16:24 +0000 (23:16 +0100)]
chore(pool): bump to next dev version number
Daniele Varrazzo [Fri, 4 Nov 2022 16:33:12 +0000 (17:33 +0100)]
chore; bump pool version number for release
Daniele Varrazzo [Fri, 4 Nov 2022 16:32:04 +0000 (17:32 +0100)]
Merge branch 'fix-219'
Daniele Varrazzo [Thu, 3 Nov 2022 19:19:03 +0000 (20:19 +0100)]
docs(pool): add entry about issue #219 resolved
Daniele Varrazzo [Thu, 3 Nov 2022 17:15:07 +0000 (18:15 +0100)]
fix(pool): make sure to throw a RuntimeError opening async pool without a loop
The error would be thrown anyway downstream, when tasks are created, but
other spurious warning might be raised too because certain tasks are not
awaited.
Daniele Varrazzo [Thu, 3 Nov 2022 16:42:58 +0000 (17:42 +0100)]
fix(pool): create asyncio objects on open rather than on init
If created on init, they get attached to the wrong loop and fail to
work.
Close #219
Daniele Varrazzo [Wed, 2 Nov 2022 11:01:42 +0000 (12:01 +0100)]
perf(c): take more care about the GIL around potentially blocking functions
Daniele Varrazzo [Wed, 2 Nov 2022 11:20:53 +0000 (12:20 +0100)]
refactor: move common notifies processing code into a common function
Daniele Varrazzo [Wed, 2 Nov 2022 10:21:21 +0000 (11:21 +0100)]
fix(c): drop duplicate PQconsumeInput call
Also add a couple of C types in the function for better C code.
Daniele Varrazzo [Fri, 4 Nov 2022 15:47:08 +0000 (16:47 +0100)]
fix: use a correct recursive definition for DumperKey
Already supported in current Mypy: https://github.com/python/mypy/issues/731.
The definition of DumperKey was wrong anyway, in a way causing mypy to
crash: see https://github.com/python/mypy/issues/14000.
Daniele Varrazzo [Fri, 4 Nov 2022 01:12:01 +0000 (02:12 +0100)]
lint: fix type hints with disabled bytearray/memoryview/bytes equivalience
This changeset makes the code compatible with the current Mypy 0.981,
but passes most of the checks that Mypy 0.990 enforces if the byte
strings equivalence is disabled.
Daniele Varrazzo [Thu, 3 Nov 2022 23:53:20 +0000 (00:53 +0100)]
fix: define Escaping method as returning bytes
This change is backward compatible because bytes has a superset of the
memoryview interface and, in Python, to make pretty much any use of it,
it must be converted to bytes beforehand.
Close #422.
Daniele Varrazzo [Thu, 3 Nov 2022 23:09:29 +0000 (00:09 +0100)]
fix: fix check for empty notice handlers list
Issue reported by Mypy 0.990. Thank you!
Daniele Varrazzo [Thu, 3 Nov 2022 19:27:00 +0000 (20:27 +0100)]
docs: use consistenty `!this` to represent parameters and non-linked code
Don't use *this*, which is more of a math use but it disagrees with the
style rendered on the docs, and don't use ``this``, left to literals.
Daniele Varrazzo [Tue, 25 Oct 2022 23:25:34 +0000 (01:25 +0200)]
perf(c): only use __cinit__ in adaptation classes
There is less overhead creating the objects.
Daniele Varrazzo [Wed, 19 Oct 2022 15:55:33 +0000 (16:55 +0100)]
perf: avoid lookups of Wait and Ready enum values
Daniele Varrazzo [Wed, 26 Oct 2022 01:07:28 +0000 (03:07 +0200)]
docs: document how to unbreak editable mode in the readme
Daniele Varrazzo [Thu, 27 Oct 2022 00:08:28 +0000 (02:08 +0200)]
chore: bump to next dev version
Daniele Varrazzo [Wed, 26 Oct 2022 23:59:24 +0000 (01:59 +0200)]
docs: typographic love to the 2to3 page
Daniele Varrazzo [Wed, 26 Oct 2022 23:58:04 +0000 (01:58 +0200)]
docs: add section about returning multiple results
Close #417.
Daniele Varrazzo [Wed, 26 Oct 2022 23:01:10 +0000 (01:01 +0200)]
Merge branch 'build-macos-py311'
Daniele Varrazzo [Wed, 26 Oct 2022 23:00:38 +0000 (01:00 +0200)]
ci: re-enable suspended job after building the missing image
Daniele Varrazzo [Wed, 26 Oct 2022 22:49:04 +0000 (00:49 +0200)]
ci: configure in order to build only Python 3.11 packages for macOS x86_64
The required image was not available at the time of building other
packages. See #424.
Daniele Varrazzo [Tue, 25 Oct 2022 18:02:28 +0000 (20:02 +0200)]
chore: bump psycopg-pool version to 3.1.3
Daniele Varrazzo [Tue, 25 Oct 2022 17:50:20 +0000 (19:50 +0200)]
chore: bump to version number 3.1.4
Daniele Varrazzo [Tue, 25 Oct 2022 14:58:23 +0000 (16:58 +0200)]
chore: upgrade binary dependencies to libpq 15.0, openssl 1.1.1r
Daniele Varrazzo [Tue, 25 Oct 2022 17:43:18 +0000 (19:43 +0200)]
Merge branch 'py311'
Daniele Varrazzo [Tue, 25 Oct 2022 14:55:24 +0000 (16:55 +0200)]
ci: build Python 3.11 macOS arm64 packages
Daniele Varrazzo [Tue, 25 Oct 2022 14:39:06 +0000 (16:39 +0200)]
ci: skip test/build on macOS x86_64 for Python 3.11
Package not available yet. See #424