]>
git.ipfire.org Git - thirdparty/psycopg.git/log
Daniele Varrazzo [Tue, 7 Jun 2022 01:52:11 +0000 (03:52 +0200)]
test(crdb): get connection class from fixture to make sure to test CRDB objects
Daniele Varrazzo [Tue, 7 Jun 2022 01:46:51 +0000 (03:46 +0200)]
refactor(crdb): don't mix pg and crdb json configuration
Daniele Varrazzo [Mon, 6 Jun 2022 23:14:59 +0000 (01:14 +0200)]
fix(crdb): base context transaction tests on DML rather than DDL
On CRDB we may incur in problems caused by the table being dropped.
Daniele Varrazzo [Mon, 6 Jun 2022 22:50:34 +0000 (00:50 +0200)]
test(crdb): skip tests with arrays of json[b]
Not supported by CRDB: https://github.com/cockroachdb/cockroach/issues/23468
Daniele Varrazzo [Mon, 6 Jun 2022 22:33:05 +0000 (00:33 +0200)]
test: add json[b] arrays dump/load tests
Also refactor tests a bit to unify the json/jsonb in a parametric
format.
Daniele Varrazzo [Mon, 6 Jun 2022 21:59:10 +0000 (23:59 +0200)]
test(crdb): skip tests based on missing features
With this changeset, all tests pass or are skipped on CRDB 21.2.10 \o/
Daniele Varrazzo [Mon, 6 Jun 2022 21:57:31 +0000 (23:57 +0200)]
test(crdb): fix random tests
- Keep into account the different supported interval range
- Allow for float representation rounding (pre-PG 12 behaviour)
- Run leak tests in transactions to allow to recreate the tables
Daniele Varrazzo [Mon, 6 Jun 2022 21:55:40 +0000 (23:55 +0200)]
test(crdb): skip tests where CRDB seems to misread binary numeric values
Reported upstream at https://github.com/cockroachdb/cockroach/issues/82492
Daniele Varrazzo [Mon, 6 Jun 2022 01:43:01 +0000 (03:43 +0200)]
test(crdb): adapt text tests
With this changeset, all fast tests pass on CRDB.
Daniele Varrazzo [Mon, 6 Jun 2022 01:11:20 +0000 (03:11 +0200)]
fix(crdb): keep into account duplicate oids iterating types in registry
In crdb there are a few aliases (e.g. json -> jsonb oid) which are not
adequately captured by the type's regtype.
Daniele Varrazzo [Mon, 6 Jun 2022 00:04:30 +0000 (02:04 +0200)]
fix(crdb): drop spurious quotes from multi-words regtypes
Daniele Varrazzo [Mon, 6 Jun 2022 00:00:29 +0000 (02:00 +0200)]
test(crdb): adapt numeric tests
Daniele Varrazzo [Sun, 5 Jun 2022 23:10:05 +0000 (01:10 +0200)]
test(crdb): adapt array tests
Daniele Varrazzo [Sun, 5 Jun 2022 21:27:10 +0000 (23:27 +0200)]
fix(crdb): pq tests adapted
Daniele Varrazzo [Sun, 5 Jun 2022 20:42:16 +0000 (22:42 +0200)]
test(crdb): skip tests relying on pg_backend_pid()
Daniele Varrazzo [Sun, 5 Jun 2022 17:30:41 +0000 (19:30 +0200)]
test(crdb): skip transaction tests, deadlocking on CRDB
I guess is it something related to rows locks and rollback. Will ask
upstream.
Daniele Varrazzo [Sun, 5 Jun 2022 13:13:46 +0000 (15:13 +0200)]
refactor: add NoneType to abc module
Daniele Varrazzo [Wed, 25 May 2022 21:12:10 +0000 (22:12 +0100)]
test(crdb): adapt or skip more tests to CRDB
Daniele Varrazzo [Wed, 25 May 2022 08:48:43 +0000 (10:48 +0200)]
fix(crdb): allow None roundtrip
Specify the TEXT oid for None, otherwise dumping with no context will
fail.
Daniele Varrazzo [Wed, 25 May 2022 08:47:53 +0000 (10:47 +0200)]
perf: add Transformer.__slots__
Daniele Varrazzo [Wed, 25 May 2022 08:45:29 +0000 (10:45 +0200)]
feat: allow to customize the oid to dump None
On Postgres >= 10, INVALID_OID is ok, because it has a fallback text
cast. CRDB doesn't have such cast, but can convert implicitly from text
to other types, so TEXT_OID is a better choice.
Daniele Varrazzo [Wed, 25 May 2022 00:03:37 +0000 (02:03 +0200)]
test(crdb): adapt pipeline tests
Transaction-related tests currently fail.
Daniele Varrazzo [Tue, 24 May 2022 23:38:30 +0000 (01:38 +0200)]
test(crdb): adapt cursor and error tests to CRDB
Daniele Varrazzo [Tue, 24 May 2022 23:03:29 +0000 (01:03 +0200)]
test(crdb): fix conninfo tests with crdb
Daniele Varrazzo [Tue, 24 May 2022 22:54:07 +0000 (00:54 +0200)]
fix(crdb): add generic records dumper/loader
Daniele Varrazzo [Tue, 24 May 2022 22:22:46 +0000 (00:22 +0200)]
test(crdb): make prepared statements tests portable between PG and CRDB
Daniele Varrazzo [Tue, 24 May 2022 13:05:13 +0000 (15:05 +0200)]
test: fix skipping versions in parametrized tests
In the previous implementation, all the tests were skipped, because
annotations were added to the function and ended up affecting all
generated tests, not the marked ones only.
Daniele Varrazzo [Tue, 24 May 2022 08:50:57 +0000 (10:50 +0200)]
fix(crdb): fix type info of CrdbConnection.connect()
It requires specifying entirely the connect() signature because of by #308.
Daniele Varrazzo [Tue, 24 May 2022 08:50:17 +0000 (10:50 +0200)]
feat(crdb): add psycopg.crdb.connect() function
Daniele Varrazzo [Mon, 23 May 2022 00:42:15 +0000 (02:42 +0200)]
test(crdb): skip tests for unsupported types
Daniele Varrazzo [Mon, 23 May 2022 00:37:58 +0000 (02:37 +0200)]
feat(crdb): add CrdbConnection class
Drop the automatic detection of crdb. It seems dangerous to change the
program types based on external factors such as the database connected
to.
Generate and use a crdb-specific types oids map, on which to build the
adapters map.
Daniele Varrazzo [Sun, 22 May 2022 00:44:17 +0000 (02:44 +0200)]
fix(crdb): fix json adaptation
The function set_json_dumps() now adapts the currently registered
adapter, not the default one, to be able to customize the Json dumper on
CRDB, without resetting its oid to the PostgreSQL standard one, which
CRDB doesn't know.
Daniele Varrazzo [Sun, 22 May 2022 00:12:01 +0000 (02:12 +0200)]
fix(crdb): configure dumpers to deal correctly with Enum types
Daniele Varrazzo [Sat, 21 May 2022 21:24:37 +0000 (23:24 +0200)]
test(crdb): adapt datetime tests
Daniele Varrazzo [Fri, 20 May 2022 07:53:20 +0000 (09:53 +0200)]
test(crdb): add async copy tests
Daniele Varrazzo [Fri, 20 May 2022 00:32:19 +0000 (02:32 +0200)]
test(crdb): add copy tests
Skip tests from psycopg, because there is no support for copy out and
copy in syntax is different. Data types are different too (int/serial
mean int8) so the binary data samples don't match the format.
Daniele Varrazzo [Wed, 18 May 2022 00:27:02 +0000 (02:27 +0200)]
test(crdb): skip connection tests which cannot run on CockroachDB
Daniele Varrazzo [Tue, 17 May 2022 21:14:16 +0000 (23:14 +0200)]
test(crdb): fix most cursor tests to run with CockroachDB
Move some common test markers to the fix_crdb module.
Remaining cursor failing test may require some tweak:
- something with binary parameters
- something with ::text cast added by crdb dumper
Daniele Varrazzo [Tue, 17 May 2022 19:51:13 +0000 (21:51 +0200)]
test: refactor skipping tests by server version
Daniele Varrazzo [Tue, 17 May 2022 01:27:34 +0000 (03:27 +0200)]
fix(crdb): allow non-normalized encoding name from the database
CockroachDB returns what stored: "utf_8", "utf-8" etc.
All ConnectionInfo tests pass on crdb.
Daniele Varrazzo [Tue, 17 May 2022 00:28:53 +0000 (02:28 +0200)]
feat(crdb): cusomize CockroachDB connection
- Add ConnectionInfo.vendor
- Add ConnectionInfo subclassing
- Add CrdbConnectionInfo subclass with crdb_version attribute
- Add 'psycopg.crdb' module with adapters
- Dump strings using the text oid by default on CockroachDB
The latter change might have wider consequences, however crdb casts
strings to other types more easily than what Postgres does. At a glance
it might work; porting the rest of the test suite will tell.
test_adapt tests ported. Tests showing a difference moved to
crdb/test_adapt.
Daniele Varrazzo [Mon, 16 May 2022 20:49:02 +0000 (22:49 +0200)]
test(crdb): skip TPC tests
Daniele Varrazzo [Mon, 16 May 2022 02:59:04 +0000 (04:59 +0200)]
test(crdb): add fixture to support CockroachDB test/skip
Daniele Varrazzo [Tue, 12 Jul 2022 11:58:17 +0000 (12:58 +0100)]
Merge branch 'codespell-all'
Christian Clauss [Tue, 28 Jun 2022 12:25:03 +0000 (14:25 +0200)]
chore: fix typos on previously missed files
Christian Clauss [Tue, 28 Jun 2022 11:54:42 +0000 (13:54 +0200)]
ci: run codespell on the entire repo
Daniele Varrazzo [Sun, 10 Jul 2022 20:23:14 +0000 (21:23 +0100)]
test: register the pipeline marker consistently with the others
Denis Laxalde [Sun, 3 Jul 2022 19:00:24 +0000 (21:00 +0200)]
docs: fix a typo in literal string examples
Daniele Varrazzo [Sun, 3 Jul 2022 08:35:01 +0000 (09:35 +0100)]
Merge branch 'literal-string'
Daniele Varrazzo [Sun, 3 Jul 2022 02:49:15 +0000 (03:49 +0100)]
docs: add literal strings examples
Move the PEP 675 section to the typing page.
Daniele Varrazzo [Mon, 27 Jun 2022 21:55:57 +0000 (22:55 +0100)]
docs: mention LiteralString and related changes in docs
Daniele Varrazzo [Mon, 27 Jun 2022 21:32:20 +0000 (22:32 +0100)]
fix: clean internal use of LiteralString
Tested one-off using pyre 0.9.13; however it gives too many differences
compared to mypy to use it at the moment.
The pyre run doesn't currently find LiteralString-related problems,
except the todo at psycopg/sql.py:251, because string.Formatter.parse()
doesn't return a LiteralString upon LiteralString input.
Daniele Varrazzo [Mon, 27 Jun 2022 21:09:19 +0000 (22:09 +0100)]
refactor: restrict Query alias to use Composed or SQL instead of Composable
Other Composable subclasses (Identifier, Literal, Placeholder...) don't
make sense as a complete query.
Daniele Varrazzo [Mon, 27 Jun 2022 21:05:54 +0000 (22:05 +0100)]
feat: use LiteralString for Query and SQL aliases
Current mypy version probably doesn't do anything with it at the moment.
Daniele Varrazzo [Sun, 3 Jul 2022 02:51:52 +0000 (03:51 +0100)]
Merge branch 'async-dns'
Daniele Varrazzo [Sun, 3 Jul 2022 01:49:35 +0000 (02:49 +0100)]
test: use the flakey marker in random weight test, instead of a comment
Daniele Varrazzo [Sun, 3 Jul 2022 01:44:41 +0000 (02:44 +0100)]
refactor: drop dnspython implementation of resolve_hostaddr_async
Expose the asyncio function in the _dns module, but mark it deprecated,
because the connection perform async resolution on its own.
Daniele Varrazzo [Sun, 3 Jul 2022 01:19:56 +0000 (02:19 +0100)]
docs: add documentation about async DNS resolution
Close #259.
Daniele Varrazzo [Sun, 3 Jul 2022 01:07:29 +0000 (02:07 +0100)]
test: drop ipaddress lazy load test
The module is now used by psycopg.
Daniele Varrazzo [Sun, 3 Jul 2022 00:25:19 +0000 (01:25 +0100)]
feat: don't block on address resolution in async connections
Use the same algorithm implemented in the `_dns` module, but based on
asyncio getaddrinfo: this avoids the need of the external dns package
and works correctly with the /etc/hosts file.
There were problems of resource leaking in Python 3.6, but as psycopg
3.1 is 3.7+ only, let's go for it!
Daniele Varrazzo [Sat, 2 Jul 2022 23:21:54 +0000 (00:21 +0100)]
fix; drop check for libpq < 10 to use hostaddr
Hostaddr has been always available (seen in 7.2 documentation...).
Daniele Varrazzo [Mon, 23 May 2022 00:46:42 +0000 (02:46 +0200)]
fix: drop "char" - character alias
"char", with quotes, is not an alias for character, bpchar is.
Don't rely on regtype to get its alias.
Daniele Varrazzo [Tue, 14 Jun 2022 23:04:55 +0000 (01:04 +0200)]
chore: bump version to release 3.0.15
Daniele Varrazzo [Tue, 14 Jun 2022 22:56:17 +0000 (00:56 +0200)]
Merge branch 'manylinux2014'
Daniele Varrazzo [Tue, 14 Jun 2022 15:31:52 +0000 (17:31 +0200)]
test: test wheel packages using PostgreSQL 14
Now all built packages support SCRAM authentication.
Daniele Varrazzo [Tue, 14 Jun 2022 14:57:01 +0000 (16:57 +0200)]
test: verify that we distribute the libpq we want to build
Daniele Varrazzo [Mon, 13 Jun 2022 07:12:01 +0000 (09:12 +0200)]
build: cache libpq build
Daniele Varrazzo [Mon, 13 Jun 2022 01:15:31 +0000 (03:15 +0200)]
build: build libpq 14 for musllinux packages
Currently Alpine ships with libpq 12, which wouldn't allow to use the
pipeline mode.
Use the system packages for the other libraries, for the moment.
Daniele Varrazzo [Mon, 13 Jun 2022 00:04:59 +0000 (02:04 +0200)]
build: build manylinux2014 packages
manylinux_2_24 is soon EOL and 2_28 will be incompatible with many
distributions to come. See <https://github.com/pypa/manylinux/issues/1332>.
Close #124
Daniele Varrazzo [Mon, 13 Jun 2022 00:01:57 +0000 (02:01 +0200)]
build: build the libpq from source when building wheels on centos platform
Daniele Varrazzo [Sun, 12 Jun 2022 23:58:11 +0000 (01:58 +0200)]
build: set expected sysconfdir in libpq shipped with the binary package
This affects where the pg_service.conf file is looked for.
See https://github.com/psycopg/psycopg2/issues/1365
Daniele Varrazzo [Sun, 12 Jun 2022 23:26:06 +0000 (01:26 +0200)]
chore: upgrade libraries to build binary packages
Daniele Varrazzo [Sun, 12 Jun 2022 23:25:29 +0000 (01:25 +0200)]
build: add script to build the libpq
Added as it currently is in the psycopg2 project.
Daniele Varrazzo [Sun, 12 Jun 2022 23:01:31 +0000 (01:01 +0200)]
chore: upgrade cibuildwheel to 2.6.1
Daniele Varrazzo [Mon, 13 Jun 2022 04:30:18 +0000 (06:30 +0200)]
refactor: drop unused variable
Daniele Varrazzo [Mon, 13 Jun 2022 04:28:33 +0000 (06:28 +0200)]
test: skip flakey test on package build
Still can't find the reason why this test keeps on failing.
Daniele Varrazzo [Sun, 12 Jun 2022 09:09:42 +0000 (11:09 +0200)]
fix: sync pipeline before rollback()
We were syncing it already, but only if we had received a "pipeline
aborted" already. This might be not the case, as probably the tests
test_errors_raised_on_commit which has been failing in CI randomly for
weeks has been trying to say. Other similar branches (e.g. on commit)
don't have the same check for aborted state.
Denis Laxalde [Thu, 9 Jun 2022 06:45:23 +0000 (08:45 +0200)]
docs: fix a typo in infinity date adaptation section
Daniele Varrazzo [Wed, 8 Jun 2022 23:12:27 +0000 (01:12 +0200)]
Merge branch 'fix-315'
Daniele Varrazzo [Wed, 8 Jun 2022 22:12:20 +0000 (00:12 +0200)]
fix: raise DataError instead of OverflowError loading intervals too large
This happened only for the binary format, not the text format.
Daniele Varrazzo [Wed, 8 Jun 2022 22:11:52 +0000 (00:11 +0200)]
docs: mention different infinity date adaptation in psycopg2 differences pages
See #315
Daniele Varrazzo [Wed, 8 Jun 2022 21:26:41 +0000 (23:26 +0200)]
fix: produce consistent error messages on timestamp overflow
- State if the date is too small or too large, not just "not supported".
- Use similar messages in text and binary format.
See #315
Daniele Varrazzo [Wed, 8 Jun 2022 16:30:13 +0000 (18:30 +0200)]
fix: produce consistent error messages on date overflow
- State if the date is too small or too large, not just "not supported".
- Use similar messages in text and binary format.
- Avoid an overflow error with the infinity date in Python.
See #315
Daniele Varrazzo [Wed, 8 Jun 2022 16:29:35 +0000 (18:29 +0200)]
test: move date-related tests to the date tests class
Daniele Varrazzo [Wed, 8 Jun 2022 15:28:43 +0000 (17:28 +0200)]
refactor: rename the no_pqexec param to force_extended
This refers to using the extended query protocol even in cases where the
singe query protocol might have been chosen instead (e.g. because we
don't have parameters to send).
The negative in the param name is confusing, and the reference to the
PQexec function is wrong, because we never use that function (which is
blocking), but rather we choose PQsendQueryParams instead of
PQsendQuery.
Daniele Varrazzo [Wed, 8 Jun 2022 14:55:31 +0000 (16:55 +0200)]
Merge branch 'message-0x33'
Daniele Varrazzo [Wed, 8 Jun 2022 14:25:28 +0000 (16:25 +0200)]
fix: don't use PQsendQuery in pipeline mode
There is no reason to use it: the only reason to not use
PQsendQueryParams is to send multiple statements, but this is not
possible in pipeline mode anyway.
Using PQsendQuery seems to produce a spurious Close message, which the
libpq is surprised to receive. Issue reported to pgsql-bugs.
https://www.postgresql.org/message-id/CA%2Bmi_8bvD0_CW3sumgwPvWdNzXY32itoG_16tDYRu_1S2gV2iw%40mail.gmail.com
Close #314
Daniele Varrazzo [Wed, 8 Jun 2022 13:18:29 +0000 (15:18 +0200)]
test: print notifications received in pipeline demo
Investigating on the conditions causing the messages reported in #314.
Daniele Varrazzo [Tue, 24 May 2022 10:35:03 +0000 (12:35 +0200)]
fix: use a generic return type to return self
Standardize the var name to _Self, waiting for PEP 0673.
Daniele Varrazzo [Tue, 7 Jun 2022 08:47:15 +0000 (10:47 +0200)]
Merge branch 'fix-stream'
Daniele Varrazzo [Tue, 7 Jun 2022 07:41:14 +0000 (09:41 +0200)]
test: add test to verify stream+close interaction
Daniele Varrazzo [Tue, 7 Jun 2022 07:21:03 +0000 (09:21 +0200)]
fix: restore the connection in usable state after an error in stream()
Daniele Varrazzo [Tue, 7 Jun 2022 04:46:56 +0000 (06:46 +0200)]
fix: shorter tracebacks on stream() error
Daniele Varrazzo [Sun, 5 Jun 2022 16:58:37 +0000 (18:58 +0200)]
test: add test to reveal unexpected message received
The test fails because we receive a notification:
message type 0x33 arrived from server while idle
See issue #314.
Daniele Varrazzo [Sat, 28 May 2022 01:02:38 +0000 (02:02 +0100)]
Merge branch 'fix-copy-escape'
Daniele Varrazzo [Wed, 25 May 2022 21:10:04 +0000 (22:10 +0100)]
fix(copy): correctly escape non-printable chars
Postgres was doing the right thing with them, but the format was wrong.
Daniele Varrazzo [Wed, 25 May 2022 17:33:18 +0000 (18:33 +0100)]
perf(copy): avoid to call _write with empty buffer
There is a call per row we can avoid if the formatter is buffering the
data produced.
Denis Laxalde [Wed, 25 May 2022 14:55:49 +0000 (16:55 +0200)]
test: drop unneeded Cursor Row type variable from typing tests
Follow up
2f6e7669f0b32e7a99dd6db0ef41a18af119909f .
Denis Laxalde [Wed, 25 May 2022 08:53:28 +0000 (10:53 +0200)]
doc: fix a typo in pipeline examples
Daniele Varrazzo [Wed, 25 May 2022 08:50:11 +0000 (10:50 +0200)]
Merge remote-tracking branch 'dlax/pipeline-sync-upon-entering-nested'