]>
git.ipfire.org Git - thirdparty/psycopg.git/log
Daniele Varrazzo [Tue, 26 Jul 2022 12:25:56 +0000 (13:25 +0100)]
docs: NOTIFY command in uppercase
Daniele Varrazzo [Tue, 26 Jul 2022 12:25:49 +0000 (13:25 +0100)]
Merge branch 'copy-writer'
Daniele Varrazzo [Sun, 24 Jul 2022 03:01:25 +0000 (04:01 +0100)]
docs(copy): add documentation for COPY Writer objects
Daniele Varrazzo [Sun, 24 Jul 2022 02:35:28 +0000 (03:35 +0100)]
refactor: unify code paths to set the rowcount
Daniele Varrazzo [Sun, 24 Jul 2022 02:22:32 +0000 (03:22 +0100)]
feat(copy): add 'binary' param to copy object
This allows to write binary format even when the query has executed no
COPY operation.
Notice that now we have introduced tests that convert sample_records
into sample_binary: because small numbers would be dumped to int2, use
values large enough to require int4.
Daniele Varrazzo [Sun, 24 Jul 2022 00:37:56 +0000 (01:37 +0100)]
refactor(cursor): rename _set_current_result -> _select_current_result
Daniele Varrazzo [Sat, 23 Jul 2022 22:13:41 +0000 (23:13 +0100)]
feat(copy): add FileWriter to write copy data to a file-like object
Daniele Varrazzo [Sat, 23 Jul 2022 22:02:21 +0000 (23:02 +0100)]
refactor(copy): rename QueueWriter to QueuedLibpqWriter
Daniele Varrazzo [Sat, 23 Jul 2022 21:44:55 +0000 (22:44 +0100)]
perf(copy): use non-thread/task copy writers by default
Further benchmarks show no performance improvement using a writer
thread, and a slight slowdown using a writer async task. If that's the
case we can keep it simpler.
Daniele Varrazzo [Sat, 23 Jul 2022 20:37:48 +0000 (21:37 +0100)]
docs: fix docstrings for copy objects
Daniele Varrazzo [Sun, 12 Jun 2022 22:53:26 +0000 (00:53 +0200)]
refactor(copy): rename ConnectionWriter to LibpqWriter
The class takes a cursor as input so the previous name was misleading.
Daniele Varrazzo [Sat, 11 Jun 2022 22:47:50 +0000 (00:47 +0200)]
refactor(copy): make the writer entirely responsible of the libpq copy state
This way we can create a writer which is entirely independent from the
libpq and the connection, which is useful, for instance, to format a
file with copy data.
Daniele Varrazzo [Sat, 11 Jun 2022 15:19:46 +0000 (17:19 +0200)]
feat(copy): add writer param AsyncCursor.copy()
Add async copy writers similar to the sync ones.
Daniele Varrazzo [Sat, 11 Jun 2022 14:58:02 +0000 (16:58 +0200)]
feat(copy): add ConnectionWriter
This writer allows direct writing to the libpw, without buffering queue,
and provides the interface for a writer using a libpq connection to
write.
Daniele Varrazzo [Sat, 11 Jun 2022 14:56:13 +0000 (16:56 +0200)]
feat(copy): add 'writer' parameter to Cursor.copy()
Daniele Varrazzo [Sat, 11 Jun 2022 14:21:38 +0000 (16:21 +0200)]
refactor(copy): add Writer object
This separation of responsibility will allow to:
- create writers that e.g. produce a file rather than writing to a
connection;
- create writers that write directly instead of using a queue (see
https://github.com/psycopg/psycopg/discussions/111 for an example of users
unhappy about that. I still don't see their nervousness justified, but
I'm ok to provide them an escape hatch if it's easy to implement).
Daniele Varrazzo [Sat, 11 Jun 2022 08:46:01 +0000 (10:46 +0200)]
refactor(copy) use already looked-up pgconn instead of the connection one
Left out of previous refactoring.
Daniele Varrazzo [Mon, 18 Jul 2022 23:43:03 +0000 (00:43 +0100)]
Merge branch 'docs-no-external-modules'
Daniele Varrazzo [Mon, 18 Jul 2022 13:39:55 +0000 (14:39 +0100)]
docs: allow to read the libpq definitions from a local file
For the poor souls which don't have external access building the docs.
See #337.
Daniele Varrazzo [Mon, 18 Jul 2022 13:28:30 +0000 (14:28 +0100)]
docs: upgrade sphinx version to build docs
The version of furo used is not compatible with the newly available
Sphinx.
Martin Weinelt [Mon, 18 Jul 2022 11:58:55 +0000 (13:58 +0200)]
docs: fix build with sphinx 5.0
The `no_docstring` argument to `Documenter.add_content()` has been
deprecated in Sphinx 3.4.0 and removed in Sphinx 5.0.
This will not break compatibility with earlier versions, since the
`no_docstring` argument always carried a default value.
https://github.com/sphinx-doc/sphinx/blob/v4.2.0/sphinx/ext/autodoc/__init__.py#L601
https://github.com/sphinx-doc/sphinx/blob/v4.5.0/sphinx/ext/autodoc/__init__.py#L596
Daniele Varrazzo [Mon, 18 Jul 2022 07:34:25 +0000 (08:34 +0100)]
docs: move documentation from shapely module docstrings to docs
This avoids the dependency on shapely to build the docs.
Daniele Varrazzo [Mon, 18 Jul 2022 07:27:22 +0000 (08:27 +0100)]
docs: move documentation from _dns module docstrings to docs
Allow to build the _dns documentation without importing the module,
which requires external dependencies.
Daniele Varrazzo [Mon, 18 Jul 2022 07:23:55 +0000 (08:23 +0100)]
docs: add skip-list to avoid importing external module
In sandboxed environments dnspython doesn't work because it cannot read
`/etc/resolv.conf`/
See #337 for details.
Daniele Varrazzo [Tue, 12 Jul 2022 23:29:19 +0000 (00:29 +0100)]
test: fix proxy-based tests confused by the PGHOST env var
Daniele Varrazzo [Tue, 12 Jul 2022 23:19:24 +0000 (00:19 +0100)]
test: fix dsn manipulation tests confused by the env vars
Add fixture to clean up the PG* vars from the env and replace them with
the values to test.
Daniele Varrazzo [Tue, 12 Jul 2022 23:18:45 +0000 (00:18 +0100)]
docs: fix install instruction in docker to test the pool too
Daniele Varrazzo [Tue, 12 Jul 2022 12:15:00 +0000 (13:15 +0100)]
Merge branch 'crdb'
Daniele Varrazzo [Tue, 12 Jul 2022 11:38:11 +0000 (12:38 +0100)]
test(crdb): test severity_nonlocalized on CRDB versions supporting it
Daniele Varrazzo [Tue, 12 Jul 2022 11:24:21 +0000 (12:24 +0100)]
test(crdb): tweak server-side cursors test to run on CRDB
Server-side cursors are well supported starting from CRDB 22.1.3.
Daniele Varrazzo [Tue, 12 Jul 2022 11:20:46 +0000 (12:20 +0100)]
docs: add note to test readme about testing with CRDB
Daniele Varrazzo [Tue, 5 Jul 2022 12:17:17 +0000 (13:17 +0100)]
docs(crdb): docs refinements
Daniele Varrazzo [Tue, 5 Jul 2022 08:44:22 +0000 (09:44 +0100)]
docs(crdb): add sentence to relate crdb adapters to pg ones
Daniele Varrazzo [Thu, 16 Jun 2022 23:42:05 +0000 (01:42 +0200)]
test: fix version check using proper PostgreSQL version schema
Previously to specify "libpq 14.2" it was needed to use 14.0.2.
Daniele Varrazzo [Sun, 12 Jun 2022 01:04:14 +0000 (03:04 +0200)]
test(crdb): add tests to expose canceled copy problem
See https://github.com/cockroachdb/cockroach/issues/81559
Daniele Varrazzo [Sun, 12 Jun 2022 00:43:20 +0000 (02:43 +0200)]
test(crdb): mark a few tests slow
Daniele Varrazzo [Sat, 11 Jun 2022 08:45:08 +0000 (10:45 +0200)]
test(crdb): add random copy test
Tests are currently skipped because of
https://github.com/cockroachdb/cockroach/issues/82792
Daniele Varrazzo [Wed, 8 Jun 2022 01:35:03 +0000 (03:35 +0200)]
test(crdb): don't use pg_backend_pid in pool tests
You can express the same test using info.backend_pid, which is supported
from CRDB 22.1.
Daniele Varrazzo [Wed, 8 Jun 2022 01:17:25 +0000 (03:17 +0200)]
test(crdb): add "suicide" tests
Daniele Varrazzo [Wed, 8 Jun 2022 00:35:19 +0000 (02:35 +0200)]
test(crdb): enable tests depending on backend_pid on CRDB 22.1
Daniele Varrazzo [Wed, 8 Jun 2022 00:34:35 +0000 (02:34 +0200)]
docs(crdb): include changes in CRDB 22.1
Daniele Varrazzo [Tue, 7 Jun 2022 23:11:45 +0000 (01:11 +0200)]
test(crdb): tag better the skip reason for server-side tests
Still skip all the other DECLARE tests; waiting for a couple of issues
being fixed in CRDB.
Daniele Varrazzo [Tue, 7 Jun 2022 22:02:19 +0000 (00:02 +0200)]
test(crdb): add tests related to session termination
Daniele Varrazzo [Tue, 7 Jun 2022 21:28:11 +0000 (23:28 +0200)]
test(crdb): use the crdb_skip marker
Enable cancel-based tests on CRDB 22.
Daniele Varrazzo [Tue, 7 Jun 2022 21:26:35 +0000 (23:26 +0200)]
test(crdb): accout for backend_pid supported since CRDB 22
Daniele Varrazzo [Tue, 7 Jun 2022 20:33:33 +0000 (22:33 +0200)]
test(crdb): add crdb_skip marker
Daniele Varrazzo [Tue, 7 Jun 2022 20:10:36 +0000 (22:10 +0200)]
test: allow to mark tests to run only or skip on certain version ranges
Daniele Varrazzo [Tue, 7 Jun 2022 06:13:07 +0000 (08:13 +0200)]
docs(crdb): add crdb module docs
Daniele Varrazzo [Tue, 7 Jun 2022 06:01:46 +0000 (08:01 +0200)]
fix(crdb): raise NotSupportedError on two-phase commit methods
Daniele Varrazzo [Tue, 7 Jun 2022 04:48:01 +0000 (06:48 +0200)]
test(crdb): add tests to test CHANGEFEED together with stream()
Daniele Varrazzo [Tue, 7 Jun 2022 03:02:07 +0000 (05:02 +0200)]
refactor(crdb): drop crdb_version; use server_version instead
The server_version returned by CRDB is a misleading dummy. The version
tested is reported as 13.0 but certain features are more akin to
different versions (e.g. lack of range support and different behaviour
w.r.t. unknown/text adaptation).
Daniele Varrazzo [Tue, 7 Jun 2022 02:22:10 +0000 (04:22 +0200)]
refactor(crdb) split crdb module into a package
Daniele Varrazzo [Tue, 7 Jun 2022 01:52:11 +0000 (03:52 +0200)]
test(crdb): get connection class from fixture to make sure to test CRDB objects
Daniele Varrazzo [Tue, 7 Jun 2022 01:46:51 +0000 (03:46 +0200)]
refactor(crdb): don't mix pg and crdb json configuration
Daniele Varrazzo [Mon, 6 Jun 2022 23:14:59 +0000 (01:14 +0200)]
fix(crdb): base context transaction tests on DML rather than DDL
On CRDB we may incur in problems caused by the table being dropped.
Daniele Varrazzo [Mon, 6 Jun 2022 22:50:34 +0000 (00:50 +0200)]
test(crdb): skip tests with arrays of json[b]
Not supported by CRDB: https://github.com/cockroachdb/cockroach/issues/23468
Daniele Varrazzo [Mon, 6 Jun 2022 22:33:05 +0000 (00:33 +0200)]
test: add json[b] arrays dump/load tests
Also refactor tests a bit to unify the json/jsonb in a parametric
format.
Daniele Varrazzo [Mon, 6 Jun 2022 21:59:10 +0000 (23:59 +0200)]
test(crdb): skip tests based on missing features
With this changeset, all tests pass or are skipped on CRDB 21.2.10 \o/
Daniele Varrazzo [Mon, 6 Jun 2022 21:57:31 +0000 (23:57 +0200)]
test(crdb): fix random tests
- Keep into account the different supported interval range
- Allow for float representation rounding (pre-PG 12 behaviour)
- Run leak tests in transactions to allow to recreate the tables
Daniele Varrazzo [Mon, 6 Jun 2022 21:55:40 +0000 (23:55 +0200)]
test(crdb): skip tests where CRDB seems to misread binary numeric values
Reported upstream at https://github.com/cockroachdb/cockroach/issues/82492
Daniele Varrazzo [Mon, 6 Jun 2022 01:43:01 +0000 (03:43 +0200)]
test(crdb): adapt text tests
With this changeset, all fast tests pass on CRDB.
Daniele Varrazzo [Mon, 6 Jun 2022 01:11:20 +0000 (03:11 +0200)]
fix(crdb): keep into account duplicate oids iterating types in registry
In crdb there are a few aliases (e.g. json -> jsonb oid) which are not
adequately captured by the type's regtype.
Daniele Varrazzo [Mon, 6 Jun 2022 00:04:30 +0000 (02:04 +0200)]
fix(crdb): drop spurious quotes from multi-words regtypes
Daniele Varrazzo [Mon, 6 Jun 2022 00:00:29 +0000 (02:00 +0200)]
test(crdb): adapt numeric tests
Daniele Varrazzo [Sun, 5 Jun 2022 23:10:05 +0000 (01:10 +0200)]
test(crdb): adapt array tests
Daniele Varrazzo [Sun, 5 Jun 2022 21:27:10 +0000 (23:27 +0200)]
fix(crdb): pq tests adapted
Daniele Varrazzo [Sun, 5 Jun 2022 20:42:16 +0000 (22:42 +0200)]
test(crdb): skip tests relying on pg_backend_pid()
Daniele Varrazzo [Sun, 5 Jun 2022 17:30:41 +0000 (19:30 +0200)]
test(crdb): skip transaction tests, deadlocking on CRDB
I guess is it something related to rows locks and rollback. Will ask
upstream.
Daniele Varrazzo [Sun, 5 Jun 2022 13:13:46 +0000 (15:13 +0200)]
refactor: add NoneType to abc module
Daniele Varrazzo [Wed, 25 May 2022 21:12:10 +0000 (22:12 +0100)]
test(crdb): adapt or skip more tests to CRDB
Daniele Varrazzo [Wed, 25 May 2022 08:48:43 +0000 (10:48 +0200)]
fix(crdb): allow None roundtrip
Specify the TEXT oid for None, otherwise dumping with no context will
fail.
Daniele Varrazzo [Wed, 25 May 2022 08:47:53 +0000 (10:47 +0200)]
perf: add Transformer.__slots__
Daniele Varrazzo [Wed, 25 May 2022 08:45:29 +0000 (10:45 +0200)]
feat: allow to customize the oid to dump None
On Postgres >= 10, INVALID_OID is ok, because it has a fallback text
cast. CRDB doesn't have such cast, but can convert implicitly from text
to other types, so TEXT_OID is a better choice.
Daniele Varrazzo [Wed, 25 May 2022 00:03:37 +0000 (02:03 +0200)]
test(crdb): adapt pipeline tests
Transaction-related tests currently fail.
Daniele Varrazzo [Tue, 24 May 2022 23:38:30 +0000 (01:38 +0200)]
test(crdb): adapt cursor and error tests to CRDB
Daniele Varrazzo [Tue, 24 May 2022 23:03:29 +0000 (01:03 +0200)]
test(crdb): fix conninfo tests with crdb
Daniele Varrazzo [Tue, 24 May 2022 22:54:07 +0000 (00:54 +0200)]
fix(crdb): add generic records dumper/loader
Daniele Varrazzo [Tue, 24 May 2022 22:22:46 +0000 (00:22 +0200)]
test(crdb): make prepared statements tests portable between PG and CRDB
Daniele Varrazzo [Tue, 24 May 2022 13:05:13 +0000 (15:05 +0200)]
test: fix skipping versions in parametrized tests
In the previous implementation, all the tests were skipped, because
annotations were added to the function and ended up affecting all
generated tests, not the marked ones only.
Daniele Varrazzo [Tue, 24 May 2022 08:50:57 +0000 (10:50 +0200)]
fix(crdb): fix type info of CrdbConnection.connect()
It requires specifying entirely the connect() signature because of by #308.
Daniele Varrazzo [Tue, 24 May 2022 08:50:17 +0000 (10:50 +0200)]
feat(crdb): add psycopg.crdb.connect() function
Daniele Varrazzo [Mon, 23 May 2022 00:42:15 +0000 (02:42 +0200)]
test(crdb): skip tests for unsupported types
Daniele Varrazzo [Mon, 23 May 2022 00:37:58 +0000 (02:37 +0200)]
feat(crdb): add CrdbConnection class
Drop the automatic detection of crdb. It seems dangerous to change the
program types based on external factors such as the database connected
to.
Generate and use a crdb-specific types oids map, on which to build the
adapters map.
Daniele Varrazzo [Sun, 22 May 2022 00:44:17 +0000 (02:44 +0200)]
fix(crdb): fix json adaptation
The function set_json_dumps() now adapts the currently registered
adapter, not the default one, to be able to customize the Json dumper on
CRDB, without resetting its oid to the PostgreSQL standard one, which
CRDB doesn't know.
Daniele Varrazzo [Sun, 22 May 2022 00:12:01 +0000 (02:12 +0200)]
fix(crdb): configure dumpers to deal correctly with Enum types
Daniele Varrazzo [Sat, 21 May 2022 21:24:37 +0000 (23:24 +0200)]
test(crdb): adapt datetime tests
Daniele Varrazzo [Fri, 20 May 2022 07:53:20 +0000 (09:53 +0200)]
test(crdb): add async copy tests
Daniele Varrazzo [Fri, 20 May 2022 00:32:19 +0000 (02:32 +0200)]
test(crdb): add copy tests
Skip tests from psycopg, because there is no support for copy out and
copy in syntax is different. Data types are different too (int/serial
mean int8) so the binary data samples don't match the format.
Daniele Varrazzo [Wed, 18 May 2022 00:27:02 +0000 (02:27 +0200)]
test(crdb): skip connection tests which cannot run on CockroachDB
Daniele Varrazzo [Tue, 17 May 2022 21:14:16 +0000 (23:14 +0200)]
test(crdb): fix most cursor tests to run with CockroachDB
Move some common test markers to the fix_crdb module.
Remaining cursor failing test may require some tweak:
- something with binary parameters
- something with ::text cast added by crdb dumper
Daniele Varrazzo [Tue, 17 May 2022 19:51:13 +0000 (21:51 +0200)]
test: refactor skipping tests by server version
Daniele Varrazzo [Tue, 17 May 2022 01:27:34 +0000 (03:27 +0200)]
fix(crdb): allow non-normalized encoding name from the database
CockroachDB returns what stored: "utf_8", "utf-8" etc.
All ConnectionInfo tests pass on crdb.
Daniele Varrazzo [Tue, 17 May 2022 00:28:53 +0000 (02:28 +0200)]
feat(crdb): cusomize CockroachDB connection
- Add ConnectionInfo.vendor
- Add ConnectionInfo subclassing
- Add CrdbConnectionInfo subclass with crdb_version attribute
- Add 'psycopg.crdb' module with adapters
- Dump strings using the text oid by default on CockroachDB
The latter change might have wider consequences, however crdb casts
strings to other types more easily than what Postgres does. At a glance
it might work; porting the rest of the test suite will tell.
test_adapt tests ported. Tests showing a difference moved to
crdb/test_adapt.
Daniele Varrazzo [Mon, 16 May 2022 20:49:02 +0000 (22:49 +0200)]
test(crdb): skip TPC tests
Daniele Varrazzo [Mon, 16 May 2022 02:59:04 +0000 (04:59 +0200)]
test(crdb): add fixture to support CockroachDB test/skip
Daniele Varrazzo [Tue, 12 Jul 2022 11:58:17 +0000 (12:58 +0100)]
Merge branch 'codespell-all'
Christian Clauss [Tue, 28 Jun 2022 12:25:03 +0000 (14:25 +0200)]
chore: fix typos on previously missed files
Christian Clauss [Tue, 28 Jun 2022 11:54:42 +0000 (13:54 +0200)]
ci: run codespell on the entire repo
Daniele Varrazzo [Sun, 10 Jul 2022 20:23:14 +0000 (21:23 +0100)]
test: register the pipeline marker consistently with the others
Denis Laxalde [Sun, 3 Jul 2022 19:00:24 +0000 (21:00 +0200)]
docs: fix a typo in literal string examples