]>
git.ipfire.org Git - thirdparty/psycopg.git/log
Daniele Varrazzo [Fri, 23 Apr 2021 00:42:54 +0000 (01:42 +0100)]
Separate C and Python pgconn_ptr attribute names
Now that we are tapping into ctypes to access to the libpq dynamically
the Python value is handy for the cython layer too (see PGconn.hostaddr
implementation).
Daniele Varrazzo [Fri, 23 Apr 2021 00:35:34 +0000 (01:35 +0100)]
Merge branch 'conn-info'
Daniele Varrazzo [Fri, 23 Apr 2021 00:30:19 +0000 (01:30 +0100)]
Allow reading needs_password, used_password on broken connections
It makes sense (especially the former: it is used to detect if a
password was needed). However they are not that easy to use because a
failed connection attempt will throw an exception. For this reason they
are not added to ConnectionInfo.
Daniele Varrazzo [Fri, 23 Apr 2021 00:29:53 +0000 (01:29 +0100)]
Add ConnectionInfo.error_message, backend_pid
Daniele Varrazzo [Thu, 22 Apr 2021 23:51:18 +0000 (00:51 +0100)]
Further ConnectionInfo docs improvements
Daniele Varrazzo [Thu, 22 Apr 2021 19:18:32 +0000 (20:18 +0100)]
More compact doc rendering of introspected property types
Daniele Varrazzo [Thu, 22 Apr 2021 13:44:28 +0000 (14:44 +0100)]
Ensure pgconn_ptr, pgresult_ptr to return int in every implementation
Daniele Varrazzo [Thu, 22 Apr 2021 12:28:53 +0000 (13:28 +0100)]
Add ConnInfo.hostaddr
Also add C implementation of PGconn.hostaddr.
Daniele Varrazzo [Fri, 16 Apr 2021 18:17:03 +0000 (19:17 +0100)]
Add ConnectionInfo.server_version, protocol_version
Daniele Varrazzo [Mon, 12 Apr 2021 17:30:40 +0000 (18:30 +0100)]
Add ConnectionInfo docs
Daniele Varrazzo [Mon, 12 Apr 2021 16:16:22 +0000 (17:16 +0100)]
Add ConnectionInfo.parameter_status
Daniele Varrazzo [Mon, 12 Apr 2021 15:59:57 +0000 (16:59 +0100)]
Drop use of PQerror in ConnectionInfo
Daniele Varrazzo [Mon, 28 Dec 2020 03:38:51 +0000 (04:38 +0100)]
Test password not included in ConnectionInfo.get_parameters()
Daniele Varrazzo [Mon, 28 Dec 2020 03:25:08 +0000 (04:25 +0100)]
Added some flesh to the connection.info object
Daniele Varrazzo [Wed, 21 Apr 2021 09:56:38 +0000 (10:56 +0100)]
Only change logging level if not set
See #43
Daniele Varrazzo [Fri, 16 Apr 2021 18:01:02 +0000 (19:01 +0100)]
Use automatic type hints in docs
The result is not perfect but it can be improved. Better than redefining
all the types.
Daniele Varrazzo [Fri, 16 Apr 2021 11:37:42 +0000 (12:37 +0100)]
Fix missing dsn in async pool tests too
Denis Laxalde [Thu, 15 Apr 2021 14:19:10 +0000 (16:19 +0200)]
Pass 'dsn' value to ConnectionPool in tests
Otherwise, those tests fail when there's no database named after current
user (typically, 'connection failed: database "denis" does not exist').
Denis Laxalde [Tue, 13 Apr 2021 08:06:32 +0000 (10:06 +0200)]
Alias Task to asyncio.Future for python version < 3.8
This resolves the following mypy error on Python 3.7:
psycopg3/psycopg3/pool/async_pool.py:72: error: Argument 1 to "append" of "list" has incompatible type "Future[None]"; expected "Task[None]" [arg-type]
Daniele Varrazzo [Mon, 12 Apr 2021 15:50:54 +0000 (16:50 +0100)]
Fix test run with no --loop selected
Daniele Varrazzo [Mon, 12 Apr 2021 15:35:43 +0000 (16:35 +0100)]
Test psycopg3 with uvloop
It works, it seems.
Daniele Varrazzo [Mon, 12 Apr 2021 15:18:01 +0000 (16:18 +0100)]
Add tests readme
Daniele Varrazzo [Mon, 12 Apr 2021 15:17:48 +0000 (16:17 +0100)]
Merge branch 'quiet-logger'
Daniele Varrazzo [Mon, 12 Apr 2021 12:28:54 +0000 (13:28 +0100)]
Use public logger names (to be documented)
Use "psycopg3" and "psycopg3.pool" as loggers which can be addressed.
Also cleanup of random inconsistencies around logging.
Daniele Varrazzo [Wed, 7 Apr 2021 23:55:47 +0000 (00:55 +0100)]
Set the psycopg3 logger at warning level
Close #43. See that ticket for relevant discussion.
Denis Laxalde [Wed, 7 Apr 2021 14:08:04 +0000 (16:08 +0200)]
Re-export names from psycopg3.types
This is useful to silent mypy errors when implicit reexport is disabled
(e.g. in strict mode) like:
error: Module 'psycopg3.types' does not explicitly export attribute
'Jsonb'; implicit reexport disabled [attr-defined]
https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-no-implicit-reexport
Accordingly, we import Int4 from psycopg3.types directly in
test_copy.py.
Daniele Varrazzo [Fri, 26 Mar 2021 10:49:46 +0000 (10:49 +0000)]
Shorten tracebacks executing on connection too
Daniele Varrazzo [Thu, 18 Mar 2021 02:27:33 +0000 (03:27 +0100)]
Cut traceback on execute short
Users don't care where in psycopg the error comes.
Daniele Varrazzo [Wed, 17 Mar 2021 17:39:44 +0000 (18:39 +0100)]
Dump all string subclasses as text by default
Useful for enums.
Daniele Varrazzo [Wed, 17 Mar 2021 17:10:04 +0000 (18:10 +0100)]
Fix dump of enum subtypes
Daniele Varrazzo [Wed, 17 Mar 2021 00:08:08 +0000 (01:08 +0100)]
Drop outdated change behaviour from docs
Daniele Varrazzo [Tue, 16 Mar 2021 13:38:40 +0000 (14:38 +0100)]
Grazie tante per il vostro aiuto, @20tab!
Daniele Varrazzo [Tue, 16 Mar 2021 01:01:09 +0000 (02:01 +0100)]
Added sentry (@getsentry) as a backer
Thank you very much :)
Daniele Varrazzo [Tue, 16 Mar 2021 00:59:17 +0000 (01:59 +0100)]
Added bit.io (@bitdotioinc) as top sponsor
Amazing, thank you!
Daniele Varrazzo [Fri, 12 Mar 2021 18:21:35 +0000 (19:21 +0100)]
Typo
Daniele Varrazzo [Fri, 12 Mar 2021 17:43:12 +0000 (18:43 +0100)]
Fix typo in pool with usage
Daniele Varrazzo [Fri, 12 Mar 2021 15:57:17 +0000 (16:57 +0100)]
Merge branch 'connection-pool'
Daniele Varrazzo [Fri, 12 Mar 2021 15:47:37 +0000 (16:47 +0100)]
Better looking default params for pool in the docs
Daniele Varrazzo [Fri, 12 Mar 2021 15:30:35 +0000 (16:30 +0100)]
Rename pool minconn, maxconn params to min_size, max_size
Daniele Varrazzo [Fri, 12 Mar 2021 14:40:49 +0000 (15:40 +0100)]
Change pool stat requests_timeout to requests_errors, include queue full
Daniele Varrazzo [Fri, 12 Mar 2021 14:27:15 +0000 (15:27 +0100)]
pool 'queue_length' stats renamed to 'requests_waiting'
Daniele Varrazzo [Fri, 12 Mar 2021 14:26:18 +0000 (15:26 +0100)]
Add pool.max_waiting
Daniele Varrazzo [Fri, 12 Mar 2021 04:21:39 +0000 (05:21 +0100)]
Pool docs corrections
Daniele Varrazzo [Fri, 12 Mar 2021 04:06:15 +0000 (05:06 +0100)]
Divide docs file into chapter directories
Daniele Varrazzo [Fri, 12 Mar 2021 03:55:03 +0000 (04:55 +0100)]
Add connection pool docs
Daniele Varrazzo [Thu, 11 Mar 2021 20:11:48 +0000 (21:11 +0100)]
Grow the pool size one connection at time
Not doing so saturates the worker processes which may not be able to do
other maintenance, including returning connections to the pool. It also
helps to limit the amount of growth in a sudden spike.
o
Daniele Varrazzo [Thu, 11 Mar 2021 20:11:05 +0000 (21:11 +0100)]
Add logging to pool.wait()
Daniele Varrazzo [Thu, 11 Mar 2021 14:31:55 +0000 (15:31 +0100)]
Don't use a worker in pool.putconn() if not needed
If _reset is not set there is little work to do so the overhead of a
context switch is measurable and not necessary.
Daniele Varrazzo [Wed, 10 Mar 2021 23:40:33 +0000 (00:40 +0100)]
pool: fix off-by-one error in shrink log message
Daniele Varrazzo [Wed, 10 Mar 2021 18:34:00 +0000 (19:34 +0100)]
Add async pool stats
Daniele Varrazzo [Wed, 10 Mar 2021 02:39:46 +0000 (03:39 +0100)]
Add pool connection_class parameter
Daniele Varrazzo [Wed, 10 Mar 2021 02:32:02 +0000 (03:32 +0100)]
Make pool params keyword only in subclasses too
Daniele Varrazzo [Wed, 10 Mar 2021 02:22:07 +0000 (03:22 +0100)]
Add pool stats
Daniele Varrazzo [Mon, 8 Mar 2021 03:43:33 +0000 (04:43 +0100)]
Add reset callback to connection pool
Daniele Varrazzo [Mon, 8 Mar 2021 03:18:25 +0000 (04:18 +0100)]
Make sure that the pool config function leaves connections in idle state
Daniele Varrazzo [Mon, 8 Mar 2021 02:32:34 +0000 (03:32 +0100)]
Rename pool.wait_ready() to pool.wait()
Daniele Varrazzo [Mon, 8 Mar 2021 02:26:12 +0000 (03:26 +0100)]
Drop especially flaky async test dealing with del behaviour
Daniele Varrazzo [Mon, 8 Mar 2021 02:25:57 +0000 (03:25 +0100)]
Add AsyncConnectionPool.check()
Daniele Varrazzo [Mon, 8 Mar 2021 01:59:05 +0000 (02:59 +0100)]
Simpler, more approximative, more reliable pool max lifetime test
Daniele Varrazzo [Mon, 8 Mar 2021 01:41:53 +0000 (02:41 +0100)]
More tolerance in schedule timing tests
Daniele Varrazzo [Mon, 8 Mar 2021 01:28:14 +0000 (02:28 +0100)]
Add test for pool config
Daniele Varrazzo [Mon, 8 Mar 2021 01:24:09 +0000 (02:24 +0100)]
Don't throw an error on context exit if the connection is closed
Daniele Varrazzo [Mon, 8 Mar 2021 01:05:09 +0000 (02:05 +0100)]
Make the async pool unavailable on Python 3.6
Because of https://bugs.python.org/issue42600 or thereabout.
Daniele Varrazzo [Sun, 7 Mar 2021 23:52:58 +0000 (00:52 +0100)]
Don't use threads to manage the scheduler and workers of async tasks
Daniele Varrazzo [Sat, 27 Feb 2021 14:28:18 +0000 (15:28 +0100)]
Add an async scheduler
Daniele Varrazzo [Mon, 8 Mar 2021 00:07:53 +0000 (01:07 +0100)]
Drop get_running_loop call on async pool close
Daniele Varrazzo [Sat, 6 Mar 2021 04:02:29 +0000 (05:02 +0100)]
Add compat module for python versions compatibility
Daniele Varrazzo [Sat, 6 Mar 2021 02:38:45 +0000 (03:38 +0100)]
Add pool.check()
Daniele Varrazzo [Sat, 6 Mar 2021 02:13:37 +0000 (03:13 +0100)]
Set a timeout for the test run
Daniele Varrazzo [Sat, 6 Mar 2021 01:21:30 +0000 (02:21 +0100)]
retry flaky max lifetime test
Daniele Varrazzo [Sun, 28 Feb 2021 15:19:49 +0000 (16:19 +0100)]
Add Connection.broken
Daniele Varrazzo [Sun, 28 Feb 2021 03:35:56 +0000 (04:35 +0100)]
Add max_lifetime to pool connections
Daniele Varrazzo [Sun, 28 Feb 2021 02:55:29 +0000 (03:55 +0100)]
Add jitter method to pool
Daniele Varrazzo [Sat, 27 Feb 2021 22:47:26 +0000 (23:47 +0100)]
Use more of the pool context manager in the tests
Daniele Varrazzo [Sat, 27 Feb 2021 21:32:34 +0000 (22:32 +0100)]
Retry flaky tests a few times
Daniele Varrazzo [Sat, 27 Feb 2021 20:44:07 +0000 (21:44 +0100)]
Add pool context managers
Daniele Varrazzo [Sat, 27 Feb 2021 20:07:38 +0000 (21:07 +0100)]
Add pool.resize()
Daniele Varrazzo [Sat, 27 Feb 2021 11:47:36 +0000 (12:47 +0100)]
Make cleanup tests more reliable
Unfortunately there is some race condition on delete and for now just
using a timer.
In order to limit the problem don't actually run the StopWorker task but
bail out from the workers as soon as seen.
Daniele Varrazzo [Sat, 27 Feb 2021 11:13:17 +0000 (12:13 +0100)]
Fix Python 3.6 compatibility issues in async pool
Daniele Varrazzo [Sat, 27 Feb 2021 01:09:44 +0000 (02:09 +0100)]
Add async connection pool
Daniele Varrazzo [Thu, 25 Feb 2021 21:38:33 +0000 (22:38 +0100)]
Move non-blocking pool functionalities to a generic base class
Note that running maintenance threads is "non-blocking" and will be
thread-based in the async pool.
Daniele Varrazzo [Thu, 25 Feb 2021 20:57:20 +0000 (21:57 +0100)]
Add pool.wait_ready, drop connection_timeout on init
This way init should never block. Always fill the connection in
background but a client can decide if they want to wait for it to be
ready.
Daniele Varrazzo [Thu, 25 Feb 2021 19:36:48 +0000 (20:36 +0100)]
Split the pool package objects in modules
Daniele Varrazzo [Thu, 25 Feb 2021 19:11:56 +0000 (20:11 +0100)]
Move task implementations to the pool object
Daniele Varrazzo [Thu, 25 Feb 2021 17:31:16 +0000 (18:31 +0100)]
Make psycopg3.pool a package
Daniele Varrazzo [Tue, 23 Feb 2021 22:39:44 +0000 (23:39 +0100)]
Use the daemon param at constructor rather than the attribute
Daniele Varrazzo [Mon, 22 Feb 2021 03:20:33 +0000 (04:20 +0100)]
Use the connections in the pool uniformly
I feel this is a better use than using some more than other (e.g. in
term of bloat of the connections associated with the resources) and
gives a more predictable performance of the connection (there won't be
some cold, some hot).
Now there aren't really "unused connections" to single out in order to
shrink the pool. So keep a tally of the number of connections unused and
use a worker thread to close some if there are above minconn unused in a
period.
Daniele Varrazzo [Mon, 22 Feb 2021 01:49:16 +0000 (02:49 +0100)]
Change state in critical section on pool.close()
Nothing bad should happen, because what maintains the state respect the
_closed state. However because the logic is complex make it more
defensive.
Daniele Varrazzo [Mon, 22 Feb 2021 01:33:36 +0000 (02:33 +0100)]
Make any pool task schedulable
Added interface to the pool to schedule a task in the future.
Daniele Varrazzo [Mon, 22 Feb 2021 01:05:14 +0000 (02:05 +0100)]
Don't lose pool connections giving them to a clients already timed out
Daniele Varrazzo [Sun, 21 Feb 2021 13:04:10 +0000 (14:04 +0100)]
Allow proxy tests to fail on Travis
Don't know why it fails, it requires interactive investigation there.
Make sure tests can run connecting on TCP socket and avoid SSH in the
proxy anyway. Not that any of this worked...
Daniele Varrazzo [Sun, 21 Feb 2021 12:41:02 +0000 (13:41 +0100)]
Wait worker thread to stop on pool close
Daniele Varrazzo [Sun, 21 Feb 2021 12:07:58 +0000 (13:07 +0100)]
Don't print warning gc'ing a pool in incomplete state
Daniele Varrazzo [Sun, 21 Feb 2021 03:48:03 +0000 (04:48 +0100)]
pool.close() called on del, idemptotent
Daniele Varrazzo [Sun, 21 Feb 2021 03:47:20 +0000 (04:47 +0100)]
Add separate setup_timeout param for pool
Allow 0 too, which would fill the pool in background.
Daniele Varrazzo [Sun, 21 Feb 2021 03:28:01 +0000 (04:28 +0100)]
Drop _sec suffix from pool timeouts
Python is consistent enough to use seconds everywhere timeouts are
involved.
Daniele Varrazzo [Sun, 21 Feb 2021 03:26:00 +0000 (04:26 +0100)]
Add pool reconnection retry
Daniele Varrazzo [Sun, 21 Feb 2021 03:20:38 +0000 (04:20 +0100)]
Don't try to rollback on exit if the connection is bad
Daniele Varrazzo [Sun, 21 Feb 2021 01:17:41 +0000 (02:17 +0100)]
Use a weak reference to avoid loops between pool and maintenance tasks
Daniele Varrazzo [Sat, 20 Feb 2021 23:07:10 +0000 (00:07 +0100)]
Add scheduler class
To be used in the connection pool to implement reconnection with
backoff, maybe periodic tasks.