]> git.ipfire.org Git - thirdparty/psycopg.git/log
thirdparty/psycopg.git
4 years agoSeparate C and Python pgconn_ptr attribute names
Daniele Varrazzo [Fri, 23 Apr 2021 00:42:54 +0000 (01:42 +0100)] 
Separate C and Python pgconn_ptr attribute names

Now that we are tapping into ctypes to access to the libpq dynamically
the Python value is handy for the cython layer too (see PGconn.hostaddr
implementation).

4 years agoMerge branch 'conn-info'
Daniele Varrazzo [Fri, 23 Apr 2021 00:35:34 +0000 (01:35 +0100)] 
Merge branch 'conn-info'

4 years agoAllow reading needs_password, used_password on broken connections
Daniele Varrazzo [Fri, 23 Apr 2021 00:30:19 +0000 (01:30 +0100)] 
Allow reading needs_password, used_password on broken connections

It makes sense (especially the former: it is used to detect if a
password was needed). However they are not that easy to use because a
failed connection attempt will throw an exception. For this reason they
are not added to ConnectionInfo.

4 years agoAdd ConnectionInfo.error_message, backend_pid
Daniele Varrazzo [Fri, 23 Apr 2021 00:29:53 +0000 (01:29 +0100)] 
Add ConnectionInfo.error_message, backend_pid

4 years agoFurther ConnectionInfo docs improvements
Daniele Varrazzo [Thu, 22 Apr 2021 23:51:18 +0000 (00:51 +0100)] 
Further ConnectionInfo docs improvements

4 years agoMore compact doc rendering of introspected property types
Daniele Varrazzo [Thu, 22 Apr 2021 19:18:32 +0000 (20:18 +0100)] 
More compact doc rendering of introspected property types

4 years agoEnsure pgconn_ptr, pgresult_ptr to return int in every implementation
Daniele Varrazzo [Thu, 22 Apr 2021 13:44:28 +0000 (14:44 +0100)] 
Ensure pgconn_ptr, pgresult_ptr to return int in every implementation

4 years agoAdd ConnInfo.hostaddr
Daniele Varrazzo [Thu, 22 Apr 2021 12:28:53 +0000 (13:28 +0100)] 
Add ConnInfo.hostaddr

Also add C implementation of PGconn.hostaddr.

4 years agoAdd ConnectionInfo.server_version, protocol_version
Daniele Varrazzo [Fri, 16 Apr 2021 18:17:03 +0000 (19:17 +0100)] 
Add ConnectionInfo.server_version, protocol_version

4 years agoAdd ConnectionInfo docs
Daniele Varrazzo [Mon, 12 Apr 2021 17:30:40 +0000 (18:30 +0100)] 
Add ConnectionInfo docs

4 years agoAdd ConnectionInfo.parameter_status
Daniele Varrazzo [Mon, 12 Apr 2021 16:16:22 +0000 (17:16 +0100)] 
Add ConnectionInfo.parameter_status

4 years agoDrop use of PQerror in ConnectionInfo
Daniele Varrazzo [Mon, 12 Apr 2021 15:59:57 +0000 (16:59 +0100)] 
Drop use of PQerror in ConnectionInfo

4 years agoTest password not included in ConnectionInfo.get_parameters()
Daniele Varrazzo [Mon, 28 Dec 2020 03:38:51 +0000 (04:38 +0100)] 
Test password not included in ConnectionInfo.get_parameters()

4 years agoAdded some flesh to the connection.info object
Daniele Varrazzo [Mon, 28 Dec 2020 03:25:08 +0000 (04:25 +0100)] 
Added some flesh to the connection.info object

4 years agoOnly change logging level if not set
Daniele Varrazzo [Wed, 21 Apr 2021 09:56:38 +0000 (10:56 +0100)] 
Only change logging level if not set

See #43

4 years agoUse automatic type hints in docs
Daniele Varrazzo [Fri, 16 Apr 2021 18:01:02 +0000 (19:01 +0100)] 
Use automatic type hints in docs

The result is not perfect but it can be improved. Better than redefining
all the types.

4 years agoFix missing dsn in async pool tests too
Daniele Varrazzo [Fri, 16 Apr 2021 11:37:42 +0000 (12:37 +0100)] 
Fix missing dsn in async pool tests too

4 years agoPass 'dsn' value to ConnectionPool in tests
Denis Laxalde [Thu, 15 Apr 2021 14:19:10 +0000 (16:19 +0200)] 
Pass 'dsn' value to ConnectionPool in tests

Otherwise, those tests fail when there's no database named after current
user (typically, 'connection failed: database "denis" does not exist').

4 years agoAlias Task to asyncio.Future for python version < 3.8
Denis Laxalde [Tue, 13 Apr 2021 08:06:32 +0000 (10:06 +0200)] 
Alias Task to asyncio.Future for python version < 3.8

This resolves the following mypy error on Python 3.7:

  psycopg3/psycopg3/pool/async_pool.py:72: error: Argument 1 to "append" of "list" has incompatible type "Future[None]"; expected "Task[None]" [arg-type]

4 years agoFix test run with no --loop selected
Daniele Varrazzo [Mon, 12 Apr 2021 15:50:54 +0000 (16:50 +0100)] 
Fix test run with no --loop selected

4 years agoTest psycopg3 with uvloop
Daniele Varrazzo [Mon, 12 Apr 2021 15:35:43 +0000 (16:35 +0100)] 
Test psycopg3 with uvloop

It works, it seems.

4 years agoAdd tests readme
Daniele Varrazzo [Mon, 12 Apr 2021 15:18:01 +0000 (16:18 +0100)] 
Add tests readme

4 years agoMerge branch 'quiet-logger'
Daniele Varrazzo [Mon, 12 Apr 2021 15:17:48 +0000 (16:17 +0100)] 
Merge branch 'quiet-logger'

4 years agoUse public logger names (to be documented)
Daniele Varrazzo [Mon, 12 Apr 2021 12:28:54 +0000 (13:28 +0100)] 
Use public logger names (to be documented)

Use "psycopg3" and "psycopg3.pool" as loggers which can be addressed.

Also cleanup of random inconsistencies around logging.

4 years agoSet the psycopg3 logger at warning level
Daniele Varrazzo [Wed, 7 Apr 2021 23:55:47 +0000 (00:55 +0100)] 
Set the psycopg3 logger at warning level

Close #43. See that ticket for relevant discussion.

4 years agoRe-export names from psycopg3.types 44/head
Denis Laxalde [Wed, 7 Apr 2021 14:08:04 +0000 (16:08 +0200)] 
Re-export names from psycopg3.types

This is useful to silent mypy errors when implicit reexport is disabled
(e.g. in strict mode) like:

    error: Module 'psycopg3.types' does not explicitly export attribute
    'Jsonb'; implicit reexport disabled  [attr-defined]

https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-no-implicit-reexport

Accordingly, we import Int4 from psycopg3.types directly in
test_copy.py.

4 years agoShorten tracebacks executing on connection too
Daniele Varrazzo [Fri, 26 Mar 2021 10:49:46 +0000 (10:49 +0000)] 
Shorten tracebacks executing on connection too

4 years agoCut traceback on execute short
Daniele Varrazzo [Thu, 18 Mar 2021 02:27:33 +0000 (03:27 +0100)] 
Cut traceback on execute short

Users don't care where in psycopg the error comes.

4 years agoDump all string subclasses as text by default
Daniele Varrazzo [Wed, 17 Mar 2021 17:39:44 +0000 (18:39 +0100)] 
Dump all string subclasses as text by default

Useful for enums.

4 years agoFix dump of enum subtypes
Daniele Varrazzo [Wed, 17 Mar 2021 17:10:04 +0000 (18:10 +0100)] 
Fix dump of enum subtypes

4 years agoDrop outdated change behaviour from docs
Daniele Varrazzo [Wed, 17 Mar 2021 00:08:08 +0000 (01:08 +0100)] 
Drop outdated change behaviour from docs

4 years agoGrazie tante per il vostro aiuto, @20tab!
Daniele Varrazzo [Tue, 16 Mar 2021 13:38:40 +0000 (14:38 +0100)] 
Grazie tante per il vostro aiuto, @20tab!

4 years agoAdded sentry (@getsentry) as a backer
Daniele Varrazzo [Tue, 16 Mar 2021 01:01:09 +0000 (02:01 +0100)] 
Added sentry (@getsentry) as a backer

Thank you very much :)

4 years agoAdded bit.io (@bitdotioinc) as top sponsor
Daniele Varrazzo [Tue, 16 Mar 2021 00:59:17 +0000 (01:59 +0100)] 
Added bit.io (@bitdotioinc) as top sponsor

Amazing, thank you!

4 years agoTypo
Daniele Varrazzo [Fri, 12 Mar 2021 18:21:35 +0000 (19:21 +0100)] 
Typo

4 years agoFix typo in pool with usage
Daniele Varrazzo [Fri, 12 Mar 2021 17:43:12 +0000 (18:43 +0100)] 
Fix typo in pool with usage

4 years agoMerge branch 'connection-pool'
Daniele Varrazzo [Fri, 12 Mar 2021 15:57:17 +0000 (16:57 +0100)] 
Merge branch 'connection-pool'

4 years agoBetter looking default params for pool in the docs
Daniele Varrazzo [Fri, 12 Mar 2021 15:47:37 +0000 (16:47 +0100)] 
Better looking default params for pool in the docs

4 years agoRename pool minconn, maxconn params to min_size, max_size
Daniele Varrazzo [Fri, 12 Mar 2021 15:30:35 +0000 (16:30 +0100)] 
Rename pool minconn, maxconn params to min_size, max_size

4 years agoChange pool stat requests_timeout to requests_errors, include queue full
Daniele Varrazzo [Fri, 12 Mar 2021 14:40:49 +0000 (15:40 +0100)] 
Change pool stat requests_timeout to requests_errors, include queue full

4 years agopool 'queue_length' stats renamed to 'requests_waiting'
Daniele Varrazzo [Fri, 12 Mar 2021 14:27:15 +0000 (15:27 +0100)] 
pool 'queue_length' stats renamed to 'requests_waiting'

4 years agoAdd pool.max_waiting
Daniele Varrazzo [Fri, 12 Mar 2021 14:26:18 +0000 (15:26 +0100)] 
Add pool.max_waiting

4 years agoPool docs corrections
Daniele Varrazzo [Fri, 12 Mar 2021 04:21:39 +0000 (05:21 +0100)] 
Pool docs corrections

4 years agoDivide docs file into chapter directories
Daniele Varrazzo [Fri, 12 Mar 2021 04:06:15 +0000 (05:06 +0100)] 
Divide docs file into chapter directories

4 years agoAdd connection pool docs
Daniele Varrazzo [Fri, 12 Mar 2021 03:55:03 +0000 (04:55 +0100)] 
Add connection pool docs

4 years agoGrow the pool size one connection at time
Daniele Varrazzo [Thu, 11 Mar 2021 20:11:48 +0000 (21:11 +0100)] 
Grow the pool size one connection at time

Not doing so saturates the worker processes which may not be able to do
other maintenance, including returning connections to the pool. It also
helps to limit the amount of growth in a sudden spike.
o

4 years agoAdd logging to pool.wait()
Daniele Varrazzo [Thu, 11 Mar 2021 20:11:05 +0000 (21:11 +0100)] 
Add logging to pool.wait()

4 years agoDon't use a worker in pool.putconn() if not needed
Daniele Varrazzo [Thu, 11 Mar 2021 14:31:55 +0000 (15:31 +0100)] 
Don't use a worker in pool.putconn() if not needed

If _reset is not set there is little work to do so the overhead of a
context switch is measurable and not necessary.

4 years agopool: fix off-by-one error in shrink log message
Daniele Varrazzo [Wed, 10 Mar 2021 23:40:33 +0000 (00:40 +0100)] 
pool: fix off-by-one error in shrink log message

4 years agoAdd async pool stats
Daniele Varrazzo [Wed, 10 Mar 2021 18:34:00 +0000 (19:34 +0100)] 
Add async pool stats

4 years agoAdd pool connection_class parameter
Daniele Varrazzo [Wed, 10 Mar 2021 02:39:46 +0000 (03:39 +0100)] 
Add pool connection_class parameter

4 years agoMake pool params keyword only in subclasses too
Daniele Varrazzo [Wed, 10 Mar 2021 02:32:02 +0000 (03:32 +0100)] 
Make pool params keyword only in subclasses too

4 years agoAdd pool stats
Daniele Varrazzo [Wed, 10 Mar 2021 02:22:07 +0000 (03:22 +0100)] 
Add pool stats

4 years agoAdd reset callback to connection pool
Daniele Varrazzo [Mon, 8 Mar 2021 03:43:33 +0000 (04:43 +0100)] 
Add reset callback to connection pool

4 years agoMake sure that the pool config function leaves connections in idle state
Daniele Varrazzo [Mon, 8 Mar 2021 03:18:25 +0000 (04:18 +0100)] 
Make sure that the pool config function leaves connections in idle state

4 years agoRename pool.wait_ready() to pool.wait()
Daniele Varrazzo [Mon, 8 Mar 2021 02:32:34 +0000 (03:32 +0100)] 
Rename pool.wait_ready() to pool.wait()

4 years agoDrop especially flaky async test dealing with del behaviour
Daniele Varrazzo [Mon, 8 Mar 2021 02:26:12 +0000 (03:26 +0100)] 
Drop especially flaky async test dealing with del behaviour

4 years agoAdd AsyncConnectionPool.check()
Daniele Varrazzo [Mon, 8 Mar 2021 02:25:57 +0000 (03:25 +0100)] 
Add AsyncConnectionPool.check()

4 years agoSimpler, more approximative, more reliable pool max lifetime test
Daniele Varrazzo [Mon, 8 Mar 2021 01:59:05 +0000 (02:59 +0100)] 
Simpler, more approximative, more reliable pool max lifetime test

4 years agoMore tolerance in schedule timing tests
Daniele Varrazzo [Mon, 8 Mar 2021 01:41:53 +0000 (02:41 +0100)] 
More tolerance in schedule timing tests

4 years agoAdd test for pool config
Daniele Varrazzo [Mon, 8 Mar 2021 01:28:14 +0000 (02:28 +0100)] 
Add test for pool config

4 years agoDon't throw an error on context exit if the connection is closed
Daniele Varrazzo [Mon, 8 Mar 2021 01:24:09 +0000 (02:24 +0100)] 
Don't throw an error on context exit if the connection is closed

4 years agoMake the async pool unavailable on Python 3.6
Daniele Varrazzo [Mon, 8 Mar 2021 01:05:09 +0000 (02:05 +0100)] 
Make the async pool unavailable on Python 3.6

Because of https://bugs.python.org/issue42600 or thereabout.

4 years agoDon't use threads to manage the scheduler and workers of async tasks
Daniele Varrazzo [Sun, 7 Mar 2021 23:52:58 +0000 (00:52 +0100)] 
Don't use threads to manage the scheduler and workers of async tasks

4 years agoAdd an async scheduler
Daniele Varrazzo [Sat, 27 Feb 2021 14:28:18 +0000 (15:28 +0100)] 
Add an async scheduler

4 years agoDrop get_running_loop call on async pool close
Daniele Varrazzo [Mon, 8 Mar 2021 00:07:53 +0000 (01:07 +0100)] 
Drop get_running_loop call on async pool close

4 years agoAdd compat module for python versions compatibility
Daniele Varrazzo [Sat, 6 Mar 2021 04:02:29 +0000 (05:02 +0100)] 
Add compat module for python versions compatibility

4 years agoAdd pool.check()
Daniele Varrazzo [Sat, 6 Mar 2021 02:38:45 +0000 (03:38 +0100)] 
Add pool.check()

4 years agoSet a timeout for the test run
Daniele Varrazzo [Sat, 6 Mar 2021 02:13:37 +0000 (03:13 +0100)] 
Set a timeout for the test run

4 years agoretry flaky max lifetime test
Daniele Varrazzo [Sat, 6 Mar 2021 01:21:30 +0000 (02:21 +0100)] 
retry flaky max lifetime test

4 years agoAdd Connection.broken
Daniele Varrazzo [Sun, 28 Feb 2021 15:19:49 +0000 (16:19 +0100)] 
Add Connection.broken

4 years agoAdd max_lifetime to pool connections
Daniele Varrazzo [Sun, 28 Feb 2021 03:35:56 +0000 (04:35 +0100)] 
Add max_lifetime to pool connections

4 years agoAdd jitter method to pool
Daniele Varrazzo [Sun, 28 Feb 2021 02:55:29 +0000 (03:55 +0100)] 
Add jitter method to pool

4 years agoUse more of the pool context manager in the tests
Daniele Varrazzo [Sat, 27 Feb 2021 22:47:26 +0000 (23:47 +0100)] 
Use more of the pool context manager in the tests

4 years agoRetry flaky tests a few times
Daniele Varrazzo [Sat, 27 Feb 2021 21:32:34 +0000 (22:32 +0100)] 
Retry flaky tests a few times

4 years agoAdd pool context managers
Daniele Varrazzo [Sat, 27 Feb 2021 20:44:07 +0000 (21:44 +0100)] 
Add pool context managers

4 years agoAdd pool.resize()
Daniele Varrazzo [Sat, 27 Feb 2021 20:07:38 +0000 (21:07 +0100)] 
Add pool.resize()

4 years agoMake cleanup tests more reliable
Daniele Varrazzo [Sat, 27 Feb 2021 11:47:36 +0000 (12:47 +0100)] 
Make cleanup tests more reliable

Unfortunately there is some race condition on delete and for now just
using a timer.

In order to limit the problem don't actually run the StopWorker task but
bail out from the workers as soon as seen.

4 years agoFix Python 3.6 compatibility issues in async pool
Daniele Varrazzo [Sat, 27 Feb 2021 11:13:17 +0000 (12:13 +0100)] 
Fix Python 3.6 compatibility issues in async pool

4 years agoAdd async connection pool
Daniele Varrazzo [Sat, 27 Feb 2021 01:09:44 +0000 (02:09 +0100)] 
Add async connection pool

4 years agoMove non-blocking pool functionalities to a generic base class
Daniele Varrazzo [Thu, 25 Feb 2021 21:38:33 +0000 (22:38 +0100)] 
Move non-blocking pool functionalities to a generic base class

Note that running maintenance threads is "non-blocking" and will be
thread-based in the async pool.

4 years agoAdd pool.wait_ready, drop connection_timeout on init
Daniele Varrazzo [Thu, 25 Feb 2021 20:57:20 +0000 (21:57 +0100)] 
Add pool.wait_ready, drop connection_timeout on init

This way init should never block. Always fill the connection in
background but a client can decide if they want to wait for it to be
ready.

4 years agoSplit the pool package objects in modules
Daniele Varrazzo [Thu, 25 Feb 2021 19:36:48 +0000 (20:36 +0100)] 
Split the pool package objects in modules

4 years agoMove task implementations to the pool object
Daniele Varrazzo [Thu, 25 Feb 2021 19:11:56 +0000 (20:11 +0100)] 
Move task implementations to the pool object

4 years agoMake psycopg3.pool a package
Daniele Varrazzo [Thu, 25 Feb 2021 17:31:16 +0000 (18:31 +0100)] 
Make psycopg3.pool a package

4 years agoUse the daemon param at constructor rather than the attribute
Daniele Varrazzo [Tue, 23 Feb 2021 22:39:44 +0000 (23:39 +0100)] 
Use the daemon param at constructor rather than the attribute

4 years agoUse the connections in the pool uniformly
Daniele Varrazzo [Mon, 22 Feb 2021 03:20:33 +0000 (04:20 +0100)] 
Use the connections in the pool uniformly

I feel this is a better use than using some more than other (e.g. in
term of bloat of the connections associated with the resources) and
gives a more predictable performance of the connection (there won't be
some cold, some hot).

Now there aren't really "unused connections" to single out in order to
shrink the pool. So keep a tally of the number of connections unused and
use a worker thread to close some if there are above minconn unused in a
period.

4 years agoChange state in critical section on pool.close()
Daniele Varrazzo [Mon, 22 Feb 2021 01:49:16 +0000 (02:49 +0100)] 
Change state in critical section on pool.close()

Nothing bad should happen, because what maintains the state respect the
_closed state. However because the logic is complex make it more
defensive.

4 years agoMake any pool task schedulable
Daniele Varrazzo [Mon, 22 Feb 2021 01:33:36 +0000 (02:33 +0100)] 
Make any pool task schedulable

Added interface to the pool to schedule a task in the future.

4 years agoDon't lose pool connections giving them to a clients already timed out
Daniele Varrazzo [Mon, 22 Feb 2021 01:05:14 +0000 (02:05 +0100)] 
Don't lose pool connections giving them to a clients already timed out

4 years agoAllow proxy tests to fail on Travis
Daniele Varrazzo [Sun, 21 Feb 2021 13:04:10 +0000 (14:04 +0100)] 
Allow proxy tests to fail on Travis

Don't know why it fails, it requires interactive investigation there.
Make sure tests can run connecting on TCP socket and avoid SSH in the
proxy anyway. Not that any of this worked...

4 years agoWait worker thread to stop on pool close
Daniele Varrazzo [Sun, 21 Feb 2021 12:41:02 +0000 (13:41 +0100)] 
Wait worker thread to stop on pool close

4 years agoDon't print warning gc'ing a pool in incomplete state
Daniele Varrazzo [Sun, 21 Feb 2021 12:07:58 +0000 (13:07 +0100)] 
Don't print warning gc'ing a pool in incomplete state

4 years agopool.close() called on del, idemptotent
Daniele Varrazzo [Sun, 21 Feb 2021 03:48:03 +0000 (04:48 +0100)] 
pool.close() called on del, idemptotent

4 years agoAdd separate setup_timeout param for pool
Daniele Varrazzo [Sun, 21 Feb 2021 03:47:20 +0000 (04:47 +0100)] 
Add separate setup_timeout param for pool

Allow 0 too, which would fill the pool in background.

4 years agoDrop _sec suffix from pool timeouts
Daniele Varrazzo [Sun, 21 Feb 2021 03:28:01 +0000 (04:28 +0100)] 
Drop _sec suffix from pool timeouts

Python is consistent enough to use seconds everywhere timeouts are
involved.

4 years agoAdd pool reconnection retry
Daniele Varrazzo [Sun, 21 Feb 2021 03:26:00 +0000 (04:26 +0100)] 
Add pool reconnection retry

4 years agoDon't try to rollback on exit if the connection is bad
Daniele Varrazzo [Sun, 21 Feb 2021 03:20:38 +0000 (04:20 +0100)] 
Don't try to rollback on exit if the connection is bad

4 years agoUse a weak reference to avoid loops between pool and maintenance tasks
Daniele Varrazzo [Sun, 21 Feb 2021 01:17:41 +0000 (02:17 +0100)] 
Use a weak reference to avoid loops between pool and maintenance tasks

4 years agoAdd scheduler class
Daniele Varrazzo [Sat, 20 Feb 2021 23:07:10 +0000 (00:07 +0100)] 
Add scheduler class

To be used in the connection pool to implement reconnection with
backoff, maybe periodic tasks.