]> git.ipfire.org Git - thirdparty/psycopg.git/log
thirdparty/psycopg.git
4 years agoFix float parsing when the data buffer is not terminated
Daniele Varrazzo [Thu, 10 Jun 2021 16:08:12 +0000 (17:08 +0100)] 
Fix float parsing when the data buffer is not terminated

4 years agoFix DB API Binary wrapper
Daniele Varrazzo [Thu, 10 Jun 2021 10:25:47 +0000 (11:25 +0100)] 
Fix DB API Binary wrapper

4 years agoConsider some tests flaky
Daniele Varrazzo [Thu, 10 Jun 2021 11:57:19 +0000 (12:57 +0100)] 
Consider some tests flaky

They fail regularly enough in the github workflow

4 years agoDrop timeout on workflow test step
Daniele Varrazzo [Thu, 10 Jun 2021 11:05:11 +0000 (12:05 +0100)] 
Drop timeout on workflow test step

4 years agoLonger test run timeout
Daniele Varrazzo [Wed, 9 Jun 2021 23:06:45 +0000 (00:06 +0100)] 
Longer test run timeout

4 years agoFix reveal_type tests to mypy 0.901 output format
Daniele Varrazzo [Wed, 9 Jun 2021 16:28:32 +0000 (17:28 +0100)] 
Fix reveal_type tests to mypy 0.901 output format

4 years agoBetter naming and typing for a couple of vars
Daniele Varrazzo [Tue, 8 Jun 2021 12:58:59 +0000 (13:58 +0100)] 
Better naming and typing for a couple of vars

Note that mypy has some issue there, so a "type: ignore" that seems
required cannot be actually added.

4 years agoMerge branch 'connect-timeout'
Daniele Varrazzo [Tue, 8 Jun 2021 16:09:40 +0000 (17:09 +0100)] 
Merge branch 'connect-timeout'

4 years agoRaise OperationalError for connection timeout
Denis Laxalde [Thu, 6 May 2021 06:42:08 +0000 (08:42 +0200)] 
Raise OperationalError for connection timeout

4 years agoAdd support for connection timeout
Denis Laxalde [Tue, 4 May 2021 15:36:33 +0000 (17:36 +0200)] 
Add support for connection timeout

In *Connection.connect(), we replace call to make_conninfo() by the new
_conninfo_connect_timeout() utility function which builds the 'conninfo'
string (using make_conninfo()) and extracts the 'connect_timeout'
parameter.

For the synchronous API, this timeout value is then handled to
waiting.wait_conn(), to be used in the select() call. There, if select()
does not return within timeout, we raise a DatabaseError.

For the asynchronous API, it is passed to waiting.wait_conn_async()
where we use asyncio.wait_for() to wait for the event and also raise a
DatabaseError in case of timeout.

4 years agoBuild conninfo in *Connection.connect()
Denis Laxalde [Wed, 5 May 2021 07:40:26 +0000 (09:40 +0200)] 
Build conninfo in *Connection.connect()

We'll need to extract 'connect_timeout' from the connection string and
use it outside Connection._connect_gen() in the next commit.

4 years agoMerge branch 'network-binary'
Daniele Varrazzo [Tue, 8 Jun 2021 10:41:17 +0000 (11:41 +0100)] 
Merge branch 'network-binary'

4 years agoGenerate random ipaddress objects for testing
Daniele Varrazzo [Tue, 8 Jun 2021 04:07:04 +0000 (05:07 +0100)] 
Generate random ipaddress objects for testing

4 years agoAdd network types binary loaders
Daniele Varrazzo [Tue, 8 Jun 2021 03:48:12 +0000 (04:48 +0100)] 
Add network types binary loaders

4 years agoAdd ipaddress binary dumpers
Daniele Varrazzo [Tue, 8 Jun 2021 03:15:34 +0000 (04:15 +0100)] 
Add ipaddress binary dumpers

4 years agoDrop utils sub-package
Daniele Varrazzo [Tue, 8 Jun 2021 01:42:43 +0000 (02:42 +0100)] 
Drop utils sub-package

4 years agoMerge branch 'c-datetime-adapters'
Daniele Varrazzo [Tue, 8 Jun 2021 01:32:26 +0000 (02:32 +0100)] 
Merge branch 'c-datetime-adapters'

4 years agoAdd C interval text loader
Daniele Varrazzo [Tue, 8 Jun 2021 01:00:43 +0000 (02:00 +0100)] 
Add C interval text loader

4 years agoAdd C interval binary loader
Daniele Varrazzo [Mon, 7 Jun 2021 22:37:11 +0000 (23:37 +0100)] 
Add C interval binary loader

4 years agoUse consistently 'us' instead of 'ms' for microseconds
Daniele Varrazzo [Mon, 7 Jun 2021 21:42:22 +0000 (22:42 +0100)] 
Use consistently 'us' instead of 'ms' for microseconds

4 years agoPad second fractions to micros using a lookup table
Daniele Varrazzo [Mon, 7 Jun 2021 20:44:00 +0000 (21:44 +0100)] 
Pad second fractions to micros using a lookup table

4 years agoAdd timestamptz c binary loader
Daniele Varrazzo [Mon, 7 Jun 2021 20:29:42 +0000 (21:29 +0100)] 
Add timestamptz c binary loader

4 years agoAdd timestamptz c text loader
Daniele Varrazzo [Sun, 6 Jun 2021 23:38:56 +0000 (00:38 +0100)] 
Add timestamptz c text loader

4 years agoDrop code duplication in date/time parsing
Daniele Varrazzo [Sun, 6 Jun 2021 23:25:20 +0000 (00:25 +0100)] 
Drop code duplication in date/time parsing

4 years agoAdd c timestamp binary loader
Daniele Varrazzo [Sun, 6 Jun 2021 21:31:33 +0000 (22:31 +0100)] 
Add c timestamp binary loader

4 years agoParse datetype without using endswith in c
Daniele Varrazzo [Sun, 6 Jun 2021 21:30:51 +0000 (22:30 +0100)] 
Parse datetype without using endswith in c

4 years agoAdd c text timestamp loader
Daniele Varrazzo [Wed, 2 Jun 2021 13:13:40 +0000 (14:13 +0100)] 
Add c text timestamp loader

4 years agoAdd c timetz binary loader
Daniele Varrazzo [Wed, 2 Jun 2021 10:42:12 +0000 (11:42 +0100)] 
Add c timetz binary loader

4 years agoAdd helper function to return timezone from second
Daniele Varrazzo [Wed, 2 Jun 2021 03:57:15 +0000 (04:57 +0100)] 
Add helper function to return timezone from second

Use a cache to return the same object for the same offset.

4 years agoAdd c text timetz loader
Daniele Varrazzo [Wed, 2 Jun 2021 03:28:07 +0000 (04:28 +0100)] 
Add c text timetz loader

4 years agoAdd c binary time loader
Daniele Varrazzo [Wed, 2 Jun 2021 02:44:48 +0000 (03:44 +0100)] 
Add c binary time loader

4 years agoAdd c text time loader
Daniele Varrazzo [Wed, 2 Jun 2021 02:27:44 +0000 (03:27 +0100)] 
Add c text time loader

4 years agoAdd c timedelta dumpers
Daniele Varrazzo [Wed, 2 Jun 2021 01:55:59 +0000 (02:55 +0100)] 
Add c timedelta dumpers

4 years agoAdd c datetime binary dumpers
Daniele Varrazzo [Wed, 2 Jun 2021 01:29:33 +0000 (02:29 +0100)] 
Add c datetime binary dumpers

4 years agoAdd c datetime text dumpers
Daniele Varrazzo [Wed, 2 Jun 2021 00:49:23 +0000 (01:49 +0100)] 
Add c datetime text dumpers

4 years agoRationalise date dumpers inheritance tree
Daniele Varrazzo [Wed, 2 Jun 2021 00:43:22 +0000 (01:43 +0100)] 
Rationalise date dumpers inheritance tree

4 years agoAdd C time dumpers
Daniele Varrazzo [Sun, 16 May 2021 22:46:50 +0000 (00:46 +0200)] 
Add C time dumpers

4 years agoSay "can't parse x", not "can't manage x"
Daniele Varrazzo [Sun, 16 May 2021 01:39:55 +0000 (03:39 +0200)] 
Say "can't parse x", not "can't manage x"

These are parsing error. Stuff we can't manage (e.g. year < 10K) is
complained of elsewhere

4 years agoUniform date/time loaders naming
Daniele Varrazzo [Sun, 16 May 2021 01:38:00 +0000 (03:38 +0200)] 
Uniform date/time loaders naming

Just use the type name with one uppercase, no camels here.

4 years agoComplete C date adapters, text and binary
Daniele Varrazzo [Sun, 16 May 2021 00:24:08 +0000 (02:24 +0200)] 
Complete C date adapters, text and binary

4 years agoAdd C date loader
Daniele Varrazzo [Sat, 15 May 2021 23:27:21 +0000 (01:27 +0200)] 
Add C date loader

8x faster than the Python loader

4 years agoRaise multi-line error messages with context
Daniele Varrazzo [Tue, 1 Jun 2021 23:03:44 +0000 (00:03 +0100)] 
Raise multi-line error messages with context

The context is not available in the diag error. Debugging without it is
pretty hard.

Make sure the error doesn't end with a newline, which is a complaint of
psycopg2.

4 years agoDrop floating point dumping of datetime objects
Daniele Varrazzo [Sun, 16 May 2021 07:33:44 +0000 (09:33 +0200)] 
Drop floating point dumping of datetime objects

The random test found rounding errors around year 2034.

4 years agoMerge branch 'tzinfo'
Daniele Varrazzo [Sun, 16 May 2021 00:52:50 +0000 (02:52 +0200)] 
Merge branch 'tzinfo'

4 years agoStreamline fixed size dumpers by dumping directly in the output buffer
Daniele Varrazzo [Sun, 16 May 2021 00:26:04 +0000 (02:26 +0200)] 
Streamline fixed size dumpers by dumping directly in the output buffer

4 years agoDrop unused import
Daniele Varrazzo [Sun, 16 May 2021 00:25:40 +0000 (02:25 +0200)] 
Drop unused import

4 years agoDon't chain exceptions raising in decimal binary loading
Daniele Varrazzo [Sun, 16 May 2021 00:24:50 +0000 (02:24 +0200)] 
Don't chain exceptions raising in decimal binary loading

4 years agoDon't chain exceptions throwing DataError in dates
Daniele Varrazzo [Sat, 15 May 2021 23:05:04 +0000 (01:05 +0200)] 
Don't chain exceptions throwing DataError in dates

4 years agoLoad dates using slices instead of regexps
Daniele Varrazzo [Sat, 15 May 2021 16:23:46 +0000 (18:23 +0200)] 
Load dates using slices instead of regexps

Some 40% faster. Thanks to @hakib for the tip :)

4 years agoFaster date/timestamp parsing
Daniele Varrazzo [Sat, 15 May 2021 15:38:45 +0000 (17:38 +0200)] 
Faster date/timestamp parsing

4 years agoUniform interval parsing to the other data types
Daniele Varrazzo [Sat, 15 May 2021 14:17:23 +0000 (16:17 +0200)] 
Uniform interval parsing to the other data types

4 years agoBase timestamptz loader on regexp
Daniele Varrazzo [Sat, 15 May 2021 13:41:26 +0000 (15:41 +0200)] 
Base timestamptz loader on regexp

About 4.5x faster than strptime.

4 years agoBase TimeTz loader on regexp
Daniele Varrazzo [Sat, 15 May 2021 12:56:15 +0000 (14:56 +0200)] 
Base TimeTz loader on regexp

4 years agoFurther tweaks to date/time loaders
Daniele Varrazzo [Sat, 15 May 2021 12:55:56 +0000 (14:55 +0200)] 
Further tweaks to date/time loaders

4 years agoBase time loader on regexp
Daniele Varrazzo [Sat, 15 May 2021 11:23:26 +0000 (13:23 +0200)] 
Base time loader on regexp

Some 5x faster

4 years agoBase date and timestamp loaders on regexp instead of strptime
Daniele Varrazzo [Sat, 15 May 2021 10:47:58 +0000 (12:47 +0200)] 
Base date and timestamp loaders on regexp instead of strptime

Not only is 4x faster, but it's more straightforward to port to C
and keep similar algorithms (even if we decide to drop regexp in C)

4 years agoSeparate date/time tests in classes
Daniele Varrazzo [Sat, 15 May 2021 10:45:48 +0000 (12:45 +0200)] 
Separate date/time tests in classes

4 years agoFix datetime timezone adjustment on Python 3.6
Daniele Varrazzo [Fri, 14 May 2021 18:04:23 +0000 (20:04 +0200)] 
Fix datetime timezone adjustment on Python 3.6

4 years agoAdd test for timestamptz timezone load behaviour
Daniele Varrazzo [Fri, 14 May 2021 17:42:29 +0000 (19:42 +0200)] 
Add test for timestamptz timezone load behaviour

4 years agoAdd docs and tests for ConnectionInfo.timezone
Daniele Varrazzo [Fri, 14 May 2021 17:26:37 +0000 (19:26 +0200)] 
Add docs and tests for ConnectionInfo.timezone

4 years agoMove timezone attribute to the Connection.info interface
Daniele Varrazzo [Fri, 14 May 2021 16:37:24 +0000 (18:37 +0200)] 
Move timezone attribute to the Connection.info interface

Type changed to the abstract `datetime.tzinfo` so that we have the
option to return something else such as a `datetime.timezone` with fixed
offset.

Added function to retrieve the object internally from the PGconn.

4 years agoWarn only once about unknown zoneinfo
Daniele Varrazzo [Fri, 14 May 2021 16:19:32 +0000 (18:19 +0200)] 
Warn only once about unknown zoneinfo

4 years agoReturn timestamptz in the connection timezone
Daniele Varrazzo [Fri, 14 May 2021 15:50:06 +0000 (17:50 +0200)] 
Return timestamptz in the connection timezone

First cut, more tests to add. All current tests pass, except the
explicit checks for UTC tzinfo returned.

See https://github.com/psycopg/psycopg3/discussions/56

4 years agoMerge branch 'master' into binary-datetime
Daniele Varrazzo [Fri, 14 May 2021 15:07:03 +0000 (17:07 +0200)] 
Merge branch 'master' into binary-datetime

4 years agoMake the dependency on typing_extension optional
Daniele Varrazzo [Fri, 14 May 2021 14:51:17 +0000 (16:51 +0200)] 
Make the dependency on typing_extension optional

4 years agoReturn timestamptz as utc datetime loading in text format
Daniele Varrazzo [Thu, 13 May 2021 21:04:19 +0000 (23:04 +0200)] 
Return timestamptz as utc datetime loading in text format

This makes text and binary loaders return the same value. It also fixes
the problem of the missing seconds in the timezone offset on Python 3.6.

The opposite choice, returning datetime in the connection timezone on
binary loading, doesn't seem really feasible: pytz cannot parse all the
Postgres timezone values (for instance, "+02:00" can't be parsed): take
a look at
https://www.postgresql.org/docs/13/datetime-posix-timezone-specs.html to
understand how much of a bad idea it would be.

4 years agoFix timestamptz binary tests with seconds in the timezone on Python 3.6
Daniele Varrazzo [Thu, 13 May 2021 16:41:04 +0000 (18:41 +0200)] 
Fix timestamptz binary tests with seconds in the timezone on Python 3.6

4 years agoConsider loss of precision in negative binary timestamps too
Daniele Varrazzo [Thu, 13 May 2021 09:49:33 +0000 (11:49 +0200)] 
Consider loss of precision in negative binary timestamps too

4 years agoAdd timedelta/interval binary adapter
Daniele Varrazzo [Thu, 13 May 2021 02:10:31 +0000 (04:10 +0200)] 
Add timedelta/interval binary adapter

4 years agoAdd timestamptz adapter
Daniele Varrazzo [Thu, 13 May 2021 01:19:11 +0000 (03:19 +0200)] 
Add timestamptz adapter

4 years agoForce text query in tests playing with datestyle
Daniele Varrazzo [Wed, 12 May 2021 23:17:50 +0000 (01:17 +0200)] 
Force text query in tests playing with datestyle

Just in case in the future we switch to binary return values and we stop
testing anything.

4 years agoAdd datetime binary adapter
Daniele Varrazzo [Wed, 12 May 2021 23:13:20 +0000 (01:13 +0200)] 
Add datetime binary adapter

Only for naive datetimes, timezone not handled yet.

4 years agoAdd timetz binary adapters
Daniele Varrazzo [Wed, 12 May 2021 20:44:26 +0000 (22:44 +0200)] 
Add timetz binary adapters

4 years agoAdd time binary adapters
Daniele Varrazzo [Wed, 12 May 2021 19:38:58 +0000 (21:38 +0200)] 
Add time binary adapters

No timezone yet, some tests fail.

4 years agoAdd date binary adapters
Daniele Varrazzo [Wed, 12 May 2021 18:55:40 +0000 (20:55 +0200)] 
Add date binary adapters

4 years agoDrop unused import
Daniele Varrazzo [Wed, 12 May 2021 16:44:22 +0000 (18:44 +0200)] 
Drop unused import

4 years agoMerge branch 'binary-numeric'
Daniele Varrazzo [Mon, 10 May 2021 08:39:19 +0000 (10:39 +0200)] 
Merge branch 'binary-numeric'

4 years agoImprove wording of docker testing notes
Daniele Varrazzo [Mon, 10 May 2021 08:26:00 +0000 (10:26 +0200)] 
Improve wording of docker testing notes

4 years agoAdd example about running tests in docker
Daniele Varrazzo [Sun, 9 May 2021 23:46:30 +0000 (01:46 +0200)] 
Add example about running tests in docker

Writing it down as I have to hack it together every time.

4 years agoFix dumping Decimal of the form 0E+n
Daniele Varrazzo [Sun, 9 May 2021 23:22:12 +0000 (01:22 +0200)] 
Fix dumping Decimal of the form 0E+n

4 years agoFix test for ConnectionInfo.hostaddr and libpq < 12
Daniele Varrazzo [Wed, 5 May 2021 16:32:06 +0000 (18:32 +0200)] 
Fix test for ConnectionInfo.hostaddr and libpq < 12

4 years agoAdd script to test the pool performance
Daniele Varrazzo [Sun, 9 May 2021 23:04:37 +0000 (01:04 +0200)] 
Add script to test the pool performance

4 years agoAdd benchmark test for binary numeric
Daniele Varrazzo [Sun, 9 May 2021 22:52:16 +0000 (00:52 +0200)] 
Add benchmark test for binary numeric

Conclusion is that for the moment it is better to leave default Decimal
dump to text format. Better performance can be obtained accessing the
Python C Decimal structure and perform conversion from there.

4 years agoUse faster C mod operator in Decimal dump
Daniele Varrazzo [Sun, 9 May 2021 21:17:59 +0000 (23:17 +0200)] 
Use faster C mod operator in Decimal dump

4 years agoAdd C implementation of Decimal dumper
Daniele Varrazzo [Sun, 9 May 2021 18:37:51 +0000 (20:37 +0200)] 
Add C implementation of Decimal dumper

4 years agoDon't dump an excess of right 0's in Decimal to numeric
Daniele Varrazzo [Sat, 8 May 2021 17:13:21 +0000 (19:13 +0200)] 
Don't dump an excess of right 0's in Decimal to numeric

4 years agoAdd Cython implementation of NumericBinaryLoader
Daniele Varrazzo [Thu, 6 May 2021 19:22:39 +0000 (21:22 +0200)] 
Add Cython implementation of NumericBinaryLoader

4 years agoAdd C implementation of text decimal adapters
Daniele Varrazzo [Thu, 6 May 2021 17:58:14 +0000 (19:58 +0200)] 
Add C implementation of text decimal adapters

4 years agoAdd dumper for Python int to numeric
Daniele Varrazzo [Thu, 6 May 2021 17:08:20 +0000 (19:08 +0200)] 
Add dumper for Python int to numeric

4 years agoHandle Decimal sNaN value
Daniele Varrazzo [Thu, 6 May 2021 02:26:06 +0000 (04:26 +0200)] 
Handle Decimal sNaN value

4 years agoDump numeric as binary without manipulating the digits tuple
Daniele Varrazzo [Thu, 6 May 2021 01:42:37 +0000 (03:42 +0200)] 
Dump numeric as binary without manipulating the digits tuple

Will make for a much faster algorithm in C

4 years agoDecimal binary dump algorithm streamlined
Daniele Varrazzo [Wed, 5 May 2021 18:20:52 +0000 (20:20 +0200)] 
Decimal binary dump algorithm streamlined

4 years agoAdd Decimal values to random tests
Daniele Varrazzo [Wed, 5 May 2021 17:47:14 +0000 (19:47 +0200)] 
Add Decimal values to random tests

4 years agoAdd prototype for a decimal binary dumper
Daniele Varrazzo [Wed, 5 May 2021 17:15:02 +0000 (19:15 +0200)] 
Add prototype for a decimal binary dumper

The algorithm is pretty unwieldy, maybe it can be refactored. But tests
show that it should be correct.

4 years agoUpgrade testing libraries (#50)
Denis Laxalde [Wed, 5 May 2021 15:02:58 +0000 (17:02 +0200)] 
Upgrade testing libraries (#50)

Use latest versions of testing libraries

pytest is broken on Python 3.10, until version 6.2.4.

4 years agoAdd numeric binary loader
Daniele Varrazzo [Tue, 4 May 2021 21:08:45 +0000 (23:08 +0200)] 
Add numeric binary loader

4 years agoAdd pytest marker to skip tests on certain server versions
Daniele Varrazzo [Tue, 4 May 2021 16:08:30 +0000 (18:08 +0200)] 
Add pytest marker to skip tests on certain server versions

4 years agoFix typo in doc title
Daniele Varrazzo [Sun, 2 May 2021 09:48:53 +0000 (11:48 +0200)] 
Fix typo in doc title

4 years agoCheck documentation build in CI
Denis Laxalde [Tue, 27 Apr 2021 07:49:30 +0000 (09:49 +0200)] 
Check documentation build in CI

4 years agoAdd regression test for the infinity date example
Daniele Varrazzo [Sun, 2 May 2021 00:23:16 +0000 (02:23 +0200)] 
Add regression test for the infinity date example