]> git.ipfire.org Git - thirdparty/psycopg.git/log
thirdparty/psycopg.git
5 years agoAdded a few module headers
Daniele Varrazzo [Thu, 14 May 2020 18:02:46 +0000 (06:02 +1200)] 
Added a few module headers

5 years agoRowLoader moved from adapt to transform module
Daniele Varrazzo [Thu, 14 May 2020 17:57:02 +0000 (05:57 +1200)] 
RowLoader moved from adapt to transform module

It's only used by the Transformer and it doesn't need to be exposed.

5 years agoOptimized dumpers registration moved to own modules
Daniele Varrazzo [Thu, 14 May 2020 07:49:40 +0000 (19:49 +1200)] 
Optimized dumpers registration moved to own modules

Dropped some of the function signatures from the .pyd files as they are
not used cross-module.

5 years agoTest both the c and the python implementation on tox
Daniele Varrazzo [Mon, 11 May 2020 06:35:58 +0000 (18:35 +1200)] 
Test both the c and the python implementation on tox

5 years agoSkip hostaddr test with libpq < 12 too for the moment
Daniele Varrazzo [Mon, 11 May 2020 06:54:24 +0000 (18:54 +1200)] 
Skip hostaddr test with libpq < 12 too for the moment

5 years agomypy test on setup.py dropped
Daniele Varrazzo [Mon, 11 May 2020 06:23:48 +0000 (18:23 +1200)] 
mypy test on setup.py dropped

Types info for distutils are utterly broken.

5 years agoInclude cython in the sdist package
Daniele Varrazzo [Mon, 11 May 2020 05:34:22 +0000 (17:34 +1200)] 
Include cython in the sdist package

Also fixed everything else that should be needed to test the C
implementation on travis.

5 years agoDropped utils.typing in favour of proto package
Daniele Varrazzo [Mon, 11 May 2020 04:50:45 +0000 (16:50 +1200)] 
Dropped utils.typing in favour of proto package

5 years agoAdded protocol for different implementations of Transform
Daniele Varrazzo [Mon, 11 May 2020 04:33:29 +0000 (16:33 +1200)] 
Added protocol for different implementations of Transform

mypy passes all the ckecks.

5 years agoDropped unknown loader, using text/bytea ones
Daniele Varrazzo [Sun, 10 May 2020 16:23:55 +0000 (04:23 +1200)] 
Dropped unknown loader, using text/bytea ones

5 years agoAdded c bytea loaders
Daniele Varrazzo [Sun, 10 May 2020 15:44:41 +0000 (03:44 +1200)] 
Added c bytea loaders

5 years agoAdded proper binary integer decoding in cython
Daniele Varrazzo [Sun, 10 May 2020 10:30:36 +0000 (22:30 +1200)] 
Added proper binary integer decoding in cython

5 years agoAdded mapping from python Loader to C and context extraction
Daniele Varrazzo [Sun, 10 May 2020 09:05:54 +0000 (21:05 +1200)] 
Added mapping from python Loader to C and context extraction

5 years agoRowLoader definition shared into adapt
Daniele Varrazzo [Fri, 8 May 2020 12:43:39 +0000 (00:43 +1200)] 
RowLoader definition shared into adapt

5 years agoBaby steps registering c optimised adapters
Daniele Varrazzo [Mon, 4 May 2020 18:26:26 +0000 (06:26 +1200)] 
Baby steps registering c optimised adapters

Just registered some so far, not using the map yet

5 years agoKeep pq_cython as an extension separate from the rest
Daniele Varrazzo [Mon, 4 May 2020 16:44:45 +0000 (04:44 +1200)] 
Keep pq_cython as an extension separate from the rest

5 years agoMore consistent names for numeric adaptation functions
Daniele Varrazzo [Mon, 4 May 2020 16:09:26 +0000 (04:09 +1200)] 
More consistent names for numeric adaptation functions

5 years agoSkip flake check of the .eggs dir
Daniele Varrazzo [Thu, 23 Apr 2020 05:02:41 +0000 (17:02 +1200)] 
Skip flake check of the .eggs dir

5 years agoDon't import Python API function from the main cpython package
Daniele Varrazzo [Thu, 23 Apr 2020 04:29:35 +0000 (16:29 +1200)] 
Don't import Python API function from the main cpython package

Importing from there seems deprecated.

[skip ci]

5 years agoFixed import of optimized Transformer
Daniele Varrazzo [Thu, 23 Apr 2020 04:28:32 +0000 (16:28 +1200)] 
Fixed import of optimized Transformer

Import it if the C module is used, not just if it's available.

5 years agoAvoid creating an intermadiate tuple
Daniele Varrazzo [Wed, 22 Apr 2020 16:55:11 +0000 (04:55 +1200)] 
Avoid creating an intermadiate tuple

We are now around 18x the basic Python performance

[skip ci] Certain type conversion are stubbed out, so tests won't pass.

5 years agoAn ounce faster by using an array of loaders instead of a list
Daniele Varrazzo [Wed, 22 Apr 2020 16:36:37 +0000 (04:36 +1200)] 
An ounce faster by using an array of loaders instead of a list

5 years agoAdded some C decoders stubs
Daniele Varrazzo [Wed, 22 Apr 2020 15:50:22 +0000 (03:50 +1200)] 
Added some C decoders stubs

The speedup compared to the pure Python implementation is now 16x.

5 years agoPlaying around using a static definition for PQresult.
Daniele Varrazzo [Sun, 19 Apr 2020 10:02:42 +0000 (22:02 +1200)] 
Playing around using a static definition for PQresult.

5 years agoAdded basic cython implementation of the Transform object
Daniele Varrazzo [Sun, 19 Apr 2020 08:28:59 +0000 (20:28 +1200)] 
Added basic cython implementation of the Transform object

Just an about straight conversion from Python

5 years agopq objects defined as proto rather than stub
Daniele Varrazzo [Sun, 19 Apr 2020 00:00:05 +0000 (12:00 +1200)] 
pq objects defined as proto rather than stub

5 years agoAdded result to transformer, fetching data from there
Daniele Varrazzo [Sat, 18 Apr 2020 06:22:54 +0000 (18:22 +1200)] 
Added result to transformer, fetching data from there

5 years agoSome ctypes tweaking to create better C
Daniele Varrazzo [Fri, 17 Apr 2020 08:42:38 +0000 (20:42 +1200)] 
Some ctypes tweaking to create better C

5 years agoSmall fixes to the ctypes module
Daniele Varrazzo [Thu, 16 Apr 2020 14:18:15 +0000 (02:18 +1200)] 
Small fixes to the ctypes module

5 years agoAdded conforming psycopg3.pq implementation in cython
Daniele Varrazzo [Thu, 16 Apr 2020 14:04:23 +0000 (02:04 +1200)] 
Added conforming psycopg3.pq implementation in cython

5 years agoPq fixtures in their own module, report the implementation being tested.
Daniele Varrazzo [Thu, 16 Apr 2020 14:18:36 +0000 (02:18 +1200)] 
Pq fixtures in their own module, report the implementation being tested.

5 years agoadded __impl__ constant to ctypes pq module
Daniele Varrazzo [Thu, 16 Apr 2020 14:17:52 +0000 (02:17 +1200)] 
added __impl__ constant to ctypes pq module

5 years agoMinor optimizations to data fetching in the cursor
Daniele Varrazzo [Tue, 14 Apr 2020 11:11:36 +0000 (23:11 +1200)] 
Minor optimizations to data fetching in the cursor

Shaved a few percents here and there out of profile indications.

5 years agoAdded async implementation for executemany, fetchmany, fetchall
Daniele Varrazzo [Tue, 14 Apr 2020 07:03:02 +0000 (19:03 +1200)] 
Added async implementation for executemany, fetchmany, fetchall

5 years agoDropped tempenv fixture
Daniele Varrazzo [Tue, 14 Apr 2020 06:48:12 +0000 (18:48 +1200)] 
Dropped tempenv fixture

The pytest monkeypatch fixture does it

5 years agoFixed connect parameters
Daniele Varrazzo [Tue, 14 Apr 2020 06:37:13 +0000 (18:37 +1200)] 
Fixed connect parameters

Raise an error with no params, accept keywords only.

5 years agoconnection and cursor's close() made async
Daniele Varrazzo [Tue, 14 Apr 2020 05:40:53 +0000 (17:40 +1200)] 
connection and cursor's close() made async

Dropped close() on __del__ for cursor: it is of no utility. The
important thing is that PGresult frees its memory and that happens at
PGresult's own __del__.

5 years agoDefine connection.close as asynchronous
Daniele Varrazzo [Tue, 14 Apr 2020 05:34:02 +0000 (17:34 +1200)] 
Define connection.close as asynchronous

5 years agoFixed dispatcher -> dispchar typo
Daniele Varrazzo [Mon, 13 Apr 2020 08:51:05 +0000 (20:51 +1200)] 
Fixed dispatcher -> dispchar typo

5 years agoAdded stub for the pq package
Daniele Varrazzo [Sun, 12 Apr 2020 13:52:30 +0000 (01:52 +1200)] 
Added stub for the pq package

Needed as the dynamic import of the modle confuses mypy

5 years agoAdded helper object to convert Python query into a Postgres query
Daniele Varrazzo [Sun, 12 Apr 2020 08:44:43 +0000 (20:44 +1200)] 
Added helper object to convert Python query into a Postgres query

The object helps keeping some state between preparing and executing,
useful in executemany so preparation can happen only once.

5 years agoAdded first implementation of executemany based on prepared queries
Daniele Varrazzo [Sat, 11 Apr 2020 15:55:32 +0000 (03:55 +1200)] 
Added first implementation of executemany based on prepared queries

There's a lot of repetition here

5 years agoAdded async query prepare functions in pq wrapper
Daniele Varrazzo [Sat, 11 Apr 2020 15:54:59 +0000 (03:54 +1200)] 
Added async query prepare functions in pq wrapper

5 years agoexec generator split in send and receive
Daniele Varrazzo [Sat, 11 Apr 2020 15:41:11 +0000 (03:41 +1200)] 
exec generator split in send and receive

It was actually not needed: I thought erroneously that sending prepared
statements only needed the send bit. However the change is ok and could
also clean up the types around these generator adding a generic PQGen[]
type (with different return values).

5 years agocursor.conn renamed to connection
Daniele Varrazzo [Sat, 11 Apr 2020 13:00:56 +0000 (01:00 +1200)] 
cursor.conn renamed to connection

5 years agolibpq protocol generators moved to their own module
Daniele Varrazzo [Sat, 11 Apr 2020 12:55:10 +0000 (00:55 +1200)] 
libpq protocol generators moved to their own module

5 years agoAdded cursor.status property and exposing ExecStatus on the cursor
Daniele Varrazzo [Sat, 11 Apr 2020 11:38:18 +0000 (23:38 +1200)] 
Added cursor.status property and exposing ExecStatus on the cursor

5 years agoAdded connection.status property
Daniele Varrazzo [Sat, 11 Apr 2020 11:36:16 +0000 (23:36 +1200)] 
Added connection.status property

Exposing ConnStatus and TransactionStatus on the connection object.

5 years agoAdded .close() and .closed on connection and cursor
Daniele Varrazzo [Sat, 11 Apr 2020 09:12:43 +0000 (21:12 +1200)] 
Added .close() and .closed on connection and cursor

5 years agoAdded connection status check on send_query
Daniele Varrazzo [Sat, 11 Apr 2020 09:01:46 +0000 (21:01 +1200)] 
Added connection status check on send_query

5 years agoAdded test to verify pq.error_message() is sane on null args
Daniele Varrazzo [Sat, 11 Apr 2020 08:33:37 +0000 (20:33 +1200)] 
Added test to verify pq.error_message() is sane on null args

5 years agoEnsure a valid connection with escaping functions
Daniele Varrazzo [Sat, 11 Apr 2020 08:23:28 +0000 (20:23 +1200)] 
Ensure a valid connection with escaping functions

5 years agoGuard for null connections in exec functions
Daniele Varrazzo [Sat, 11 Apr 2020 08:13:58 +0000 (20:13 +1200)] 
Guard for null connections in exec functions

5 years agoAdded test for idempotent PGresult.clear()
Daniele Varrazzo [Sat, 11 Apr 2020 08:01:10 +0000 (20:01 +1200)] 
Added test for idempotent PGresult.clear()

5 years agoreset and poll fail too if the connection is gone
Daniele Varrazzo [Sat, 11 Apr 2020 07:58:39 +0000 (19:58 +1200)] 
reset and poll fail too if the connection is gone

5 years agoMore care with functions returning null in PGconn methods
Daniele Varrazzo [Sat, 11 Apr 2020 07:48:48 +0000 (19:48 +1200)] 
More care with functions returning null in PGconn methods

ctypes stub changed: by default the function returning bytes can return
NULL. The ones which never do it are few, so these are the ones which
get special-cased.

5 years agoMore care in None return values from libpq result functions
Daniele Varrazzo [Sat, 11 Apr 2020 06:03:26 +0000 (18:03 +1200)] 
More care in None return values from libpq result functions

5 years agoAdded barebone implementation of dbapi 2.0
Daniele Varrazzo [Sat, 11 Apr 2020 04:54:47 +0000 (16:54 +1200)] 
Added barebone implementation of dbapi 2.0

Several methods not tested yet, fetch method not added yet to async
cursor.

5 years agoFixed some shenanigan around connection params
Daniele Varrazzo [Sat, 11 Apr 2020 02:20:15 +0000 (14:20 +1200)] 
Fixed some shenanigan around connection params

The libpq function return None for an invalid connection: take that into
account in the function signature and usage.

Dropped asymmetry between sync and async connection for the encoding
setter: use for both a set_client_encoding() function. This way it's
easy to keep the getter on the base class.

5 years agoThe big rename: adapters -> dumpers, typecasters -> loaders.
Daniele Varrazzo [Thu, 9 Apr 2020 07:12:52 +0000 (19:12 +1200)] 
The big rename: adapters -> dumpers, typecasters -> loaders.

5 years agoFixed composite info query to work for pg <= 10
Daniele Varrazzo [Thu, 9 Apr 2020 00:41:56 +0000 (12:41 +1200)] 
Fixed composite info query to work for pg <= 10

creating a row() on the fly caused the error "record type has not been
registered"

5 years agoAdded tuple text adaptation
Daniele Varrazzo [Wed, 8 Apr 2020 15:52:15 +0000 (03:52 +1200)] 
Added tuple text adaptation

It cannot return a record oid, Postgres is not happy to parse it, so
return it as text.

5 years agoFixed formats in query with repeated arguments
Daniele Varrazzo [Wed, 8 Apr 2020 15:48:11 +0000 (03:48 +1200)] 
Fixed formats in query with repeated arguments

Fixed test that wasn't checking the format.

5 years agoAdded composite.fetch_info_async
Daniele Varrazzo [Wed, 8 Apr 2020 10:10:45 +0000 (22:10 +1200)] 
Added composite.fetch_info_async

5 years agofetchone() made async on async cursor
Daniele Varrazzo [Wed, 8 Apr 2020 10:10:12 +0000 (22:10 +1200)] 
fetchone() made async on async cursor

5 years agoregister_array() renamed to array.register()
Daniele Varrazzo [Wed, 8 Apr 2020 09:59:47 +0000 (21:59 +1200)] 
register_array() renamed to array.register()

5 years agoFixed registration scope of composite array adapters
Daniele Varrazzo [Wed, 8 Apr 2020 09:55:37 +0000 (21:55 +1200)] 
Fixed registration scope of composite array adapters

5 years agoAdded typecasting of composite types
Daniele Varrazzo [Wed, 8 Apr 2020 09:00:16 +0000 (21:00 +1200)] 
Added typecasting of composite types

5 years agoAdded cast from varchar, bpchar, name
Daniele Varrazzo [Wed, 8 Apr 2020 06:15:36 +0000 (18:15 +1200)] 
Added cast from varchar, bpchar, name

Note that bpchar (i.e. the output format for "char" - with quotes) and
name are always decoded, even if the db is SQL_ASCII, because these
types are mostly used in system catalogs and there should be no such
crap as unencded data there.

5 years agoAdded tests for all numeric types arrays
Daniele Varrazzo [Tue, 7 Apr 2020 14:30:33 +0000 (02:30 +1200)] 
Added tests for all numeric types arrays

5 years agoDropped ArrayCaster, expose register_array function instead
Daniele Varrazzo [Tue, 7 Apr 2020 12:39:37 +0000 (00:39 +1200)] 
Dropped ArrayCaster, expose register_array function instead

The function only associates array_oid -> base_oid. Looking up the right
typecaster happens at cast time: this way composite adapters (record,
list) can use the caster register in the right context.

All the types for which there is an adapter gets an array caster now.

5 years agoReally use the context to transform components types
Daniele Varrazzo [Tue, 7 Apr 2020 11:18:59 +0000 (23:18 +1200)] 
Really use the context to transform components types

It works for arrays and records, except for text arrays because the
decorator registers the TypeCaster directly, not the oid. This must
change, we need a different decorator/registration.

5 years agoThe Transformer is an adaptation context
Daniele Varrazzo [Tue, 7 Apr 2020 10:24:09 +0000 (22:24 +1200)] 
The Transformer is an adaptation context

Dropped cursor from Adapter, TypeCaster, Transformer. Added adapters and
casters map on Trasformer.

Solves a problem of customization in composite types: if a type is
customized now composite types containing that type use the same
customization. Also fixed a circular reference between transformer and
cursor.

5 years agoWork around mypy error of Format not exposed by psycopg3.adapt
Daniele Varrazzo [Tue, 7 Apr 2020 09:26:49 +0000 (21:26 +1200)] 
Work around mypy error of Format not exposed by psycopg3.adapt

5 years agoAdded binary cast of record type
Daniele Varrazzo [Tue, 7 Apr 2020 09:23:42 +0000 (21:23 +1200)] 
Added binary cast of record type

5 years agoAdded Transformer.cast_sequence() in place of cast_row()
Daniele Varrazzo [Tue, 7 Apr 2020 08:51:52 +0000 (20:51 +1200)] 
Added Transformer.cast_sequence() in place of cast_row()

The function is more generic and the Transformer doesn't have to wrap a
PGresult anymore.

5 years agoAdded text cast of records
Daniele Varrazzo [Tue, 7 Apr 2020 06:44:04 +0000 (18:44 +1200)] 
Added text cast of records

5 years agoDropped accepting None as Transformer.cast() input
Daniele Varrazzo [Tue, 7 Apr 2020 06:43:00 +0000 (18:43 +1200)] 
Dropped accepting None as Transformer.cast() input

More efficient if we only accept non-null results.

5 years agoLess global objects in types.array module
Daniele Varrazzo [Tue, 7 Apr 2020 03:26:28 +0000 (15:26 +1200)] 
Less global objects in types.array module

5 years agoAdded xfailing tests to remember what is left to implement
Daniele Varrazzo [Sun, 5 Apr 2020 05:37:05 +0000 (17:37 +1200)] 
Added xfailing tests to remember what is left to implement

5 years agoCleanup of the array adapter algorithm
Daniele Varrazzo [Sun, 5 Apr 2020 04:44:31 +0000 (16:44 +1200)] 
Cleanup of the array adapter algorithm

5 years agoCheck that arrays contain consistent objects.
Daniele Varrazzo [Sun, 5 Apr 2020 02:07:04 +0000 (14:07 +1200)] 
Check that arrays contain consistent objects.

5 years agoDropped spurious comment
Daniele Varrazzo [Sun, 5 Apr 2020 01:20:21 +0000 (13:20 +1200)] 
Dropped spurious comment

5 years agoAdded adaptation of binary arrays
Daniele Varrazzo [Sun, 5 Apr 2020 01:10:44 +0000 (13:10 +1200)] 
Added adaptation of binary arrays

Added binary array adaptation for test types and some numeric types.

5 years agoConsistent name of 'format' parameter
Daniele Varrazzo [Sat, 4 Apr 2020 15:27:12 +0000 (03:27 +1200)] 
Consistent name of 'format' parameter

Also really use function caches in Transformer

5 years agoConsistent 'src' name for the first argument of the adapters
Daniele Varrazzo [Sat, 4 Apr 2020 15:14:08 +0000 (03:14 +1200)] 
Consistent 'src' name for the first argument of the adapters

'cls' is messy when it comes to class methods.

5 years agoAdded cast of binary arrays
Daniele Varrazzo [Sat, 4 Apr 2020 14:12:10 +0000 (02:12 +1200)] 
Added cast of binary arrays

5 years agoFixed typing typo
Daniele Varrazzo [Sat, 4 Apr 2020 09:19:21 +0000 (22:19 +1300)] 
Fixed typing typo

5 years agoAdaptation context transferred to the adapters objects
Daniele Varrazzo [Fri, 3 Apr 2020 14:23:33 +0000 (03:23 +1300)] 
Adaptation context transferred to the adapters objects

5 years agoUse classmethod instead of staticmethod to play better with inheritance
Daniele Varrazzo [Fri, 3 Apr 2020 13:04:35 +0000 (02:04 +1300)] 
Use classmethod instead of staticmethod to play better with inheritance

5 years agoAdded array tests for sql_ascii db and bytea
Daniele Varrazzo [Fri, 3 Apr 2020 12:53:35 +0000 (01:53 +1300)] 
Added array tests for sql_ascii db and bytea

5 years agoFixed TypeCasterType and AdapterType types
Daniele Varrazzo [Fri, 3 Apr 2020 11:58:34 +0000 (00:58 +1300)] 
Fixed TypeCasterType and AdapterType types

The previous signature meant an instance of Adapter/TypeCaster or a function,
but we meant a class actually. The error costed some cursing and some
casting to the right type (in mypy sense, not in psyco sense)

5 years agoAdded wrapper for escape_bytea with no connection
Daniele Varrazzo [Fri, 3 Apr 2020 11:57:21 +0000 (00:57 +1300)] 
Added wrapper for escape_bytea with no connection

Changed unescape_bytea to be a normal method. The pgconn is now
optional.

5 years agoTypecaster renamed to TypeCaster
Daniele Varrazzo [Fri, 3 Apr 2020 11:19:46 +0000 (00:19 +1300)] 
Typecaster renamed to TypeCaster

Because I have ArrayCaster, will have RangeCaster etc.

5 years agoAdded basic array adaptation infrastructure
Daniele Varrazzo [Fri, 3 Apr 2020 11:17:31 +0000 (00:17 +1300)] 
Added basic array adaptation infrastructure

5 years agoFixed names of wrapper functions
Daniele Varrazzo [Fri, 3 Apr 2020 11:09:16 +0000 (00:09 +1300)] 
Fixed names of wrapper functions

5 years agoAdded object to map oid and data types
Daniele Varrazzo [Fri, 3 Apr 2020 00:27:59 +0000 (13:27 +1300)] 
Added object to map oid and data types

5 years agoAdded array delimiter to the data types info
Daniele Varrazzo [Thu, 2 Apr 2020 23:32:58 +0000 (12:32 +1300)] 
Added array delimiter to the data types info

5 years agoCleanup of the encodings table and lookup
Daniele Varrazzo [Thu, 2 Apr 2020 15:09:51 +0000 (04:09 +1300)] 
Cleanup of the encodings table and lookup

5 years agoHandle sql_ascii encoding as binary
Daniele Varrazzo [Thu, 2 Apr 2020 14:57:09 +0000 (03:57 +1300)] 
Handle sql_ascii encoding as binary