]> git.ipfire.org Git - thirdparty/sqlalchemy/sqlalchemy.git/commit
rearchitect reflection for batched performance
authorFederico Caselli <cfederico87@gmail.com>
Thu, 14 Oct 2021 19:45:57 +0000 (21:45 +0200)
committerMike Bayer <mike_mp@zzzcomputing.com>
Sat, 18 Jun 2022 18:57:26 +0000 (14:57 -0400)
commitdb08a699489c9b0259579d7ff7fd6bf3496ca3a2
tree741feb8714d9f94f0ddfd03af437f94d2d5a505b
parent964c26feecc7607d6d3a66240c3f33f4ae9215d4
rearchitect reflection for batched performance

Rearchitected the schema reflection API to allow some dialects to make use
of high performing batch queries to reflect the schemas of many tables at
once using much fewer queries. The new performance features are targeted
first at the PostgreSQL and Oracle backends, and may be applied to any
dialect that makes use of SELECT queries against system catalog tables to
reflect tables (currently this omits the MySQL and SQLite dialects which
instead make use of parsing the "CREATE TABLE" statement, however these
dialects do not have a pre-existing performance issue with reflection. MS
SQL Server is still a TODO).

The new API is backwards compatible with the previous system, and should
require no changes to third party dialects to retain compatibility;
third party dialects can also opt into the new system by implementing
batched queries for schema reflection.

Along with this change is an updated reflection API that is fully
:pep:`484` typed, features many new methods and some changes.

Fixes: #4379
Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
48 files changed:
README.unittests.rst
doc/build/changelog/unreleased_20/4379.rst [new file with mode: 0644]
doc/build/changelog/unreleased_20/oracle_views.rst [new file with mode: 0644]
doc/build/changelog/unreleased_20/postgresql_version.rst [new file with mode: 0644]
doc/build/tutorial/metadata.rst
lib/sqlalchemy/dialects/mssql/base.py
lib/sqlalchemy/dialects/mysql/base.py
lib/sqlalchemy/dialects/oracle/base.py
lib/sqlalchemy/dialects/oracle/cx_oracle.py
lib/sqlalchemy/dialects/oracle/dictionary.py [new file with mode: 0644]
lib/sqlalchemy/dialects/oracle/provision.py
lib/sqlalchemy/dialects/oracle/types.py [new file with mode: 0644]
lib/sqlalchemy/dialects/postgresql/__init__.py
lib/sqlalchemy/dialects/postgresql/_psycopg_common.py
lib/sqlalchemy/dialects/postgresql/asyncpg.py
lib/sqlalchemy/dialects/postgresql/base.py
lib/sqlalchemy/dialects/postgresql/pg8000.py
lib/sqlalchemy/dialects/postgresql/pg_catalog.py [new file with mode: 0644]
lib/sqlalchemy/dialects/postgresql/types.py [new file with mode: 0644]
lib/sqlalchemy/dialects/sqlite/base.py
lib/sqlalchemy/engine/__init__.py
lib/sqlalchemy/engine/default.py
lib/sqlalchemy/engine/interfaces.py
lib/sqlalchemy/engine/reflection.py
lib/sqlalchemy/sql/base.py
lib/sqlalchemy/sql/cache_key.py
lib/sqlalchemy/sql/schema.py
lib/sqlalchemy/testing/assertions.py
lib/sqlalchemy/testing/plugin/pytestplugin.py
lib/sqlalchemy/testing/provision.py
lib/sqlalchemy/testing/requirements.py
lib/sqlalchemy/testing/schema.py
lib/sqlalchemy/testing/suite/test_reflection.py
lib/sqlalchemy/testing/suite/test_sequence.py
lib/sqlalchemy/testing/util.py
lib/sqlalchemy/util/topological.py
lib/sqlalchemy/util/typing.py
pyproject.toml
setup.cfg
test/dialect/mysql/test_reflection.py
test/dialect/oracle/test_reflection.py
test/dialect/postgresql/test_dialect.py
test/dialect/postgresql/test_reflection.py
test/dialect/postgresql/test_types.py
test/dialect/test_sqlite.py
test/engine/test_reflection.py
test/perf/many_table_reflection.py [new file with mode: 0644]
test/requirements.py