]> git.ipfire.org Git - thirdparty/sqlalchemy/sqlalchemy.git/commit
Add support for server side cursors to mysqldb and pymysql
authorRoman Podoliaka <roman.podoliaka@gmail.com>
Thu, 3 Nov 2016 22:31:05 +0000 (00:31 +0200)
committerMike Bayer <mike_mp@zzzcomputing.com>
Thu, 10 Nov 2016 17:09:27 +0000 (12:09 -0500)
commitd1e31ab1582e2d9275c70a89b72efc2a8651df3f
tree36518daa1d1a2468ba2d68e903b233978afbcab2
parent6a688b736429e27a892bc02111414491fe4103b0
Add support for server side cursors to mysqldb and pymysql

This allows to skip buffering of the results on the client side, e.g.
the following snippet:

    table = sa.Table(
        'testtbl', sa.MetaData(),
        sa.Column('id', sa.Integer, primary_key=True),
        sa.Column('a', sa.Integer),
        sa.Column('b', sa.String(512))
    )
    table.create(eng, checkfirst=True)

    with eng.connect() as conn:
        result = conn.execute(table.select().limit(1)).fetchone()
        if result is None:
            for _ in range(1000):
                conn.execute(
                    table.insert(),
                    [{'a': random.randint(1, 100000),
                      'b': ''.join(random.choice(string.ascii_letters) for _ in range(100))}
                      for _ in range(1000)]
                )

    with eng.connect() as conn:
        for row in conn.execution_options(stream_results=True).execute(table.select()):
            pass

now uses ~23 MB of memory instead of ~327 MB on CPython 3.5.2 and
PyMySQL 0.7.9.

psycopg2 implementation and execution options (stream_results,
server_side_cursors) are reused.

Change-Id: I4dc23ce3094f027bdff51b896b050361991c62e2
doc/build/changelog/changelog_11.rst
lib/sqlalchemy/dialects/mysql/base.py
lib/sqlalchemy/dialects/mysql/mysqldb.py
lib/sqlalchemy/dialects/mysql/pymysql.py
lib/sqlalchemy/dialects/postgresql/psycopg2.py
lib/sqlalchemy/engine/base.py
lib/sqlalchemy/engine/default.py
lib/sqlalchemy/orm/query.py
lib/sqlalchemy/testing/requirements.py
lib/sqlalchemy/testing/suite/test_results.py
test/dialect/postgresql/test_query.py