--- /dev/null
+.. change::
+ :tags: bug, sql
+ :tickets: 7394
+
+ Custom SQL elements, third party dialects, custom or third party datatypes
+ will all generate consistent warnings when they do not clearly opt in or
+ out of SQL statement caching, which is achieved by setting the appropriate
+ attributes on each type of class. The warning links to documentation
+ sections which indicate the appropriate approach for each type of object in
+ order for caching to be enabled.
+
+.. change::
+ :tags: bug, sql
+ :tickets: 7394
+
+ Fixed missing caching directives for a few lesser used classes in SQL Core
+ which would cause ``[no key]`` to be logged for elements which made use of
+ these.
+
+.. change::
+ :tags: bug, postgresql
+ :tickets: 7394
+
+ Fixed missing caching directives for :class:`_postgresql.hstore` and
+ :class:`_postgresql.array` constructs which would cause ``[no key]``
+ to be logged for these elements.
+
+.. change::
+ :tags: bug, orm
+ :tickets: 7394
+
+ User defined ORM options, such as those illustrated in the dogpile.caching
+ example which subclass :class:`_orm.UserDefinedOption`, by definition are
+ handled on every statement execution and do not need to be considered as
+ part of the cache key for the statement. Caching of the base
+ :class:`.ExecutableOption` class has been modified so that it is no longer
+ a :class:`.HasCacheKey` subclass directly, so that the presence of user
+ defined option objects will not have the unwanted side effect of disabling
+ statement caching. Only ORM specific loader and criteria options, which are
+ all internal to SQLAlchemy, now participate within the caching system.
+
+.. change::
+ :tags: bug, orm
+ :tickets: 7394
+
+ Fixed issue where mappings that made use of :func:`_orm.synonym` and
+ potentially other kinds of "proxy" attributes would not in all cases
+ successfully generate a cache key for their SQL statements, leading to
+ degraded performance for those statements.
\ No newline at end of file
cache's behavior, described in the next section.
+.. _sql_caching_logging:
+
Estimating Cache Performance Using Logging
------------------------------------------
Caching for Third Party Dialects
---------------------------------
-The caching feature requires that the dialect's compiler produces a SQL
-construct that is generically reusable given a particular cache key. This means
+The caching feature requires that the dialect's compiler produces SQL
+strings that are safe to reuse for many statement invocations, given
+a particular cache key that is keyed to that SQL string. This means
that any literal values in a statement, such as the LIMIT/OFFSET values for
a SELECT, can not be hardcoded in the dialect's compilation scheme, as
the compiled string will not be re-usable. SQLAlchemy supports rendered
bound parameters using the :meth:`_sql.BindParameter.render_literal_execute`
method which can be applied to the existing ``Select._limit_clause`` and
-``Select._offset_clause`` attributes by a custom compiler.
-
-As there are many third party dialects, many of which may be generating
-literal values from SQL statements without the benefit of the newer "literal execute"
-feature, SQLAlchemy as of version 1.4.5 has added a flag to dialects known as
-:attr:`_engine.Dialect.supports_statement_cache`. This flag is tested to be present
-directly on a dialect class, and not any superclasses, so that even a third
-party dialect that subclasses an existing cacheable SQLAlchemy dialect such
-as ``sqlalchemy.dialects.postgresql.PGDialect`` must still specify this flag,
+``Select._offset_clause`` attributes by a custom compiler, which
+are illustrated later in this section.
+
+As there are many third party dialects, many of which may be generating literal
+values from SQL statements without the benefit of the newer "literal execute"
+feature, SQLAlchemy as of version 1.4.5 has added an attribute to dialects
+known as :attr:`_engine.Dialect.supports_statement_cache`. This attribute is
+checked at runtime for its presence directly on a particular dialect's class,
+even if it's already present on a superclass, so that even a third party
+dialect that subclasses an existing cacheable SQLAlchemy dialect such as
+``sqlalchemy.dialects.postgresql.PGDialect`` must still explicitly include this
+attribute for caching to be enabled. The attribute should **only** be enabled
once the dialect has been altered as needed and tested for reusability of
compiled SQL statements with differing parameters.
-For all third party dialects that don't support this flag, the logging for
-such a dialect will indicate ``dialect does not support caching``. Dialect
-authors can apply the flag as follows::
+For all third party dialects that don't support this attribute, the logging for
+such a dialect will indicate ``dialect does not support caching``.
+
+When a dialect has been tested against caching, and in particular the SQL
+compiler has been updated to not render any literal LIMIT / OFFSET within
+a SQL string directly, dialect authors can apply the attribute as follows::
from sqlalchemy.engine.default import DefaultDialect
.. versionadded:: 1.4.5
+ Added the :attr:`.Dialect.supports_statement_cache` attribute.
+
+The typical case for dialect modification follows.
+
+Example: Rendering LIMIT / OFFSET with post compile parameters
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+As an example, suppose a dialect overrides the :meth:`.SQLCompiler.limit_clause`
+method, which produces the "LIMIT / OFFSET" clause for a SQL statement,
+like this::
+
+ # pre 1.4 style code
+ def limit_clause(self, select, **kw):
+ text = ""
+ if select._limit is not None:
+ text += " \n LIMIT %d" % (select._limit, )
+ if select._offset is not None:
+ text += " \n OFFSET %d" % (select._offset, )
+ return text
+
+The above routine renders the :attr:`.Select._limit` and
+:attr:`.Select._offset` integer values as literal integers embedded in the SQL
+statement. This is a common requirement for databases that do not support using
+a bound parameter within the LIMIT/OFFSET clauses of a SELECT statement.
+However, rendering the integer value within the initial compilation stage is
+directly **incompatible** with caching as the limit and offset integer values
+of a :class:`.Select` object are not part of the cache key, so that many
+:class:`.Select` statements with different limit/offset values would not render
+with the correct value.
+
+The correction for the above code is to move the literal integer into
+SQLAlchemy's :ref:`post-compile <change_4808>` facility, which will render the
+literal integer outside of the initial compilation stage, but instead at
+execution time before the statement is sent to the DBAPI. This is accessed
+within the compilation stage using the :meth:`_sql.BindParameter.render_literal_execute`
+method, in conjunction with using the :attr:`.Select._limit_clause` and
+:attr:`.Select._offset_clause` attributes, which represent the LIMIT/OFFSET
+as a complete SQL expression, as follows::
+
+ # 1.4 cache-compatible code
+ def limit_clause(self, select, **kw):
+ text = ""
+
+ limit_clause = select._limit_clause
+ offset_clause = select._offset_clause
+
+ if select._simple_int_clause(limit_clause):
+ text += " \n LIMIT %s" % (
+ self.process(limit_clause.render_literal_execute(), **kw)
+ )
+ elif limit_clause is not None:
+ # assuming the DB doesn't support SQL expressions for LIMIT.
+ # Otherwise render here normally
+ raise exc.CompileError(
+ "dialect 'mydialect' can only render simple integers for LIMIT"
+ )
+ if select._simple_int_clause(offset_clause):
+ text += " \n OFFSET %s" % (
+ self.process(offset_clause.render_literal_execute(), **kw)
+ )
+ elif offset_clause is not None:
+ # assuming the DB doesn't support SQL expressions for OFFSET.
+ # Otherwise render here normally
+ raise exc.CompileError(
+ "dialect 'mydialect' can only render simple integers for OFFSET"
+ )
+
+ return text
+
+The approach above will generate a compiled SELECT statement that looks like::
+
+ SELECT x FROM y
+ LIMIT __[POSTCOMPILE_param_1]
+ OFFSET __[POSTCOMPILE_param_2]
+
+Where above, the ``__[POSTCOMPILE_param_1]`` and ``__[POSTCOMPILE_param_2]``
+indicators will be populated with their corresponding integer values at
+statement execution time, after the SQL string has been retrieved from the
+cache.
+
+After changes like the above have been made as appropriate, the
+:attr:`.Dialect.supports_statement_cache` flag should be set to ``True``.
+It is strongly recommended that third party dialects make use of the
+`dialect third party test suite <https://github.com/sqlalchemy/sqlalchemy/blob/main/README.dialects.rst>`_
+which will assert that operations like
+SELECTs with LIMIT/OFFSET are correctly rendered and cached.
+
+.. seealso::
+
+ :ref:`faq_new_caching` - in the :ref:`faq_toplevel` section
.. _engine_lambda_caching:
.. toctree::
:maxdepth: 3
+ foundation
sqlelement
operators
selectable
--- /dev/null
+.. _core_foundation_toplevel:
+
+=================================================
+SQL Expression Language Foundational Constructs
+=================================================
+
+Base classes and mixins that are used to compose SQL Expression Language
+elements.
+
+.. currentmodule:: sqlalchemy.sql.expression
+
+.. autoclass:: CacheKey
+ :members:
+
+.. autoclass:: ClauseElement
+ :members:
+ :inherited-members:
+
+
+.. autoclass:: sqlalchemy.sql.base.DialectKWArgs
+ :members:
+
+
+.. autoclass:: sqlalchemy.sql.traversals.HasCacheKey
+ :members:
+
+.. autoclass:: LambdaElement
+ :members:
+
+.. autoclass:: StatementLambdaElement
+ :members:
+
.. autoclass:: BindParameter
:members:
-.. autoclass:: CacheKey
- :members:
-
.. autoclass:: Case
:members:
.. autoclass:: Cast
:members:
-.. autoclass:: ClauseElement
- :members:
- :inherited-members:
-
-
.. autoclass:: ClauseList
:members:
:special-members:
:inherited-members:
-.. autoclass:: sqlalchemy.sql.base.DialectKWArgs
- :members:
.. autoclass:: Extract
:members:
.. autoclass:: Label
:members:
-.. autoclass:: LambdaElement
- :members:
-
.. autoclass:: Null
:members:
.. autoclass:: Over
:members:
-.. autoclass:: StatementLambdaElement
- :members:
-
.. autoclass:: TextClause
:members:
.. automodule:: sqlalchemy.sql.visitors
:members:
- :private-members:
\ No newline at end of file
+ :private-members:
+
:paramref:`_orm.Session.future` flag is set on :class:`_orm.sessionmaker`
or :class:`_orm.Session`.
+.. _error_cprf:
+.. _caching_caveats:
+
+Object will not produce a cache key, Performance Implications
+--------------------------------------------------------------
+
+SQLAlchemy as of version 1.4 includes a
+:ref:`SQL compilation caching facility <sql_caching>` which will allow
+Core and ORM SQL constructs to cache their stringified form, along with other
+structural information used to fetch results from the statement, allowing the
+relatively expensive string compilation process to be skipped when another
+structurally equivalent construct is next used. This system
+relies upon functionality that is implemented for all SQL constructs, including
+objects such as :class:`_schema.Column`,
+:func:`_sql.select`, and :class:`_types.TypeEngine` objects, to produce a
+**cache key** which fully represents their state to the degree that it affects
+the SQL compilation process.
+
+If the warnings in question refer to widely used objects such as
+:class:`_schema.Column` objects, and are shown to be affecting the majority of
+SQL constructs being emitted (using the estimation techniques described at
+:ref:`sql_caching_logging`) such that caching is generally not enabled for an
+application, this will negatively impact performance and can in some cases
+effectively produce a **performance degradation** compared to prior SQLAlchemy
+versions. The FAQ at :ref:`faq_new_caching` covers this in additional detail.
+
+Caching disables itself if there's any doubt
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Caching relies on being able to generate a cache key that accurately represents
+the **complete structure** of a statement in a **consistent** fashion. If a particular
+SQL construct (or type) does not have the appropriate directives in place which
+allow it to generate a proper cache key, then caching cannot be safely enabled:
+
+* The cache key must represent the **complete structure**: If the usage of two
+ separate instances of that construct may result in different SQL being
+ rendered, caching the SQL against the first instance of the element using a
+ cache key that does not capture the distinct differences between the first and
+ second elements will result in incorrect SQL being cached and rendered for the
+ second instance.
+
+* The cache key must be **consistent**: If a construct represents state that
+ changes every time, such as a literal value, producing unique SQL for every
+ instance of it, this construct is also not safe to cache, as repeated use of
+ the construct will quickly fill up the statement cache with unique SQL strings
+ that will likely not be used again, defeating the purpose of the cache.
+
+For the above two reasons, SQLAlchemy's caching system is **extremely
+conservative** about deciding to cache the SQL corresponding to an object.
+
+Assertion attributes for caching
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The warning is emitted based on the criteria below. For further detail on
+each, see the section :ref:`faq_new_caching`.
+
+* The :class:`.Dialect` itself (i.e. the module that is specified by the
+ first part of the URL we pass to :func:`_sa.create_engine`, like
+ ``postgresql+psycopg2://``), must indicate it has been reviewed and tested
+ to support caching correctly, which is indicated by the
+ :attr:`.Dialect.supports_statement_cache` attribute being set to ``True``.
+ When using third party dialects, consult with the maintainers of the dialect
+ so that they may follow the :ref:`steps to ensure caching may be enabled
+ <engine_thirdparty_caching>` in their dialect and publish a new release.
+
+* Third party or user defined types that inherit from either
+ :class:`.TypeDecorator` or :class:`.UserDefinedType` must include the
+ :attr:`.ExternalType.cache_ok` attribute in their definition, including for
+ all derived subclasses, following the guidelines described in the docstring
+ for :attr:`.ExternalType.cache_ok`. As before, if these datatypes are
+ imported from third party libraries, consult with the maintainers of that
+ library so that they may provide the necessary changes to their library and
+ publish a new release.
+
+* Third party or user defined SQL constructs that subclass from classes such
+ as :class:`.ClauseElement`, :class:`_schema.Column`, :class:`_dml.Insert`
+ etc, including simple subclasses as well as those which are designed to
+ work with the :ref:`sqlalchemy.ext.compiler_toplevel`, should normally
+ include the :attr:`.HasCacheKey.inherit_cache` attribute set to ``True``
+ or ``False`` based on the design of the construct, following the guidelines
+ described at :ref:`compilerext_caching`.
+
+.. seealso::
+
+ :ref:`sql_caching_logging` - background on observing cache behavior
+ and efficiency
+
+ :ref:`faq_new_caching` - in the :ref:`faq_toplevel` section
+
.. _error_s9r1:
Object is being merged into a Session along the backref cascade
:class: faq
:backlinks: none
+.. _faq_new_caching:
+
+Why is my application slow after upgrading to 1.4 and/or 2.x?
+--------------------------------------------------------------
+
+SQLAlchemy as of version 1.4 includes a
+:ref:`SQL compilation caching facility <sql_caching>` which will allow
+Core and ORM SQL constructs to cache their stringified form, along with other
+structural information used to fetch results from the statement, allowing the
+relatively expensive string compilation process to be skipped when another
+structurally equivalent construct is next used. This system
+relies upon functionality that is implemented for all SQL constructs, including
+objects such as :class:`_schema.Column`,
+:func:`_sql.select`, and :class:`_types.TypeEngine` objects, to produce a
+**cache key** which fully represents their state to the degree that it affects
+the SQL compilation process.
+
+The caching system allows SQLAlchemy 1.4 and above to be more performant than
+SQLAlchemy 1.3 with regards to the time spent converting SQL constructs into
+strings repeatedly. However, this only works if caching is enabled for the
+dialect and SQL constructs in use; if not, string compilation is usually
+similar to that of SQLAlchemy 1.3, with a slight decrease in speed in some
+cases.
+
+There is one case however where if SQLAlchemy's new caching system has been
+disabled (for reasons below), performance for the ORM may be in fact
+significantly poorer than that of 1.3 or other prior releases which is due to
+the lack of caching within ORM lazy loaders and object refresh queries, which
+in the 1.3 and earlier releases used the now-legacy ``BakedQuery`` system. If
+an application is seeing significant (30% or higher) degradations in
+performance (measured in time for operations to complete) when switching to
+1.4, this is the likely cause of the issue, with steps to mitigate below.
+
+.. seealso::
+
+ :ref:`sql_caching` - overview of the caching system
+
+ :ref:`caching_caveats` - additional information regarding the warnings
+ generated for elements that don't enable caching.
+
+Step one - turn on SQL logging and confirm whether or not caching is working
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Here, we want to use the technique described at
+:ref:`engine logging <sql_caching_logging>`, looking for statements with the
+``[no key]`` indicator or even ``[dialect does not support caching]``.
+The indicators we would see for SQL statements that are successfully participating
+in the caching system would be indicating ``[generated in Xs]`` when
+statements are invoked for the first time and then
+``[cached since Xs ago]`` for the vast majority of statements subsequent.
+If ``[no key]`` is prevalent in particular for SELECT statements, or
+if caching is disabled entirely due to ``[dialect does not support caching]``,
+this can be the cause of significant performance degradation.
+
+.. seealso::
+
+ :ref:`sql_caching_logging`
+
+
+Step two - identify what constructs are blocking caching from being enabled
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Assuming statements are not being cached, there should be warnings emitted
+early in the application's log (SQLAlchemy 1.4.28 and above only) indicating
+dialects, :class:`.TypeEngine` objects, and SQL constructs that are not
+participating in caching.
+
+For user defined datatypes such as those which extend :class:`_types.TypeDecorator`
+and :class:`_types.UserDefinedType`, the warnings will look like::
+
+ sqlalchemy.ext.SAWarning: MyType will not produce a cache key because the
+ ``cache_ok`` attribute is not set to True. This can have significant
+ performance implications including some performance degradations in
+ comparison to prior SQLAlchemy versions. Set this attribute to True if this
+ type object's state is safe to use in a cache key, or False to disable this
+ warning.
+
+For custom and third party SQL elements, such as those constructed using
+the techniques described at :ref:`sqlalchemy.ext.compiler_toplevel`, these
+warnings will look like::
+
+ sqlalchemy.exc.SAWarning: Class MyClass will not make use of SQL
+ compilation caching as it does not set the 'inherit_cache' attribute to
+ ``True``. This can have significant performance implications including some
+ performance degradations in comparison to prior SQLAlchemy versions. Set
+ this attribute to True if this object can make use of the cache key
+ generated by the superclass. Alternatively, this attribute may be set to
+ False which will disable this warning.
+
+For custom and third party dialects which make use of the :class:`.Dialect`
+class hierarchy, the warnings will look like::
+
+ sqlalchemy.exc.SAWarning: Dialect database:driver will not make use of SQL
+ compilation caching as it does not set the 'supports_statement_cache'
+ attribute to ``True``. This can have significant performance implications
+ including some performance degradations in comparison to prior SQLAlchemy
+ versions. Dialect maintainers should seek to set this attribute to True
+ after appropriate development and testing for SQLAlchemy 1.4 caching
+ support. Alternatively, this attribute may be set to False which will
+ disable this warning.
+
+
+Step three - enable caching for the given objects and/or seek alternatives
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Steps to mitigate the lack of caching include:
+
+* Review and set :attr:`.ExternalType.cache_ok` to ``True`` for all custom types
+ which extend from :class:`_types.TypeDecorator`,
+ :class:`_types.UserDefinedType`, as well as subclasses of these such as
+ :class:`_types.PickleType`. Set this **only** if the custom type does not
+ include any additional state attributes which affect how it renders SQL::
+
+ class MyCustomType(TypeDecorator):
+ cache_ok = True
+ impl = String
+
+ If the types in use are from a third-party library, consult with the
+ maintainers of that library so that it may be adjusted and released.
+
+ .. seealso::
+
+ :attr:`.ExternalType.cache_ok` - background on requirements to enable
+ caching for custom datatypes.
+
+* Make sure third party dialects set :attr:`.Dialect.supports_statement_cache`
+ to ``True``. What this indicates is that the maintainers of a third party
+ dialect have made sure their dialect works with SQLAlchemy 1.4 or greater,
+ and that their dialect doesn't include any compilation features which may get
+ in the way of caching. As there are some common compilation patterns which
+ can in fact interfere with caching, it's important that dialect maintainers
+ check and test this carefully, adjusting for any of the legacy patterns
+ which won't work with caching.
+
+ .. seealso::
+
+ :ref:`engine_thirdparty_caching` - background and examples for third-party
+ dialects to participate in SQL statement caching.
+
+* Custom SQL classes, including all DQL / DML constructs one might create
+ using the :ref:`sqlalchemy.ext.compiler_toplevel`, as well as ad-hoc
+ subclasses of objects such as :class:`_schema.Column` or
+ :class:`_schema.Table`. The :attr:`.HasCacheKey.inherit_cache` attribute
+ may be set to ``True`` for trivial subclasses, which do not contain any
+ subclass-specific state information which affects the SQL compilation.
+
+ .. seealso::
+
+ :ref:`compilerext_caching` - guidelines for applying the
+ :attr:`.HasCacheKey.inherit_cache` attribute.
+
+
+.. seealso::
+
+ :ref:`sql_caching` - caching system overview
+
+ :ref:`caching_caveats` - background on warnings emitted when caching
+ is not enabled for specific constructs and/or dialects.
+
+
.. _faq_how_to_profile:
How can I profile a SQLAlchemy powered application?
self.expiration_time = expiration_time
self.ignore_expiration = ignore_expiration
+ # this is not needed as of SQLAlchemy 1.4.28;
+ # UserDefinedOption classes no longer participate in the SQL
+ # compilation cache key
def _gen_cache_key(self, anon_map, bindparams):
return None
def _generate_cache_key(self, statement, parameters, orm_cache):
+ """generate a cache key with which to key the results of a statement.
+
+ This leverages the use of the SQL compilation cache key which is
+ repurposed as a SQL results key.
+
+ """
statement_cache_key = statement._generate_cache_key()
key = statement_cache_key.to_offline_string(
__visit_name__ = "try_cast"
stringify_dialect = "mssql"
+ inherit_cache = True
def __init__(self, *arg, **kw):
"""Create a TRY_CAST expression.
"""
stringify_dialect = "mysql"
+ inherit_cache = False
@property
def inserted(self):
__visit_name__ = "array"
stringify_dialect = "postgresql"
+ inherit_cache = True
def __init__(self, clauses, **kw):
clauses = [
"""
stringify_dialect = "postgresql"
+ inherit_cache = False
@util.memoized_property
def excluded(self):
__visit_name__ = "aggregate_order_by"
stringify_dialect = "postgresql"
+ inherit_cache = False
def __init__(self, target, *order_by):
self.target = coercions.expect(roles.ExpressionElementRole, target)
__visit_name__ = "exclude_constraint"
where = None
+ inherit_cache = False
create_drop_stringify_dialect = "postgresql"
type = HSTORE
name = "hstore"
+ inherit_cache = True
class _HStoreDefinedFunction(sqlfunc.GenericFunction):
type = sqltypes.Boolean
name = "defined"
+ inherit_cache = True
class _HStoreDeleteFunction(sqlfunc.GenericFunction):
type = HSTORE
name = "delete"
+ inherit_cache = True
class _HStoreSliceFunction(sqlfunc.GenericFunction):
type = HSTORE
name = "slice"
+ inherit_cache = True
class _HStoreKeysFunction(sqlfunc.GenericFunction):
type = ARRAY(sqltypes.Text)
name = "akeys"
+ inherit_cache = True
class _HStoreValsFunction(sqlfunc.GenericFunction):
type = ARRAY(sqltypes.Text)
name = "avals"
+ inherit_cache = True
class _HStoreArrayFunction(sqlfunc.GenericFunction):
type = ARRAY(sqltypes.Text)
name = "hstore_to_array"
+ inherit_cache = True
class _HStoreMatrixFunction(sqlfunc.GenericFunction):
type = ARRAY(sqltypes.Text)
name = "hstore_to_matrix"
+ inherit_cache = True
#
"""
stringify_dialect = "sqlite"
+ inherit_cache = False
@util.memoized_property
def excluded(self):
@util.memoized_property
def _supports_statement_cache(self):
- return (
- self.__class__.__dict__.get("supports_statement_cache", False)
- is True
- )
+ ssc = self.__class__.__dict__.get("supports_statement_cache", None)
+ if ssc is None:
+ util.warn(
+ "Dialect %s:%s will not make use of SQL compilation caching "
+ "as it does not set the 'supports_statement_cache' attribute "
+ "to ``True``. This can have "
+ "significant performance implications including some "
+ "performance degradations in comparison to prior SQLAlchemy "
+ "versions. Dialect maintainers should seek to set this "
+ "attribute to True after appropriate development and testing "
+ "for SQLAlchemy 1.4 caching support. Alternatively, this "
+ "attribute may be set to False which will disable this "
+ "warning." % (self.name, self.driver),
+ code="cprf",
+ )
+
+ return bool(ssc)
@util.memoized_property
def _type_memos(self):
from sqlalchemy.sql.expression import ColumnClause
class MyColumn(ColumnClause):
- pass
+ inherit_cache = True
@compiles(MyColumn)
def compile_mycolumn(element, compiler, **kw):
from sqlalchemy.schema import DDLElement
class AlterColumn(DDLElement):
+ inherit_cache = False
def __init__(self, column, cmd):
self.column = column
The second ``visit_alter_table`` will be invoked when any ``postgresql``
dialect is used.
+.. _compilerext_compiling_subelements:
+
Compiling sub-elements of a custom expression construct
=======================================================
from sqlalchemy.sql.expression import Executable, ClauseElement
class InsertFromSelect(Executable, ClauseElement):
+ inherit_cache = False
+
def __init__(self, table, select):
self.table = table
self.select = select
class timestamp(ColumnElement):
type = TIMESTAMP()
+ inherit_cache = True
* :class:`~sqlalchemy.sql.functions.FunctionElement` - This is a hybrid of a
``ColumnElement`` and a "from clause" like object, and represents a SQL
class coalesce(FunctionElement):
name = 'coalesce'
+ inherit_cache = True
@compiles(coalesce)
def compile(element, compiler, **kw):
SQL statement that can be passed directly to an ``execute()`` method. It
is already implicit within ``DDLElement`` and ``FunctionElement``.
+Most of the above constructs also respond to SQL statement caching. A
+subclassed construct will want to define the caching behavior for the object,
+which usually means setting the flag ``inherit_cache`` to the value of
+``False`` or ``True``. See the next section :ref:`compilerext_caching`
+for background.
+
+
+.. _compilerext_caching:
+
+Enabling Caching Support for Custom Constructs
+==============================================
+
+SQLAlchemy as of version 1.4 includes a
+:ref:`SQL compilation caching facility <sql_caching>` which will allow
+equivalent SQL constructs to cache their stringified form, along with other
+structural information used to fetch results from the statement.
+
+For reasons discussed at :ref:`caching_caveats`, the implementation of this
+caching system takes a conservative approach towards including custom SQL
+constructs and/or subclasses within the caching system. This includes that
+any user-defined SQL constructs, including all the examples for this
+extension, will not participate in caching by default unless they positively
+assert that they are able to do so. The :attr:`.HasCacheKey.inherit_cache`
+attribute when set to ``True`` at the class level of a specific subclass
+will indicate that instances of this class may be safely cached, using the
+cache key generation scheme of the immediate superclass. This applies
+for example to the "synopsis" example indicated previously::
+
+ class MyColumn(ColumnClause):
+ inherit_cache = True
+
+ @compiles(MyColumn)
+ def compile_mycolumn(element, compiler, **kw):
+ return "[%s]" % element.name
+
+Above, the ``MyColumn`` class does not include any new state that
+affects its SQL compilation; the cache key of ``MyColumn`` instances will
+make use of that of the ``ColumnClause`` superclass, meaning it will take
+into account the class of the object (``MyColumn``), the string name and
+datatype of the object::
+
+ >>> MyColumn("some_name", String())._generate_cache_key()
+ CacheKey(
+ key=('0', <class '__main__.MyColumn'>,
+ 'name', 'some_name',
+ 'type', (<class 'sqlalchemy.sql.sqltypes.String'>,
+ ('length', None), ('collation', None))
+ ), bindparams=[])
+
+For objects that are likely to be **used liberally as components within many
+larger statements**, such as :class:`_schema.Column` subclasses and custom SQL
+datatypes, it's important that **caching be enabled as much as possible**, as
+this may otherwise negatively affect performance.
+
+An example of an object that **does** contain state which affects its SQL
+compilation is the one illustrated at :ref:`compilerext_compiling_subelements`;
+this is an "INSERT FROM SELECT" construct that combines together a
+:class:`_schema.Table` as well as a :class:`_sql.Select` construct, each of
+which independently affect the SQL string generation of the construct. For
+this class, the example illustrates that it simply does not participate in
+caching::
+
+ class InsertFromSelect(Executable, ClauseElement):
+ inherit_cache = False
+
+ def __init__(self, table, select):
+ self.table = table
+ self.select = select
+
+ @compiles(InsertFromSelect)
+ def visit_insert_from_select(element, compiler, **kw):
+ return "INSERT INTO %s (%s)" % (
+ compiler.process(element.table, asfrom=True, **kw),
+ compiler.process(element.select, **kw)
+ )
+
+While it is also possible that the above ``InsertFromSelect`` could be made to
+produce a cache key that is composed of that of the :class:`_schema.Table` and
+:class:`_sql.Select` components together, the API for this is not at the moment
+fully public. However, for an "INSERT FROM SELECT" construct, which is only
+used by itself for specific operations, caching is not as critical as in the
+previous example.
+
+For objects that are **used in relative isolation and are generally
+standalone**, such as custom :term:`DML` constructs like an "INSERT FROM
+SELECT", **caching is generally less critical** as the lack of caching for such
+a construct will have only localized implications for that specific operation.
+
+
Further Examples
================
class utcnow(expression.FunctionElement):
type = DateTime()
+ inherit_cache = True
@compiles(utcnow, 'postgresql')
def pg_utcnow(element, compiler, **kw):
class greatest(expression.FunctionElement):
type = Numeric()
name = 'greatest'
+ inherit_cache = True
@compiles(greatest)
def default_greatest(element, compiler, **kw):
from sqlalchemy.ext.compiler import compiles
class sql_false(expression.ColumnElement):
- pass
+ inherit_cache = True
@compiles(sql_false)
def default_false(element, compiler, **kw):
_is_internal_proxy = True
+ _cache_key_traversal = [
+ ("key", visitors.ExtendedInternalTraversal.dp_string),
+ ("_parententity", visitors.ExtendedInternalTraversal.dp_multi),
+ ]
+
@property
def _impl_uses_objects(self):
return (
_is_strategy_option = False
-class LoaderOption(ORMOption):
- """Describe a loader modification to an ORM statement at compilation time.
+class CompileStateOption(HasCacheKey, ORMOption):
+ """base for :class:`.ORMOption` classes that affect the compilation of
+ a SQL query and therefore need to be part of the cache key.
+
+ .. note:: :class:`.CompileStateOption` is generally non-public and
+ should not be used as a base class for user-defined options; instead,
+ use :class:`.UserDefinedOption`, which is easier to use as it does not
+ interact with ORM compilation internals or caching.
+
+ :class:`.CompileStateOption` defines an internal attribute
+ ``_is_compile_state=True`` which has the effect of the ORM compilation
+ routines for SELECT and other statements will call upon these options when
+ a SQL string is being compiled. As such, these classes implement
+ :class:`.HasCacheKey` and need to provide robust ``_cache_key_traversal``
+ structures.
+
+ The :class:`.CompileStateOption` class is used to implement the ORM
+ :class:`.LoaderOption` and :class:`.CriteriaOption` classes.
+
+ .. versionadded:: 1.4.28
- .. versionadded:: 1.4
"""
_is_compile_state = True
+ def process_compile_state(self, compile_state):
+ """Apply a modification to a given :class:`.CompileState`."""
+
+ def process_compile_state_replaced_entities(
+ self, compile_state, mapper_entities
+ ):
+ """Apply a modification to a given :class:`.CompileState`,
+ given entities that were replaced by with_only_columns() or
+ with_entities().
+
+ .. versionadded:: 1.4.19
+
+ """
+
+
+class LoaderOption(CompileStateOption):
+ """Describe a loader modification to an ORM statement at compilation time.
+
+ .. versionadded:: 1.4
+
+ """
+
def process_compile_state_replaced_entities(
self, compile_state, mapper_entities
):
"""Apply a modification to a given :class:`.CompileState`."""
-class CriteriaOption(ORMOption):
+class CriteriaOption(CompileStateOption):
"""Describe a WHERE criteria modification to an ORM statement at
compilation time.
"""
- _is_compile_state = True
_is_criteria_option = True
def process_compile_state(self, compile_state):
"""
+ inherit_cache = False
+
def process_compile_state(self, compile_state):
pass
return HasCacheKey._generate_cache_key_for_object(self)
-class ExecutableOption(HasCopyInternals, HasCacheKey):
+class ExecutableOption(HasCopyInternals):
_annotations = util.EMPTY_DICT
__visit_name__ = "executable_option"
+ _is_has_cache_key = False
+
def _clone(self, **kw):
"""Create a shallow copy of this ExecutableOption."""
c = self.__class__.__new__(self.__class__)
"""
self._with_options += tuple(
- coercions.expect(roles.HasCacheKeyRole, opt) for opt in options
+ coercions.expect(roles.ExecutableOptionRole, opt)
+ for opt in options
)
@_generative
from . import operators
from . import roles
from . import visitors
+from .base import ExecutableOption
from .base import Options
from .traversals import HasCacheKey
from .visitors import Visitable
return element
+class ExecutableOptionImpl(RoleImpl):
+ __slots__ = ()
+
+ def _implicit_coercions(
+ self, original_element, resolved, argname=None, **kw
+ ):
+ if isinstance(original_element, ExecutableOption):
+ return original_element
+ else:
+ self._raise_for_expected(original_element, argname, resolved)
+
+ def _literal_coercion(self, element, **kw):
+ return element
+
+
class ExpressionElementImpl(_ColumnCoercions, RoleImpl):
__slots__ = ()
class _DDLCompiles(ClauseElement):
+ _hierarchy_supports_caching = False
+ """disable cache warnings for all _DDLCompiles subclasses. """
+
def _compiler(self, dialect, **kw):
"""Return a compiler appropriate for this ClauseElement, given a
Dialect."""
"""
+ inherit_cache = True
+
@classmethod
def _create_any(cls, expr):
"""Produce an ANY expression.
"""Represent the class of expressions that are like an "index"
operation."""
- pass
+ inherit_cache = True
class GroupedElement(ClauseElement):
class SavepointClause(_IdentifiedClause):
__visit_name__ = "savepoint"
+ inherit_cache = False
class RollbackToSavepointClause(_IdentifiedClause):
__visit_name__ = "rollback_to_savepoint"
+ inherit_cache = False
class ReleaseSavepointClause(_IdentifiedClause):
__visit_name__ = "release_savepoint"
+ inherit_cache = False
class quoted_name(util.MemoizedSlots, util.text_type):
class as_utc(GenericFunction):
type = DateTime
+ inherit_cache = True
print(select(func.as_utc()))
class as_utc(GenericFunction):
type = DateTime
package = "time"
+ inherit_cache = True
The above function would be available from :data:`.func`
using the package name ``time``::
package = "geo"
name = "ST_Buffer"
identifier = "buffer"
+ inherit_cache = True
The above function will render as follows::
package = "geo"
name = quoted_name("ST_Buffer", True)
identifier = "buffer"
+ inherit_cache = True
The above function will render as::
_role_name = "Cacheable Core or ORM object"
+class ExecutableOptionRole(SQLRole):
+ __slots__ = ()
+ _role_name = "ExecutionOption Core or ORM object"
+
+
class LiteralValueRole(SQLRole):
_role_name = "Literal Python value"
class HasCacheKey(object):
+ """Mixin for objects which can produce a cache key.
+
+ .. seealso::
+
+ :class:`.CacheKey`
+
+ :ref:`sql_caching`
+
+ """
+
_cache_key_traversal = NO_CACHE
+
+ _is_has_cache_key = True
+
+ _hierarchy_supports_caching = True
+ """private attribute which may be set to False to prevent the
+ inherit_cache warning from being emitted for a hierarchy of subclasses.
+
+ Currently applies to the DDLElement hierarchy which does not implement
+ caching.
+
+ """
+
+ inherit_cache = None
+ """Indicate if this :class:`.HasCacheKey` instance should make use of the
+ cache key generation scheme used by its immediate superclass.
+
+ The attribute defaults to ``None``, which indicates that a construct has
+ not yet taken into account whether or not its appropriate for it to
+ participate in caching; this is functionally equivalent to setting the
+ value to ``False``, except that a warning is also emitted.
+
+ This flag can be set to ``True`` on a particular class, if the SQL that
+ corresponds to the object does not change based on attributes which
+ are local to this class, and not its superclass.
+
+ .. seealso::
+
+ :ref:`compilerext_caching` - General guideslines for setting the
+ :attr:`.HasCacheKey.inherit_cache` attribute for third-party or user
+ defined SQL constructs.
+
+ """
+
__slots__ = ()
@classmethod
so should only be called once per class.
"""
- inherit = cls.__dict__.get("inherit_cache", False)
+ inherit_cache = cls.__dict__.get("inherit_cache", None)
+ inherit = bool(inherit_cache)
if inherit:
_cache_key_traversal = getattr(cls, "_cache_key_traversal", None)
)
if _cache_key_traversal is None:
cls._generated_cache_key_traversal = NO_CACHE
+ if (
+ inherit_cache is None
+ and cls._hierarchy_supports_caching
+ ):
+ util.warn(
+ "Class %s will not make use of SQL compilation "
+ "caching as it does not set the 'inherit_cache' "
+ "attribute to ``True``. This can have "
+ "significant performance implications including "
+ "some performance degradations in comparison to "
+ "prior SQLAlchemy versions. Set this attribute "
+ "to True if this object can make use of the cache "
+ "key generated by the superclass. Alternatively, "
+ "this attribute may be set to False which will "
+ "disable this warning." % (cls.__name__),
+ code="cprf",
+ )
return NO_CACHE
return _cache_key_traversal_visitor.generate_dispatch(
class CacheKey(namedtuple("CacheKey", ["key", "bindparams"])):
+ """The key used to identify a SQL statement construct in the
+ SQL compilation cache.
+
+ .. seealso::
+
+ :ref:`sql_caching`
+
+ """
+
def __hash__(self):
"""CacheKey itself is not hashable - hash the .key portion"""
tuple(elem._gen_cache_key(anon_map, bindparams) for elem in obj),
)
- visit_executable_options = visit_has_cache_key_list
+ def visit_executable_options(
+ self, attrname, obj, parent, anon_map, bindparams
+ ):
+ if not obj:
+ return ()
+ return (
+ attrname,
+ tuple(
+ elem._gen_cache_key(anon_map, bindparams)
+ for elem in obj
+ if elem._is_has_cache_key
+ ),
+ )
def visit_inspectable_list(
self, attrname, obj, parent, anon_map, bindparams
):
return COMPARE_FAILED
- visit_executable_options = visit_has_cache_key_list
+ def visit_executable_options(
+ self, attrname, left_parent, left, right_parent, right, **kw
+ ):
+ for l, r in util.zip_longest(left, right, fillvalue=None):
+ if (
+ l._gen_cache_key(self.anon_map[0], [])
+ if l._is_has_cache_key
+ else l
+ ) != (
+ r._gen_cache_key(self.anon_map[1], [])
+ if r._is_has_cache_key
+ else r
+ ):
+ return COMPARE_FAILED
def visit_clauseelement(
self, attrname, left_parent, left, right_parent, right, **kw
@property
def _static_cache_key(self):
- if self.cache_ok is None:
+ cache_ok = self.__class__.__dict__.get("cache_ok", None)
+
+ if cache_ok is None:
subtype_idx = self.__class__.__mro__.index(ExternalType)
subtype = self.__class__.__mro__[max(subtype_idx - 1, 0)]
util.warn(
"%s %r will not produce a cache key because "
- "the ``cache_ok`` flag is not set to True. "
- "Set this flag to True if this type object's "
+ "the ``cache_ok`` attribute is not set to True. This can "
+ "have significant performance implications including some "
+ "performance degradations in comparison to prior SQLAlchemy "
+ "versions. Set this attribute to True if this type object's "
"state is safe to use in a cache key, or False to "
- "disable this warning." % (subtype.__name__, self)
+ "disable this warning." % (subtype.__name__, self),
+ code="cprf",
)
- elif self.cache_ok is True:
+ elif cache_ok is True:
return super(ExternalType, self)._static_cache_key
return NO_CACHE
# are the "self.statement" element
c = CheckCompilerAccess(clause).compile(dialect=dialect, **kw)
+ if isinstance(clause, sqltypes.TypeEngine):
+ cache_key_no_warnings = clause._static_cache_key
+ if cache_key_no_warnings:
+ hash(cache_key_no_warnings)
+ else:
+ cache_key_no_warnings = clause._generate_cache_key()
+ if cache_key_no_warnings:
+ hash(cache_key_no_warnings[0])
+
param_str = repr(getattr(c, "params", {}))
if util.py3k:
param_str = param_str.encode("utf-8").decode("ascii", "ignore")
t.update()
.where(t.c.somecolumn == "q")
.values(somecolumn="x")
- .with_hint("XYZ", "mysql"),
+ .with_hint("XYZ", dialect_name="mysql"),
"UPDATE sometable SET somecolumn=:somecolumn "
"WHERE sometable.somecolumn = :somecolumn_1",
)
class MyDialect(SQLiteDialect_pysqlite):
statement_compiler = MyCompiler
+ supports_statement_cache = False
from sqlalchemy.dialects import registry
from sqlalchemy.orm.interfaces import UserDefinedOption
class RelationshipCache(UserDefinedOption):
+ inherit_cache = True
propagate_to_loaders = True
def test_column(self):
class MyThingy(ColumnClause):
+ inherit_cache = False
+
def __init__(self, arg=None):
super(MyThingy, self).__init__(arg or "MYTHINGY!")
def test_no_compile_for_col_label(self):
class MyThingy(FunctionElement):
- pass
+ inherit_cache = True
@compiles(MyThingy)
def visit_thingy(thingy, compiler, **kw):
def test_stateful(self):
class MyThingy(ColumnClause):
+ inherit_cache = False
+
def __init__(self):
super(MyThingy, self).__init__("MYTHINGY!")
def test_callout_to_compiler(self):
class InsertFromSelect(ClauseElement):
+ inherit_cache = False
+
def __init__(self, table, select):
self.table = table
self.select = select
def test_no_default_but_has_a_visit(self):
class MyThingy(ColumnClause):
- pass
+ inherit_cache = False
@compiles(MyThingy, "postgresql")
def visit_thingy(thingy, compiler, **kw):
def test_no_default_has_no_visit(self):
class MyThingy(TypeEngine):
- pass
+ inherit_cache = False
@compiles(MyThingy, "postgresql")
def visit_thingy(thingy, compiler, **kw):
@testing.combinations((True,), (False,))
def test_no_default_proxy_generation(self, named):
class my_function(FunctionElement):
+ inherit_cache = False
if named:
name = "my_function"
type = Numeric()
def test_no_default_message(self):
class MyThingy(ClauseElement):
- pass
+ inherit_cache = False
@compiles(MyThingy, "postgresql")
def visit_thingy(thingy, compiler, **kw):
from sqlalchemy.dialects import postgresql
class MyUtcFunction(FunctionElement):
- pass
+ inherit_cache = True
@compiles(MyUtcFunction)
def visit_myfunc(element, compiler, **kw):
def test_functions_args_noname(self):
class myfunc(FunctionElement):
- pass
+ inherit_cache = True
@compiles(myfunc)
def visit_myfunc(element, compiler, **kw):
class greatest(FunctionElement):
type = Numeric()
name = "greatest"
+ inherit_cache = True
@compiles(greatest)
def default_greatest(element, compiler, **kw):
def test_function_subclasses_one(self):
class Base(FunctionElement):
+ inherit_cache = True
name = "base"
class Sub1(Base):
+ inherit_cache = True
name = "sub1"
class Sub2(Base):
+ inherit_cache = True
name = "sub2"
@compiles(Base)
name = "base"
class Sub1(Base):
+ inherit_cache = True
name = "sub1"
@compiles(Base)
return element.name
class Sub2(Base):
+ inherit_cache = True
name = "sub2"
class SubSub1(Sub1):
+ inherit_cache = True
name = "subsub1"
self.assert_compile(
@testing.fixture()
def insert_fixture(self):
class MyInsert(Executable, ClauseElement):
- pass
+ inherit_cache = True
@compiles(MyInsert)
def _run_myinsert(element, compiler, **kw):
@testing.fixture()
def select_fixture(self):
class MySelect(Executable, ClauseElement):
- pass
+ inherit_cache = True
@compiles(MySelect)
def _run_myinsert(element, compiler, **kw):
id = Column(Integer, primary_key=True)
class MySpecialColumn(Column):
- pass
+ inherit_cache = True
class B(A):
__tablename__ = "b"
from sqlalchemy.orm import selectinload
from sqlalchemy.orm import Session
from sqlalchemy.orm import subqueryload
+from sqlalchemy.orm import synonym
from sqlalchemy.orm import with_expression
from sqlalchemy.orm import with_loader_criteria
from sqlalchemy.orm import with_polymorphic
compare_values=True,
)
+ def test_synonyms(self, registry):
+ """test for issue discovered in #7394"""
+
+ @registry.mapped
+ class User2(object):
+ __table__ = self.tables.users
+
+ name_syn = synonym("name")
+
+ @registry.mapped
+ class Address2(object):
+ __table__ = self.tables.addresses
+
+ name_syn = synonym("email_address")
+
+ self._run_cache_key_fixture(
+ lambda: (
+ User2.id,
+ User2.name,
+ User2.name_syn,
+ Address2.name_syn,
+ Address2.email_address,
+ aliased(User2).name_syn,
+ aliased(User2, name="foo").name_syn,
+ aliased(User2, name="bar").name_syn,
+ ),
+ compare_values=True,
+ )
+
def test_more_with_entities_sanity_checks(self):
"""test issue #6503"""
User, Address, Keyword, Order, Item = self.classes(
assert_raises_message(
exc.ArgumentError,
- "Cacheable Core or ORM object expected, got",
+ "ExecutionOption Core or ORM object expected, got",
select(lambda: User).options,
lambda: subqueryload(User.addresses),
)
class max_(expression.FunctionElement):
name = "max"
+ inherit_cache = True
@compiles(max_)
def visit_max(element, compiler, **kw):
class not_named_max(expression.ColumnElement):
name = "not_named_max"
+ inherit_cache = True
@compiles(not_named_max)
def visit_max(element, compiler, **kw):
from sqlalchemy import literal_column
from sqlalchemy import MetaData
from sqlalchemy import or_
+from sqlalchemy import PickleType
from sqlalchemy import select
from sqlalchemy import String
from sqlalchemy import Table
# the None for cache key will prevent objects
# which contain these elements from being cached.
f1 = Foobar1()
- eq_(f1._generate_cache_key(), None)
+ with expect_warnings(
+ "Class Foobar1 will not make use of SQL compilation caching"
+ ):
+ eq_(f1._generate_cache_key(), None)
f2 = Foobar2()
- eq_(f2._generate_cache_key(), None)
+ with expect_warnings(
+ "Class Foobar2 will not make use of SQL compilation caching"
+ ):
+ eq_(f2._generate_cache_key(), None)
s1 = select(column("q"), Foobar2())
+ # warning is memoized, won't happen the second time
eq_(s1._generate_cache_key(), None)
def test_get_children_no_method(self):
and (
"__init__" in cls.__dict__
or issubclass(cls, AliasedReturnsRows)
+ or "inherit_cache" not in cls.__dict__
)
and not issubclass(cls, (Annotated))
and "orm" not in cls.__module__
eq_(c1, c2)
ne_(c1, c3)
eq_(c1, c4)
+
+ def test_thirdparty_sub_subclass_no_cache(self):
+ class MyType(PickleType):
+ pass
+
+ expr = column("q", MyType()) == 1
+
+ with expect_warnings(
+ r"TypeDecorator MyType\(\) will not produce a cache key"
+ ):
+ is_(expr._generate_cache_key(), None)
+
+ def test_userdefined_sub_subclass_no_cache(self):
+ class MyType(UserDefinedType):
+ cache_ok = True
+
+ class MySubType(MyType):
+ pass
+
+ expr = column("q", MySubType()) == 1
+
+ with expect_warnings(
+ r"UserDefinedType MySubType\(\) will not produce a cache key"
+ ):
+ is_(expr._generate_cache_key(), None)
+
+ def test_userdefined_sub_subclass_cache_ok(self):
+ class MyType(UserDefinedType):
+ cache_ok = True
+
+ class MySubType(MyType):
+ cache_ok = True
+
+ def go1():
+ expr = column("q", MySubType()) == 1
+ return expr
+
+ def go2():
+ expr = column("p", MySubType()) == 1
+ return expr
+
+ c1 = go1()._generate_cache_key()[0]
+ c2 = go1()._generate_cache_key()[0]
+ c3 = go2()._generate_cache_key()[0]
+
+ eq_(c1, c2)
+ ne_(c1, c3)
+
+ def test_thirdparty_sub_subclass_cache_ok(self):
+ class MyType(PickleType):
+ cache_ok = True
+
+ def go1():
+ expr = column("q", MyType()) == 1
+ return expr
+
+ def go2():
+ expr = column("p", MyType()) == 1
+ return expr
+
+ c1 = go1()._generate_cache_key()[0]
+ c2 = go1()._generate_cache_key()[0]
+ c3 = go2()._generate_cache_key()[0]
+
+ eq_(c1, c2)
+ ne_(c1, c3)
# test generic function compile
class fake_func(GenericFunction):
+ inherit_cache = True
__return_type__ = sqltypes.Integer
def __init__(self, arg, **kwargs):
if use_custom:
class MyFunc(FunctionElement):
+ inherit_cache = True
name = "myfunc"
type = Integer()
def test_use_labels_function_element(self):
class max_(FunctionElement):
name = "max"
+ inherit_cache = True
@compiles(max_)
def visit_max(element, compiler, **kw):
def test_custom_default_namespace(self):
class myfunc(GenericFunction):
- pass
+ inherit_cache = True
assert isinstance(func.myfunc(), myfunc)
self.assert_compile(func.myfunc(), "myfunc()")
def test_custom_type(self):
class myfunc(GenericFunction):
type = DateTime
+ inherit_cache = True
assert isinstance(func.myfunc().type, DateTime)
self.assert_compile(func.myfunc(), "myfunc()")
def test_custom_legacy_type(self):
# in case someone was using this system
class myfunc(GenericFunction):
+ inherit_cache = True
__return_type__ = DateTime
assert isinstance(func.myfunc().type, DateTime)
def test_case_sensitive(self):
class MYFUNC(GenericFunction):
+ inherit_cache = True
type = DateTime
assert isinstance(func.MYFUNC().type, DateTime)
def test_custom_w_custom_name(self):
class myfunc(GenericFunction):
+ inherit_cache = True
name = "notmyfunc"
assert isinstance(func.notmyfunc(), myfunc)
def test_custom_w_quoted_name(self):
class myfunc(GenericFunction):
+ inherit_cache = True
name = quoted_name("NotMyFunc", quote=True)
identifier = "myfunc"
def test_custom_w_quoted_name_no_identifier(self):
class myfunc(GenericFunction):
+ inherit_cache = True
name = quoted_name("NotMyFunc", quote=True)
# note this requires that the quoted name be lower cased for
def test_custom_package_namespace(self):
def cls1(pk_name):
class myfunc(GenericFunction):
+ inherit_cache = True
package = pk_name
return myfunc
def test_custom_name(self):
class MyFunction(GenericFunction):
name = "my_func"
+ inherit_cache = True
def __init__(self, *args):
args = args + (3,)
package = "geo"
name = "BufferOne"
identifier = "buf1"
+ inherit_cache = True
class GeoBuffer2(GenericFunction):
type = Integer
name = "BufferTwo"
identifier = "buf2"
+ inherit_cache = True
class BufferThree(GenericFunction):
type = Integer
identifier = "buf3"
+ inherit_cache = True
class GeoBufferFour(GenericFunction):
type = Integer
name = "BufferFour"
identifier = "Buf4"
+ inherit_cache = True
self.assert_compile(func.geo.buf1(), "BufferOne()")
self.assert_compile(func.buf2(), "BufferTwo()")
def test_custom_args(self):
class myfunc(GenericFunction):
- pass
+ inherit_cache = True
self.assert_compile(
myfunc(1, 2, 3), "myfunc(:myfunc_1, :myfunc_2, :myfunc_3)"
from sqlalchemy.ext.compiler import compiles
class myfunc(FunctionElement):
+ inherit_cache = True
type = Date()
@compiles(myfunc)
def _fixture(self):
class SomeColThing(WrapsColumnExpression, ColumnElement):
+ inherit_cache = False
+
def __init__(self, expression):
self.clause = coercions.expect(
roles.ExpressionElementRole, expression
from sqlalchemy.sql import select
from sqlalchemy.sql import table
from sqlalchemy.sql import util as sql_util
+from sqlalchemy.sql.base import ExecutableOption
from sqlalchemy.sql.traversals import HasCacheKey
from sqlalchemy.testing import assert_raises_message
from sqlalchemy.testing import AssertsCompiledSQL
stmt = lambdas.lambda_stmt(lambda: select(column("x")))
- opts = {column("x"), column("y")}
+ class MyUncacheable(ExecutableOption):
+ pass
+
+ opts = {MyUncacheable()}
assert_raises_message(
exc.InvalidRequestError,
return stmt
- s1 = go([column("a"), column("b")])
+ class SomeOpt(HasCacheKey, ExecutableOption):
+ def __init__(self, x):
+ self.x = x
+
+ def _gen_cache_key(self, anon_map, bindparams):
+ return (SomeOpt, self.x)
+
+ s1 = go([SomeOpt("a"), SomeOpt("b")])
- s2 = go([column("a"), column("b")])
+ s2 = go([SomeOpt("a"), SomeOpt("b")])
- s3 = go([column("q"), column("b")])
+ s3 = go([SomeOpt("q"), SomeOpt("b")])
s1key = s1._generate_cache_key()
s2key = s2._generate_cache_key()
return stmt
- class SomeOpt(HasCacheKey):
+ class SomeOpt(HasCacheKey, ExecutableOption):
def _gen_cache_key(self, anon_map, bindparams):
return ("fixed_key",)
return stmt
- class SomeOpt(HasCacheKey):
- pass
+ class SomeOpt(HasCacheKey, ExecutableOption):
+ inherit_cache = False
# generates no key, will not be cached
eq_(SomeOpt()._generate_cache_key(), None)
def test_contains(self):
class MyType(UserDefinedType):
+ cache_ok = True
+
class comparator_factory(UserDefinedType.Comparator):
def contains(self, other, **kw):
return self.op("->")(other)
def test_getitem(self):
class MyType(UserDefinedType):
+ cache_ok = True
+
class comparator_factory(UserDefinedType.Comparator):
def __getitem__(self, index):
return self.op("->")(index)
def test_lshift(self):
class MyType(UserDefinedType):
+ cache_ok = True
+
class comparator_factory(UserDefinedType.Comparator):
def __lshift__(self, other):
return self.op("->")(other)
def test_rshift(self):
class MyType(UserDefinedType):
+ cache_ok = True
+
class comparator_factory(UserDefinedType.Comparator):
def __rshift__(self, other):
return self.op("->")(other)
def test_keyed_targeting_no_label_at_all_one(self, connection):
class not_named_max(expression.ColumnElement):
name = "not_named_max"
+ inherit_cache = True
@compiles(not_named_max)
def visit_max(element, compiler, **kw):
def test_keyed_targeting_no_label_at_all_two(self, connection):
class not_named_max(expression.ColumnElement):
name = "not_named_max"
+ inherit_cache = True
@compiles(not_named_max)
def visit_max(element, compiler, **kw):
return process
class UTypeThree(types.UserDefinedType):
+ cache_ok = True
+
def get_col_spec(self):
return "UTYPETHREE"
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
