From: Mike Bayer Date: Sun, 25 Aug 2013 18:22:27 +0000 (-0400) Subject: - add version_id documentation [ticket:867] X-Git-Tag: rel_0_8_3~39 X-Git-Url: http://git.ipfire.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=be594493e6a6af39e2b5512f3b14228811f6e15a;p=thirdparty%2Fsqlalchemy%2Fsqlalchemy.git - add version_id documentation [ticket:867] - add the RETURNING glossary entry - other "seealso's" fixed - add doc for eager_defaults option --- diff --git a/doc/build/core/types.rst b/doc/build/core/types.rst index ccbba5d246..a40363135e 100644 --- a/doc/build/core/types.rst +++ b/doc/build/core/types.rst @@ -343,6 +343,8 @@ many decimal places. Here's a recipe that rounds them down:: value = value.quantize(self.quantize) return value +.. _custom_guid_type: + Backend-agnostic GUID Type ^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/doc/build/glossary.rst b/doc/build/glossary.rst index 7c497df769..8dadfc20da 100644 --- a/doc/build/glossary.rst +++ b/doc/build/glossary.rst @@ -521,3 +521,42 @@ Glossary http://en.wikipedia.org/wiki/Durability_(database_systems) + RETURNING + This is a non-SQL standard clause provided in various forms by + certain backends, which provides the service of returning a result + set upon execution of an INSERT, UPDATE or DELETE statement. Any set + of columns from the matched rows can be returned, as though they were + produced from a SELECT statement. + + The RETURNING clause provides both a dramatic performance boost to + common update/select scenarios, including retrieval of inline- or + default- generated primary key values and defaults at the moment they + were created, as well as a way to get at server-generated + default values in an atomic way. + + An example of RETURNING, idiomatic to Postgresql, looks like:: + + INSERT INTO user_account (name) VALUES ('new name') RETURNING id, timestamp + + Above, the INSERT statement will provide upon execution a result set + which includes the values of the columns ``user_account.id`` and + ``user_account.timestamp``, which above should have been generated as default + values as they are not included otherwise (but note any series of columns + or SQL expressions can be placed into RETURNING, not just default-value columns). + + The backends that currently support + RETURNING or a similar construct are Postgresql, SQL Server, Oracle, + and Firebird. The Postgresql and Firebird implementations are generally + full featured, whereas the implementations of SQL Server and Oracle + have caveats. On SQL Server, the clause is known as "OUTPUT INSERTED" + for INSERT and UPDATE statements and "OUTPUT DELETED" for DELETE statements; + the key caveat is that triggers are not supported in conjunction with this + keyword. On Oracle, it is known as "RETURNING...INTO", and requires that the + value be placed into an OUT paramter, meaning not only is the syntax awkward, + but it can also only be used for one row at a time. + + SQLAlchemy's :meth:`.UpdateBase.returning` system provides a layer of abstraction + on top of the RETURNING systems of these backends to provide a consistent + interface for returning columns. The ORM also includes many optimizations + that make use of RETURNING when available. + diff --git a/doc/build/orm/exceptions.rst b/doc/build/orm/exceptions.rst index 7a760e26fa..f95b26eed4 100644 --- a/doc/build/orm/exceptions.rst +++ b/doc/build/orm/exceptions.rst @@ -2,5 +2,4 @@ ORM Exceptions ============== .. automodule:: sqlalchemy.orm.exc - :members: \ No newline at end of file diff --git a/doc/build/orm/mapper_config.rst b/doc/build/orm/mapper_config.rst index d3dc1cf6b4..4d70d166c2 100644 --- a/doc/build/orm/mapper_config.rst +++ b/doc/build/orm/mapper_config.rst @@ -1115,6 +1115,125 @@ of these events. .. autofunction:: reconstructor + +.. _mapper_version_counter: + +Configuring a Version Counter +============================= + +The :class:`.Mapper` supports management of a :term:`version id column`, which +is a single table column that increments or otherwise updates its value +each time an ``UPDATE`` to the mapped table occurs. This value is checked each +time the ORM emits an ``UPDATE`` or ``DELETE`` against the row to ensure that +the value held in memory matches the database value. + +The purpose of this feature is to detect when two concurrent transactions +are modifying the same row at roughly the same time, or alternatively to provide +a guard against the usage of a "stale" row in a system that might be re-using +data from a previous transaction without refreshing (e.g. if one sets ``expire_on_commit=False`` +with a :class:`.Session`, it is possible to re-use the data from a previous +transaction). + +.. topic:: Concurrent transaction updates + + When detecting concurrent updates within transactions, it is typically the + case that the database's transaction isolation level is below the level of + :term:`repeatable read`; otherwise, the transaction will not be exposed + to a new row value created by a concurrent update which conflicts with + the locally updated value. In this case, the SQLAlchemy versioning + feature will typically not be useful for in-transaction conflict detection, + though it still can be used for cross-transaction staleness detection. + + The database that enforces repeatable reads will typically either have locked the + target row against a concurrent update, or is employing some form + of multi version concurrency control such that it will emit an error + when the transaction is committed. SQLAlchemy's version_id_col is an alternative + which allows version tracking to occur for specific tables within a transaction + that otherwise might not have this isolation level set. + + .. seealso:: + + `Repeatable Read Isolation Level `_ - Postgresql's implementation of repeatable read, including a description of the error condition. + +Simple Version Counting +----------------------- + +The most straightforward way to track versions is to add an integer column +to the mapped table, then establish it as the ``version_id_col`` within the +mapper options:: + + class User(Base): + __tablename__ = 'user' + + id = Column(Integer, primary_key=True) + version_id = Column(Integer, nullable=False) + name = Column(String(50), nullable=False) + + __mapper_args__ = { + "version_id_col": version_id + } + +Above, the ``User`` mapping tracks integer versions using the column +``version_id``. When an object of type ``User`` is first flushed, the +``version_id`` column will be given a value of "1". Then, an UPDATE +of the table later on will always be emitted in a manner similar to the +following:: + + UPDATE user SET version_id=:version_id, name=:name + WHERE user.id = :user_id AND user.version_id = :user_version_id + {"name": "new name", "version_id": 2, "user_id": 1, "user_version_id": 1} + +The above UPDATE statement is updating the row that not only matches +``user.id = 1``, it also is requiring that ``user.version_id = 1``, where "1" +is the last version identifier we've been known to use on this object. +If a transaction elsewhere has modifed the row independently, this version id +will no longer match, and the UPDATE statement will report that no rows matched; +this is the condition that SQLAlchemy tests, that exactly one row matched our +UPDATE (or DELETE) statement. If zero rows match, that indicates our version +of the data is stale, and a :class:`.StaleDataError` is raised. + +.. _custom_version_counter: + +Custom Version Counters / Types +------------------------------- + +Other kinds of values or counters can be used for versioning. Common types include +dates and GUIDs. When using an alternate type or counter scheme, SQLAlchemy +provides a hook for this scheme using the ``version_id_generator`` argument, +which accepts a version generation callable. This callable is passed the value of the current +known version, and is expected to return the subsequent version. + +For example, if we wanted to track the versioning of our ``User`` class +using a randomly generated GUID, we could do this (note that some backends +support a native GUID type, but we illustrate here using a simple string):: + + import uuid + + class User(Base): + __tablename__ = 'user' + + id = Column(Integer, primary_key=True) + version_uuid = Column(String(32)) + name = Column(String(50), nullable=False) + + __mapper_args__ = { + 'version_id_col':version_uuid, + 'version_id_generator':lambda version: uuid.uuid4().hex + } + +The persistence engine will call upon ``uuid.uuid4()`` each time a +``User`` object is subject to an INSERT or an UPDATE. In this case, our +version generation function can disregard the incoming value of ``version``, +as the ``uuid4()`` function +generates identifiers without any prerequisite value. If we were using +a sequential versioning scheme such as numeric or a special character system, +we could make use of the given ``version`` in order to help determine the +subsequent value. + +.. seealso:: + + :ref:`custom_guid_type` + Class Mapping API ================= diff --git a/lib/sqlalchemy/orm/__init__.py b/lib/sqlalchemy/orm/__init__.py index 35315d5d1e..0984b3b374 100644 --- a/lib/sqlalchemy/orm/__init__.py +++ b/lib/sqlalchemy/orm/__init__.py @@ -946,6 +946,15 @@ def mapper(class_, local_table=None, *args, **params): See the section :ref:`concrete_inheritance` for an example. + :param eager_defaults: if True, the ORM will immediately fetch the + value of server-generated default values after an INSERT or UPDATE, + rather than leaving them as expired to be fetched on next access. + This can be used for event schemes where the server-generated values + are needed immediately before the flush completes. + This scheme will emit an individual ``SELECT`` statement per row + inserted or updated, which note can add significant performance + overhead. + :param exclude_properties: A list or set of string column names to be excluded from mapping. @@ -968,9 +977,9 @@ def mapper(class_, local_table=None, *args, **params): is passed automatically as a result of the natural class hierarchy of the declared classes. - See also: + .. seealso:: - :ref:`inheritance_toplevel` + :ref:`inheritance_toplevel` :param inherit_condition: For joined table inheritance, a SQL expression which will @@ -1037,10 +1046,10 @@ def mapper(class_, local_table=None, *args, **params): emit an UPDATE statement for the dependent columns during a primary key change. - See also: + .. seealso:: - :ref:`passive_updates` - description of a similar feature as - used with :func:`.relationship` + :ref:`passive_updates` - description of a similar feature as + used with :func:`.relationship` :param polymorphic_on: Specifies the column, attribute, or SQL expression used to determine the target class for an @@ -1129,9 +1138,9 @@ def mapper(class_, local_table=None, *args, **params): thus persisting the value to the ``discriminator`` column in the database. - See also: + .. seealso:: - :ref:`inheritance_toplevel` + :ref:`inheritance_toplevel` :param polymorphic_identity: Specifies the value which identifies this particular class as returned by the @@ -1157,34 +1166,33 @@ def mapper(class_, local_table=None, *args, **params): can be overridden here. :param version_id_col: A :class:`.Column` - that will be used to keep a running version id of mapped entities - in the database. This is used during save operations to ensure that - no other thread or process has updated the instance during the - lifetime of the entity, else a + that will be used to keep a running version id of rows + in the table. This is used to detect concurrent updates or + the presence of stale data in a flush. The methodology is to + detect if an UPDATE statement does not match the last known + version id, a :class:`~sqlalchemy.orm.exc.StaleDataError` exception is - thrown. By default the column must be of :class:`.Integer` type, - unless ``version_id_generator`` specifies a new generation - algorithm. + thrown. + By default, the column must be of :class:`.Integer` type, + unless ``version_id_generator`` specifies an alternative version + generator. - :param version_id_generator: A callable which defines the algorithm - used to generate new version ids. Defaults to an integer - generator. Can be replaced with one that generates timestamps, - uuids, etc. e.g.:: + .. seealso:: - import uuid + :ref:`mapper_version_counter` - discussion of version counting + and rationale. - class MyClass(Base): - __tablename__ = 'mytable' - id = Column(Integer, primary_key=True) - version_uuid = Column(String(32)) + :param version_id_generator: Define how new version ids should + be generated. Defaults to ``None``, which indicates that + a simple integer counting scheme be employed. To provide a custom + versioning scheme, provide a callable function of the form:: - __mapper_args__ = { - 'version_id_col':version_uuid, - 'version_id_generator':lambda version:uuid.uuid4().hex - } + def generate_version(version): + return next_version + + .. seealso:: - The callable receives the current version identifier as its - single argument. + :ref:`custom_version_counter` :param with_polymorphic: A tuple in the form ``(, )`` indicating the default style of "polymorphic" @@ -1196,13 +1204,9 @@ def mapper(class_, local_table=None, *args, **params): indicates a selectable that will be used to query for multiple classes. - See also: - - :ref:`concrete_inheritance` - typically uses ``with_polymorphic`` - to specify a UNION statement to select from. + .. seealso:: - :ref:`with_polymorphic` - usage example of the related - :meth:`.Query.with_polymorphic` method + :ref:`with_polymorphic` """ return Mapper(class_, local_table, *args, **params)