.. _session_partitioning:
-Partitioning Strategies
-=======================
+Partitioning Strategies (e.g. multiple database backends per Session)
+=====================================================================
Simple Vertical Partitioning
----------------------------
-Vertical partitioning places different kinds of objects, or different tables,
-across multiple databases::
+Vertical partitioning places different classes, class hierarchies,
+or mapped tables, across multiple databases, by configuring the
+:class:`.Session` with the :paramref:`.Session.binds` argument. This
+argument receives a dictionary that contains any combination of
+ORM-mapped classes, arbitrary classes within a mapped hierarchy (such
+as declarative base classes or mixins), :class:`.Table` objects,
+and :class:`.Mapper` objects as keys, which then refer typically to
+:class:`.Engine` or less typically :class:`.Connection` objects as targets.
+The dictionary is consulted whenever the :class:`.Session` needs to
+emit SQL on behalf of a particular kind of mapped class in order to locate
+the appropriate source of database connectivity::
engine1 = create_engine('postgresql://db1')
engine2 = create_engine('postgresql://db2')
- Session = sessionmaker(twophase=True)
+ Session = sessionmaker()
# bind User operations to engine 1, Account operations to engine 2
Session.configure(binds={User:engine1, Account:engine2})
session = Session()
-Above, operations against either class will make usage of the :class:`.Engine`
-linked to that class. Upon a flush operation, similar rules take place
-to ensure each class is written to the right database.
+Above, SQL operations against either class will make usage of the :class:`.Engine`
+linked to that class. The functionality is comprehensive across both
+read and write operations; a :class:`.Query` that is against entities
+mapped to ``engine1`` (determined by looking at the first entity in the
+list of items requested) will make use of ``engine1`` to run the query. A
+flush operation will make use of **both** engines on a per-class basis as it
+flushes objects of type ``User`` and ``Account``.
+
+In the more common case, there are typically base or mixin classes that can be
+used to distinguish between operations that are destined for different database
+connections. The :paramref:`.Session.binds` argument can accomodate any
+arbitrary Python class as a key, which will be used if it is found to be in the
+``__mro__`` (Python method resolution order) for a particular mapped class.
+Supposing two declarative bases are representing two different database
+connections::
+
+ BaseA = declarative_base()
+
+ BaseB = declarative_base()
+
+ class User(BaseA):
+ # ...
+
+ class Address(BaseA):
+ # ...
+
-The transactions among the multiple databases can optionally be coordinated
-via two phase commit, if the underlying backend supports it. See
-:ref:`session_twophase` for an example.
+ class GameInfo(BaseB):
+ # ...
+
+ class GameStats(BaseB):
+ # ...
+
+
+ Session = sessionmaker()
+
+ # all User/Address operations will be on engine 1, all
+ # Game operations will be on engine 2
+ Session.configure(binds={BaseA:engine1, BaseB:engine2})
+
+Above, classes which descend from ``BaseA`` and ``BaseB`` will have their
+SQL operations routed to one of two engines based on which superclass
+they descend from, if any. In the case of a class that descends from more
+than one "bound" superclass, the superclass that is highest in the target
+class' hierarchy will be chosen to represent which engine should be used.
+
+.. seealso::
+
+ :paramref:`.Session.binds`
+
+
+Coordination of Transactions for a multiple-engine Session
+----------------------------------------------------------
+
+One caveat to using multiple bound engines is in the case where a commit
+operation may fail on one backend after the commit has succeeded on another.
+This is an inconsistency problem that in relational databases is solved
+using a "two phase transaction", which adds an additional "prepare" step
+to the commit sequence that allows for multiple databases to agree to commit
+before actually completing the transaction.
+
+Due to limited support within DBAPIs, SQLAlchemy has limited support for two-
+phase transactions across backends. Most typically, it is known to work well
+with the PostgreSQL backend and to a lesser extent with the MySQL backend.
+However, the :class:`.Session` is fully capable of taking advantage of the two
+phase transaction feature when the backend supports it, by setting the
+:paramref:`.Session.use_twophase` flag within :class:`.sessionmaker` or
+:class:`.Session`. See :ref:`session_twophase` for an example.
+
+
+.. _session_custom_partitioning:
Custom Vertical Partitioning
----------------------------
using an approach such as that of using the declarative ``__abstract__``
keyword, described at :ref:`declarative_abstract`.
+.. seealso::
+
+ `Django-style Database Routers in SQLAlchemy <http://techspot.zzzeek.org/2012/01/11/django-style-database-routers-in-sqlalchemy/>`_ - blog post on a more comprehensive example of :meth:`.Session.get_bind`
+
Horizontal Partitioning
-----------------------
Horizontal partitioning partitions the rows of a single table (or a set of
-tables) across multiple databases.
-
-See the "sharding" example: :ref:`examples_sharding`.
+tables) across multiple databases. The SQLAlchemy :class:`.Session`
+contains support for this concept, however to use it fully requires that
+:class:`.Session` and :class:`.Query` subclasses are used. A basic version
+of these subclasses are available in the :ref:`horizontal_sharding_toplevel`
+ORM extension. An example of use is at: :ref:`examples_sharding`.
.. _bulk_operations:
operations performed by this session will execute via this
connectable.
- :param binds: An optional dictionary which contains more granular
- "bind" information than the ``bind`` parameter provides. This
- dictionary can map individual :class`.Table`
- instances as well as :class:`~.Mapper` instances to individual
- :class:`.Engine` or :class:`.Connection` objects. Operations which
- proceed relative to a particular :class:`.Mapper` will consult this
- dictionary for the direct :class:`.Mapper` instance as
- well as the mapper's ``mapped_table`` attribute in order to locate
- a connectable to use. The full resolution is described in the
- :meth:`.Session.get_bind`.
- Usage looks like::
+ :param binds: A dictionary which may specify any number of
+ :class:`.Engine` or :class:`.Connection` objects as the source of
+ connectivity for SQL operations on a per-entity basis. The keys
+ of the dictionary consist of any series of mapped classes,
+ arbitrary Python classes that are bases for mapped classes,
+ :class:`.Table` objects and :class:`.Mapper` objects. The
+ values of the dictionary are then instances of :class:`.Engine`
+ or less commonly :class:`.Connection` objects. Operations which
+ proceed relative to a particular mapped class will consult this
+ dictionary for the closest matching entity in order to determine
+ which :class:`.Engine` should be used for a particular SQL
+ operation. The complete heuristics for resolution are
+ described at :meth:`.Session.get_bind`. Usage looks like::
Session = sessionmaker(binds={
SomeMappedClass: create_engine('postgresql://engine1'),
- somemapper: create_engine('postgresql://engine2'),
- some_table: create_engine('postgresql://engine3'),
+ SomeDeclarativeBase: create_engine('postgresql://engine2'),
+ some_mapper: create_engine('postgresql://engine3'),
+ some_table: create_engine('postgresql://engine4'),
})
- Also see the :meth:`.Session.bind_mapper`
- and :meth:`.Session.bind_table` methods.
+ .. seealso::
+
+ :ref:`session_partitioning`
+
+ :meth:`.Session.bind_mapper`
+
+ :meth:`.Session.bind_table`
+
+ :meth:`.Session.get_bind`
+
:param \class_: Specify an alternate class other than
``sqlalchemy.orm.session.Session`` which should be used by the
"Not an acceptable bind target: %s" % key)
def bind_mapper(self, mapper, bind):
- """Associate a :class:`.Mapper` with a "bind", e.g. a :class:`.Engine`
- or :class:`.Connection`.
+ """Associate a :class:`.Mapper` or arbitrary Python class with a
+ "bind", e.g. an :class:`.Engine` or :class:`.Connection`.
- The given mapper is added to a lookup used by the
+ The given entity is added to a lookup used by the
:meth:`.Session.get_bind` method.
+ :param mapper: a :class:`.Mapper` object, or an instance of a mapped
+ class, or any Python class that is the base of a set of mapped
+ classes.
+
+ :param bind: an :class:`.Engine` or :class:`.Connection` object.
+
+ .. seealso::
+
+ :ref:`session_partitioning`
+
+ :paramref:`.Session.binds`
+
+ :meth:`.Session.bind_table`
+
+
"""
self._add_bind(mapper, bind)
def bind_table(self, table, bind):
- """Associate a :class:`.Table` with a "bind", e.g. a :class:`.Engine`
+ """Associate a :class:`.Table` with a "bind", e.g. an :class:`.Engine`
or :class:`.Connection`.
- The given mapper is added to a lookup used by the
+ The given :class:`.Table` is added to a lookup used by the
:meth:`.Session.get_bind` method.
+ :param table: a :class:`.Table` object, which is typically the target
+ of an ORM mapping, or is present within a selectable that is
+ mapped.
+
+ :param bind: an :class:`.Engine` or :class:`.Connection` object.
+
+ .. seealso::
+
+ :ref:`session_partitioning`
+
+ :paramref:`.Session.binds`
+
+ :meth:`.Session.bind_mapper`
+
+
"""
self._add_bind(table, bind)
The order of resolution is:
1. if mapper given and session.binds is present,
- locate a bind based on mapper.
+ locate a bind based first on the mapper in use, then
+ on the mapped class in use, then on any base classes that are
+ present in the ``__mro__`` of the mapped class, from more specific
+ superclasses to more general.
2. if clause given and session.binds is present,
locate a bind based on :class:`.Table` objects
found in the given clause present in session.binds.
6. No bind can be found, :exc:`~sqlalchemy.exc.UnboundExecutionError`
is raised.
+ Note that the :meth:`.Session.get_bind` method can be overridden on
+ a user-defined subclass of :class:`.Session` to provide any kind
+ of bind resolution scheme. See the example at
+ :ref:`session_custom_partitioning`.
+
:param mapper:
Optional :func:`.mapper` mapped class or instance of
:class:`.Mapper`. The bind can be derived from a :class:`.Mapper`
for a bound element, typically a :class:`.Table` associated with
bound :class:`.MetaData`.
+ .. seealso::
+
+ :ref:`session_partitioning`
+
+ :paramref:`.Session.binds`
+
+ :meth:`.Session.bind_mapper`
+
+ :meth:`.Session.bind_table`
+
"""
if mapper is clause is None:
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
