.. changelog::
:version: 0.9.2
+ .. change::
+ :tags: feature, orm
+
+ Support is improved for supplying a :func:`.join` construct as the
+ target of :paramref:`.relationship.secondary` for the purposes
+ of creating very complex :func:`.relationship` join conditions.
+ The change includes adjustments to query joining, joined eager loading
+ to not render a SELECT subquery, changes to lazy loading such that
+ the "secondary" target is properly included in the SELECT, and
+ changes to declarative to better support specification of a
+ join() object with classes as targets.
+
+ The new use case is somewhat experimental, but a new documentation section
+ has been added.
+
+ .. seealso::
+
+ :ref:`composite_secondary_join`
+
.. change::
:tags: bug, mysql, sql
:tickets: 2917
One potential use case for another mapper to exist at the same time is if we
wanted to load instances of our class not just from the immediate :class:`.Table`
to which it is mapped, but from another selectable that is a derivation of that
-:class:`.Table`. While there technically is a way to create such a :func:`.mapper`,
-using the ``non_primary=True`` option, this approach is virtually never needed.
-Instead, we use the functionality of the :class:`.Query` object to achieve this,
-using a method such as :meth:`.Query.select_from`
-or :meth:`.Query.from_statement` to specify a derived selectable.
+:class:`.Table`. To create a second mapper that only handles querying
+when used explicitly, we can use the :paramref:`.mapper.non_primary` argument.
+In practice, this approach is usually not needed, as we
+can do this sort of thing at query time using methods such as
+:meth:`.Query.select_from`, however it is useful in the rare case that we
+wish to build a :func:`.relationship` to such a mapper. An example of this is
+at :ref:`relationship_non_primary_mapper`.
Another potential use is if we genuinely want instances of our class to
be persisted into different tables at different times; certain kinds of
The default behavior of :func:`.relationship` when constructing a join
is that it equates the value of primary key columns
on one side to that of foreign-key-referring columns on the other.
-We can change this criterion to be anything we'd like using the ``primaryjoin``
-argument, as well as the ``secondaryjoin`` argument in the case when
-a "secondary" table is used.
+We can change this criterion to be anything we'd like using the
+:paramref:`.relationship.primaryjoin`
+argument, as well as the :paramref:`.relationship.secondaryjoin`
+argument in the case when a "secondary" table is used.
In the example below, using the ``User`` class
as well as an ``Address`` class which stores a street address, we
backref - when :func:`.relationship` creates the second relationship in the reverse
direction, it's smart enough to reverse the ``primaryjoin`` and ``secondaryjoin`` arguments.
+.. _composite_secondary_join:
+
+Composite "Secondary" Joins
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. note::
+
+ This section features some new and experimental features of SQLAlchemy.
+
+Sometimes, when one seeks to build a :func:`.relationship` between two tables
+there is a need for more than just two or three tables to be involved in
+order to join them. This is an area of :func:`.relationship` where one seeks
+to push the boundaries of what's possible, and often the ultimate solution to
+many of these exotic use cases needs to be hammered out on the SQLAlchemy mailing
+list.
+
+In more recent versions of SQLAlchemy, the :paramref:`.relationship.secondary`
+parameter can be used in some of these cases in order to provide a composite
+target consisting of multiple tables. Below is an example of such a
+join condition (requires version 0.9.2 at least to function as is)::
+
+ class A(Base):
+ __tablename__ = 'a'
+
+ id = Column(Integer, primary_key=True)
+ b_id = Column(ForeignKey('b.id'))
+
+ d = relationship("D",
+ secondary="join(B, D, B.d_id == D.id)."
+ "join(C, C.d_id == D.id)",
+ primaryjoin="and_(A.b_id == B.id, A.id == C.a_id)",
+ secondaryjoin="D.id == B.d_id",
+ uselist=False
+ )
+
+ class B(Base):
+ __tablename__ = 'b'
+
+ id = Column(Integer, primary_key=True)
+ d_id = Column(ForeignKey('d.id'))
+
+ class C(Base):
+ __tablename__ = 'c'
+
+ id = Column(Integer, primary_key=True)
+ a_id = Column(ForeignKey('a.id'))
+ d_id = Column(ForeignKey('d.id'))
+
+ class D(Base):
+ __tablename__ = 'd'
+
+ id = Column(Integer, primary_key=True)
+
+In the above example, we provide all three of :paramref:`.relationship.secondary`,
+:paramref:`.relationship.primaryjoin`, and :paramref:`.relationship.secondaryjoin`,
+in the declarative style referring to the named tables ``a``, ``b``, ``c``, ``d``
+directly. A query from ``A`` to ``D`` looks like:
+
+.. sourcecode:: python+sql
+
+ sess.query(A).join(A.d).all()
+
+ {opensql}SELECT a.id AS a_id, a.b_id AS a_b_id
+ FROM a JOIN (
+ b AS b_1 JOIN d AS d_1 ON b_1.d_id = d_1.id
+ JOIN c AS c_1 ON c_1.d_id = d_1.id)
+ ON a.b_id = b_1.id AND a.id = c_1.a_id JOIN d ON d.id = b_1.d_id
+
+In the above example, we take advantage of being able to stuff multiple
+tables into a "secondary" container, so that we can join across many
+tables while still keeping things "simple" for :func:`.relationship`, in that
+there's just "one" table on both the "left" and the "right" side; the
+complexity is kept within the middle.
+
+.. versionadded:: 0.9.2 Support is improved for allowing a :func:`.join()`
+ construct to be used directly as the target of the :paramref:`.relationship.secondary`
+ argument, including support for joins, eager joins and lazy loading,
+ as well as support within declarative to specify complex conditions such
+ as joins involving class names as targets.
+
+.. _relationship_non_primary_mapper:
+
+Relationship to Non Primary Mapper
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In the previous section, we illustrated a technique where we used
+:paramref:`.relationship.secondary` in order to place additional tables
+within a join condition. There is one complex join case where even this technique is not
+sufficient; when we seek to join from ``A`` to ``B``, making use of any
+number of ``C``, ``D``, etc. in between, however there are also join conditions
+between ``A`` and ``B`` *directly*. In this case, the join from ``A`` to ``B``
+may be difficult to express with just a complex ``primaryjoin`` condition, as the
+intermediary tables may need special handling, and it is also not expressable
+with a ``secondary`` object, since the ``A->secondary->B`` pattern does not support
+any references between ``A`` and ``B`` directly. When this **extremely advanced**
+case arises, we can resort to creating a second mapping as a target for
+the relationship. This is where we use :func:`.mapper` in order to make a mapping to
+a class that includes all the additional tables we need for this join.
+In order to produce this mapper as an "alternative" mapping for our class,
+we use the :paramref:`.mapper.non_primary` flag.
+
+Below illustrates a :func:`.relationship` with a simple join from ``A`` to
+``B``, however the primaryjoin condition is augmented with two additional
+entities ``C`` and ``D``, which also must have rows that line up with
+the rows in both ``A`` and ``B`` simultaneously::
+
+ class A(Base):
+ __tablename__ = 'a'
+
+ id = Column(Integer, primary_key=True)
+ b_id = Column(ForeignKey('b.id'))
+
+ class B(Base):
+ __tablename__ = 'b'
+
+ id = Column(Integer, primary_key=True)
+
+ class C(Base):
+ __tablename__ = 'c'
+
+ id = Column(Integer, primary_key=True)
+ a_id = Column(ForeignKey('a.id'))
+
+ class D(Base):
+ __tablename__ = 'd'
+
+ id = Column(Integer, primary_key=True)
+ c_id = Column(ForeignKey('c.id'))
+ b_id = Column(ForeignKey('b.id'))
+
+ # 1. set up the join() as a variable, so we can refer
+ # to it in the mapping multiple times.
+ j = join(B, D, D.b_id == B.id).join(C, C.id == D.c_id)
+
+ # 2. Create a new mapper() to B, with non_primary=True.
+ # Columns in the join with the same name must be
+ # disambiguated within the mapping, using named properties.
+ B_viacd = mapper(B, j, non_primary=True, properties={
+ "b_id": [j.c.b_id, j.c.d_b_id],
+ "d_id": j.c.d_id
+ })
+
+ A.b = relationship(B_viacd, primaryjoin=A.b_id == B_viacd.c.b_id)
+
+In the above case, our non-primary mapper for ``B`` will emit for
+additional columns when we query; these can be ignored:
+
+.. sourcecode:: python+sql
+
+ sess.query(A).join(A.b).all()
+
+ {opensql}SELECT a.id AS a_id, a.b_id AS a_b_id, d_1.id AS d_1_id,
+ b_1.id AS b_1_id, d_1.b_id AS d_1_b_id, d_1.c_id AS d_1_c_id,
+ c_1.id AS c_1_id, c_1.a_id AS c_1_a_id
+ FROM a LEFT OUTER JOIN (b AS b_1 JOIN d AS d_1 ON d_1.b_id = b_1.id
+ JOIN c AS c_1 ON c_1.id = d_1.c_id) ON a.b_id = b_1.id
+
Building Query-Enabled Properties
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
from ...schema import _get_table_key
from ...orm import class_mapper, interfaces
from ... import util
+from ... import inspection
from ... import exc
import weakref
" directly to a Column)." % key)
return getattr(self.cls, key)
+inspection._inspects(_GetColumns)(
+ lambda target: inspection.inspect(target.cls))
+
class _GetTable(object):
def __init__(self, key, metadata):
mapping of the class to an alternate selectable, for loading
only.
- The ``non_primary`` feature is rarely needed with modern
- usage.
+ :paramref:`.Mapper.non_primary` is not an often used option, but
+ is useful in some specific :func:`.relationship` cases.
+
+ .. seealso::
+
+ :ref:`relationship_non_primary_mapper`
:param order_by: A single :class:`.Column` or list of :class:`.Column`
objects for which selection operations should use as the default
strategy_class=None, _local_remote_pairs=None,
query_class=None,
info=None):
- """Provide a relationship of a primary Mapper to a secondary Mapper.
+ """Provide a relationship between two mapped classes.
This corresponds to a parent-child or associative table relationship. The
constructed class is an instance of :class:`.RelationshipProperty`.
+ For an overview of basic patterns with :func:`.relationship` as
+ used with Declarative, see :ref:`relationship_patterns`.
+
A typical :func:`.relationship`, used in a classical mapping::
mapper(Parent, properties={
:param secondary:
for a many-to-many relationship, specifies the intermediary
- table, and is an instance of :class:`.Table`. The ``secondary`` keyword
- argument should generally only
+ table, and is typically an instance of :class:`.Table`.
+ In less common circumstances, the argument may also be specified
+ as an :class:`.Alias` construct, or even a :class:`.Join` construct.
+
+ The ``secondary`` keyword argument should generally only
be used for a table that is not otherwise expressed in any class
mapping, unless this relationship is declared as view only, otherwise
conflicting persistence operations can occur.
also be passed as a callable function which is evaluated at
mapper initialization time.
+ .. seealso::
+
+ :ref:`relationships_many_to_many`
+
+ .. versionadded:: 0.9.2 :paramref:`.relationship.secondary` works
+ more effectively when referring to a :class:`.Join` instance.
+
:param active_history=False:
When ``True``, indicates that the "previous" value for a
many-to-one reference should be loaded when replaced, if
which is evaluated at mapper initialization time, and may be passed as a
Python-evaluable string when using Declarative.
+ .. seealso::
+
+ :ref:`relationship_primaryjoin`
+
:param remote_side:
used for self-referential relationships, indicates the column or
list of columns that form the "remote side" of the relationship.
which is evaluated at mapper initialization time, and may be passed as a
Python-evaluable string when using Declarative.
+ .. seealso::
+
+ :ref:`relationship_primaryjoin`
+
:param single_parent=(True|False):
when True, installs a validator which will prevent objects
from being associated with more than one parent at a time.
if aliased:
if secondary is not None:
- secondary = secondary.alias()
+ secondary = secondary.alias(flat=True)
primary_aliasizer = ClauseAdapter(secondary)
secondary_aliasizer = \
ClauseAdapter(dest_selectable,
def _emit_lazyload(self, strategy_options, session, state, ident_key, passive):
q = session.query(self.mapper)._adapt_all_clauses()
+
+ if self.parent_property.secondary is not None:
+ q = q.select_from(self.mapper, self.parent_property.secondary)
+
q = q._with_invoke_all_eagers(False)
pending = not state.key
"user_1.firstname || :firstname_1 || user_1.lastname"
)
+ def test_string_dependency_resolution_asselectable(self):
+ class A(Base):
+ __tablename__ = 'a'
+
+ id = Column(Integer, primary_key=True)
+ b_id = Column(ForeignKey('b.id'))
+
+ d = relationship("D",
+ secondary="join(B, D, B.d_id == D.id)."
+ "join(C, C.d_id == D.id)",
+ primaryjoin="and_(A.b_id == B.id, A.id == C.a_id)",
+ secondaryjoin="D.id == B.d_id",
+ )
+
+ class B(Base):
+ __tablename__ = 'b'
+
+ id = Column(Integer, primary_key=True)
+ d_id = Column(ForeignKey('d.id'))
+
+ class C(Base):
+ __tablename__ = 'c'
+
+ id = Column(Integer, primary_key=True)
+ a_id = Column(ForeignKey('a.id'))
+ d_id = Column(ForeignKey('d.id'))
+
+ class D(Base):
+ __tablename__ = 'd'
+
+ id = Column(Integer, primary_key=True)
+ s = Session()
+ self.assert_compile(
+ s.query(A).join(A.d),
+ "SELECT a.id AS a_id, a.b_id AS a_b_id FROM a JOIN "
+ "(b AS b_1 JOIN d AS d_1 ON b_1.d_id = d_1.id "
+ "JOIN c AS c_1 ON c_1.d_id = d_1.id) ON a.b_id = b_1.id "
+ "AND a.id = c_1.a_id JOIN d ON d.id = b_1.d_id",
+ )
+
def test_string_dependency_resolution_no_table(self):
class User(Base, fixtures.ComparableEntity):
class Blub(Bar):
def __repr__(self):
- return "Blub id %d, data %s, bars %s, foos %s" % (self.id, self.data, repr([b for b in self.bars]), repr([f for f in self.foos]))
+ return "Blub id %d, data %s, bars %s, foos %s" % (
+ self.id, self.data, repr([b for b in self.bars]),
+ repr([f for f in self.foos]))
mapper(Blub, blub, inherits=Bar, properties={
- 'bars':relationship(Bar, secondary=blub_bar, lazy='joined'),
- 'foos':relationship(Foo, secondary=blub_foo, lazy='joined'),
+ 'bars': relationship(Bar, secondary=blub_bar, lazy='joined'),
+ 'foos': relationship(Foo, secondary=blub_foo, lazy='joined'),
})
sess = create_session()
])).order_by(Node.id).all(),
[Node(data='n1'), Node(data='n2')]
)
+
+
backref, create_session, configure_mappers, \
clear_mappers, sessionmaker, attributes,\
Session, composite, column_property, foreign,\
- remote, synonym
+ remote, synonym, joinedload
from sqlalchemy.orm.interfaces import ONETOMANY, MANYTOONE, MANYTOMANY
from sqlalchemy.testing import eq_, startswith_, AssertsCompiledSQL, is_
from sqlalchemy.testing import fixtures
sa.orm.configure_mappers()
+class SecondaryNestedJoinTest(fixtures.MappedTest, AssertsCompiledSQL,
+ testing.AssertsExecutionResults):
+ """test support for a relationship where the 'secondary' table is a
+ compound join().
+
+ join() and joinedload() should use a "flat" alias, lazyloading needs
+ to ensure the join renders.
+
+ """
+ run_setup_mappers = 'once'
+ run_inserts = 'once'
+ run_deletes = None
+
+ @classmethod
+ def define_tables(cls, metadata):
+ Table('a', metadata,
+ Column('id', Integer, primary_key=True, test_needs_autoincrement=True),
+ Column('name', String(30)),
+ Column('b_id', ForeignKey('b.id'))
+ )
+ Table('b', metadata,
+ Column('id', Integer, primary_key=True, test_needs_autoincrement=True),
+ Column('name', String(30)),
+ Column('d_id', ForeignKey('d.id'))
+ )
+ Table('c', metadata,
+ Column('id', Integer, primary_key=True, test_needs_autoincrement=True),
+ Column('name', String(30)),
+ Column('a_id', ForeignKey('a.id')),
+ Column('d_id', ForeignKey('d.id'))
+ )
+ Table('d', metadata,
+ Column('id', Integer, primary_key=True, test_needs_autoincrement=True),
+ Column('name', String(30)),
+ )
+
+ @classmethod
+ def setup_classes(cls):
+ class A(cls.Comparable):
+ pass
+ class B(cls.Comparable):
+ pass
+ class C(cls.Comparable):
+ pass
+ class D(cls.Comparable):
+ pass
+
+ @classmethod
+ def setup_mappers(cls):
+ A, B, C, D = cls.classes.A, cls.classes.B, cls.classes.C, cls.classes.D
+ a, b, c, d = cls.tables.a, cls.tables.b, cls.tables.c, cls.tables.d
+ j = sa.join(b, d, b.c.d_id == d.c.id).join(c, c.c.d_id == d.c.id)
+ #j = join(b, d, b.c.d_id == d.c.id).join(c, c.c.d_id == d.c.id).alias()
+ mapper(A, a, properties={
+ "b": relationship(B),
+ "d": relationship(D, secondary=j,
+ primaryjoin=and_(a.c.b_id == b.c.id, a.c.id == c.c.a_id),
+ secondaryjoin=d.c.id == b.c.d_id,
+ #primaryjoin=and_(a.c.b_id == j.c.b_id, a.c.id == j.c.c_a_id),
+ #secondaryjoin=d.c.id == j.c.b_d_id,
+ uselist=False
+ )
+ })
+ mapper(B, b, properties={
+ "d": relationship(D)
+ })
+ mapper(C, c, properties={
+ "a": relationship(A),
+ "d": relationship(D)
+ })
+ mapper(D, d)
+
+ @classmethod
+ def insert_data(cls):
+ A, B, C, D = cls.classes.A, cls.classes.B, cls.classes.C, cls.classes.D
+ sess = Session()
+ a1, a2, a3, a4 = A(name='a1'), A(name='a2'), A(name='a3'), A(name='a4')
+ b1, b2, b3, b4 = B(name='b1'), B(name='b2'), B(name='b3'), B(name='b4')
+ c1, c2, c3, c4 = C(name='c1'), C(name='c2'), C(name='c3'), C(name='c4')
+ d1, d2 = D(name='d1'), D(name='d2')
+
+ a1.b = b1
+ a2.b = b2
+ a3.b = b3
+ a4.b = b4
+
+ c1.a = a1
+ c2.a = a2
+ c3.a = a2
+ c4.a = a4
+
+ c1.d = d1
+ c2.d = d2
+ c3.d = d1
+ c4.d = d2
+
+ b1.d = d1
+ b2.d = d1
+ b3.d = d2
+ b4.d = d2
+
+ sess.add_all([a1, a2, a3, a4, b1, b2, b3, b4, c1, c2, c4, c4, d1, d2])
+ sess.commit()
+
+ def test_render_join(self):
+ A, D = self.classes.A, self.classes.D
+ sess = Session()
+ self.assert_compile(
+ sess.query(A).join(A.d),
+ "SELECT a.id AS a_id, a.name AS a_name, a.b_id AS a_b_id "
+ "FROM a JOIN (b AS b_1 JOIN d AS d_1 ON b_1.d_id = d_1.id "
+ "JOIN c AS c_1 ON c_1.d_id = d_1.id) ON a.b_id = b_1.id "
+ "AND a.id = c_1.a_id JOIN d ON d.id = b_1.d_id",
+ dialect="postgresql"
+ )
+
+ def test_render_joinedload(self):
+ A, D = self.classes.A, self.classes.D
+ sess = Session()
+ self.assert_compile(
+ sess.query(A).options(joinedload(A.d)),
+ "SELECT a.id AS a_id, a.name AS a_name, a.b_id AS a_b_id, "
+ "d_1.id AS d_1_id, d_1.name AS d_1_name FROM a LEFT OUTER JOIN "
+ "(b AS b_1 JOIN d AS d_2 ON b_1.d_id = d_2.id JOIN c AS c_1 "
+ "ON c_1.d_id = d_2.id JOIN d AS d_1 ON d_1.id = b_1.d_id) "
+ "ON a.b_id = b_1.id AND a.id = c_1.a_id",
+ dialect="postgresql"
+ )
+
+ def test_render_lazyload(self):
+ from sqlalchemy.testing.assertsql import CompiledSQL
+
+ A, D = self.classes.A, self.classes.D
+ sess = Session()
+ a1 = sess.query(A).filter(A.name == 'a1').first()
+
+ def go():
+ a1.d
+
+ # here, the "lazy" strategy has to ensure the "secondary"
+ # table is part of the "select_from()", since it's a join().
+ # referring to just the columns wont actually render all those
+ # join conditions.
+ self.assert_sql_execution(
+ testing.db,
+ go,
+ CompiledSQL(
+ "SELECT d.id AS d_id, d.name AS d_name FROM b "
+ "JOIN d ON b.d_id = d.id JOIN c ON c.d_id = d.id "
+ "WHERE :param_1 = b.id AND :param_2 = c.a_id AND d.id = b.d_id",
+ {'param_1': a1.id, 'param_2': a1.id}
+ )
+ )
+
+ mapping = {
+ "a1": "d1",
+ "a2": None,
+ "a3": None,
+ "a4": "d2"
+ }
+
+ def test_join(self):
+ A, D = self.classes.A, self.classes.D
+ sess = Session()
+
+ for a, d in sess.query(A, D).outerjoin(A.d):
+ eq_(self.mapping[a.name], d.name if d is not None else None)
+
+
+ def test_joinedload(self):
+ A, D = self.classes.A, self.classes.D
+ sess = Session()
+
+ for a in sess.query(A).options(joinedload(A.d)):
+ d = a.d
+ eq_(self.mapping[a.name], d.name if d is not None else None)
+
+ def test_lazyload(self):
+ A, D = self.classes.A, self.classes.D
+ sess = Session()
+
+ for a in sess.query(A):
+ d = a.d
+ eq_(self.mapping[a.name], d.name if d is not None else None)
+
class InvalidRelationshipEscalationTest(_RelationshipErrors, fixtures.MappedTest):
@classmethod
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
