--- /dev/null
+.. change::
+ :tags: bug, orm
+ :tickets: 10348
+
+ Added new capability to the :func:`_orm.selectin_polymorphic` loader option
+ which allows other loader options to be bundled as siblings, referring to
+ one of its subclasses, within the sub-options of parent loader option.
+ Previously, this pattern was only supported if the
+ :func:`_orm.selectin_polymorphic` were at the top level of the options for
+ the query. See new documentation section for example.
+
+ As part of this change, improved the behavior of the
+ :meth:`_orm.Load.selectin_polymorphic` method / loader strategy so that the
+ subclass load does not load most already-loaded columns from the parent
+ table, when the option is used against a class that is already being
+ relationship-loaded. Previously, the logic to load only the subclass
+ columns worked only for a top level class load.
+
+ .. seealso::
+
+ :ref:`polymorphic_selectin_as_loader_option_target_plus_opts`
:doc:`View the ORM setup for this page <_inheritance_setup>`.
SELECTing from the base class vs. specific sub-classes
---------------------------------------------------------
+------------------------------------------------------
A SELECT statement constructed against a class in a joined inheritance
hierarchy will query against the table to which the class is mapped, as well as
the :func:`_orm.selectinload` relationship strategy which is more
sophisticated in this regard and can factor out the JOIN when not needed.
+.. _polymorphic_selectin_as_loader_option_target:
-.. _polymorphic_selectin_w_loader_options:
-
-Combining additional loader options with selectin_polymorphic() subclass loads
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Applying selectin_polymorphic() to an existing eager load
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. Setup code, not for display
>>> session.close()
ROLLBACK
+In addition to :func:`_orm.selectin_polymorphic` being specified as an option
+for a top-level entity loaded by a statement, we may also indicate
+:func:`_orm.selectin_polymorphic` on the target of an existing load.
+As our :doc:`setup <_inheritance_setup>` mapping includes a parent
+``Company`` entity with a ``Company.employees`` :func:`_orm.relationship`
+referring to ``Employee`` entities, we may illustrate a SELECT against
+the ``Company`` entity that eagerly loads all ``Employee`` objects as well as
+all attributes on their subtypes as follows, by applying :meth:`.Load.selectin_polymorphic`
+as a chained loader option; in this form, the first argument is implicit from
+the previous loader option (in this case :func:`_orm.selectinload`), so
+we only indicate the additional target subclasses we wish to load::
+
+ >>> from sqlalchemy.orm import selectinload
+ >>> stmt = select(Company).options(
+ ... selectinload(Company.employees).selectin_polymorphic([Manager, Engineer])
+ ... )
+ >>> for company in session.scalars(stmt):
+ ... print(f"company: {company.name}")
+ ... print(f"employees: {company.employees}")
+ {execsql}BEGIN (implicit)
+ SELECT company.id, company.name
+ FROM company
+ [...] ()
+ SELECT employee.company_id AS employee_company_id, employee.id AS employee_id,
+ employee.name AS employee_name, employee.type AS employee_type
+ FROM employee
+ WHERE employee.company_id IN (?)
+ [...] (1,)
+ SELECT manager.id AS manager_id, employee.id AS employee_id,
+ employee.type AS employee_type,
+ manager.manager_name AS manager_manager_name
+ FROM employee JOIN manager ON employee.id = manager.id
+ WHERE employee.id IN (?) ORDER BY employee.id
+ [...] (1,)
+ SELECT engineer.id AS engineer_id, employee.id AS employee_id,
+ employee.type AS employee_type,
+ engineer.engineer_info AS engineer_engineer_info
+ FROM employee JOIN engineer ON employee.id = engineer.id
+ WHERE employee.id IN (?, ?) ORDER BY employee.id
+ [...] (2, 3)
+ {stop}company: Krusty Krab
+ employees: [Manager('Mr. Krabs'), Engineer('SpongeBob'), Engineer('Squidward')]
+
+.. seealso::
+
+ :ref:`eagerloading_polymorphic_subtypes` - illustrates the equivalent example
+ as above using :func:`_orm.with_polymorphic` instead
+
+
+.. _polymorphic_selectin_w_loader_options:
+
+Applying loader options to the subclasses loaded by selectin_polymorphic
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
The SELECT statements emitted by :func:`_orm.selectin_polymorphic` are themselves
ORM statements, so we may also add other loader options (such as those
documented at :ref:`orm_queryguide_relationship_loaders`) that refer to specific
-subclasses. For example, if we considered that the ``Manager`` mapper had
+subclasses. These options should be applied as **siblings** to a
+:func:`_orm.selectin_polymorphic` option, that is, comma separated within
+:meth:`_sql.select.options`.
+
+For example, if we considered that the ``Manager`` mapper had
a :ref:`one to many <relationship_patterns_o2m>` relationship to an entity
called ``Paperwork``, we could combine the use of
:func:`_orm.selectin_polymorphic` and :func:`_orm.selectinload` to eagerly load
this collection on all ``Manager`` objects, where the sub-attributes of
``Manager`` objects were also themselves eagerly loaded::
- >>> from sqlalchemy.orm import selectinload
>>> from sqlalchemy.orm import selectin_polymorphic
>>> stmt = (
... select(Employee)
... )
... )
>>> objects = session.scalars(stmt).all()
- {execsql}BEGIN (implicit)
- SELECT employee.id, employee.name, employee.type, employee.company_id
+ {execsql}SELECT employee.id, employee.name, employee.type, employee.company_id
FROM employee ORDER BY employee.id
[...] ()
SELECT manager.id AS manager_id, employee.id AS employee_id, employee.type AS employee_type, manager.manager_name AS manager_manager_name
>>> print(objects[0].paperwork)
[Paperwork('Secret Recipes'), Paperwork('Krabby Patty Orders')]
-.. _polymorphic_selectin_as_loader_option_target:
+.. _polymorphic_selectin_as_loader_option_target_plus_opts:
-Applying selectin_polymorphic() to an existing eager load
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Applying loader options when selectin_polymorphic is itself a sub-option
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-In addition to being able to add loader options to the right side of a
-:func:`_orm.selectin_polymorphic` load, we may also indicate
-:func:`_orm.selectin_polymorphic` on the target of an existing load.
-As our :doc:`setup <_inheritance_setup>` mapping includes a parent
-``Company`` entity with a ``Company.employees`` :func:`_orm.relationship`
-referring to ``Employee`` entities, we may illustrate a SELECT against
-the ``Company`` entity that eagerly loads all ``Employee`` objects as well as
-all attributes on their subtypes as follows, by applying :meth:`.Load.selectin_polymorphic`
-as a chained loader option; in this form, the first argument is implicit from
-the previous loader option (in this case :func:`_orm.selectinload`), so
-we only indicate the additional target subclasses we wish to load::
+.. Setup code, not for display
+
+
+ >>> session.close()
+ ROLLBACK
+
+.. versionadded:: 2.0.21
+
+The previous section illustrated
+:func:`_orm.selectin_polymorphic` and :func:`_orm.selectinload` used as siblings
+within a call to :func:`_sql.select.options`. If the target entity is one that
+is already being loaded from a parent relationship, as in the example at
+:ref:`polymorphic_selectin_as_loader_option_target`, we can apply this
+"sibling" pattern using the :meth:`_orm.Load.options` method that applies
+sub-options to a parent, as illustrated at
+:ref:`orm_queryguide_relationship_sub_options`. Below we combine the two examples
+to load `Company.employees`, also loading the attributes for the `Manager` and
+`Engineer` classes, as well as eagerly loading the `Manager.paperwork`
+attribute::
+ >>> from sqlalchemy.orm import selectinload
>>> stmt = select(Company).options(
- ... selectinload(Company.employees).selectin_polymorphic([Manager, Engineer])
+ ... selectinload(Company.employees).options(
+ ... selectin_polymorphic(Employee, [Manager, Engineer]),
+ ... selectinload(Manager.paperwork),
+ ... )
... )
>>> for company in session.scalars(stmt):
... print(f"company: {company.name}")
- ... print(f"employees: {company.employees}")
- {execsql}SELECT company.id, company.name
+ ... for employee in company.employees:
+ ... if isinstance(employee, Manager):
+ ... print(f"manager: {employee.name} paperwork: {employee.paperwork}")
+ BEGIN (implicit)
+ SELECT company.id, company.name
FROM company
[...] ()
- SELECT employee.company_id AS employee_company_id, employee.id AS employee_id,
- employee.name AS employee_name, employee.type AS employee_type
+ SELECT employee.company_id AS employee_company_id, employee.id AS employee_id, employee.name AS employee_name, employee.type AS employee_type
FROM employee
WHERE employee.company_id IN (?)
[...] (1,)
- SELECT manager.id AS manager_id, employee.id AS employee_id, employee.name AS employee_name,
- employee.type AS employee_type, employee.company_id AS employee_company_id,
- manager.manager_name AS manager_manager_name
+ SELECT manager.id AS manager_id, employee.id AS employee_id, employee.type AS employee_type, manager.manager_name AS manager_manager_name
FROM employee JOIN manager ON employee.id = manager.id
WHERE employee.id IN (?) ORDER BY employee.id
[...] (1,)
- SELECT engineer.id AS engineer_id, employee.id AS employee_id, employee.name AS employee_name,
- employee.type AS employee_type, employee.company_id AS employee_company_id,
- engineer.engineer_info AS engineer_engineer_info
+ SELECT paperwork.manager_id AS paperwork_manager_id, paperwork.id AS paperwork_id, paperwork.document_name AS paperwork_document_name
+ FROM paperwork
+ WHERE paperwork.manager_id IN (?)
+ [...] (1,)
+ SELECT engineer.id AS engineer_id, employee.id AS employee_id, employee.type AS employee_type, engineer.engineer_info AS engineer_engineer_info
FROM employee JOIN engineer ON employee.id = engineer.id
WHERE employee.id IN (?, ?) ORDER BY employee.id
[...] (2, 3)
{stop}company: Krusty Krab
- employees: [Manager('Mr. Krabs'), Engineer('SpongeBob'), Engineer('Squidward')]
-
-.. seealso::
-
- :ref:`eagerloading_polymorphic_subtypes` - illustrates the equivalent example
- as above using :func:`_orm.with_polymorphic` instead
+ manager: Mr. Krabs paperwork: [Paperwork('Secret Recipes'), Paperwork('Krabby Patty Orders')]
Configuring selectin_polymorphic() on mappers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The behavior of :func:`_orm.selectin_polymorphic` may be configured on specific
mappers so that it takes place by default, by using the
.. _with_polymorphic_mapper_config:
Configuring with_polymorphic() on mappers
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
As is the case with :func:`_orm.selectin_polymorphic`, the
:func:`_orm.with_polymorphic` construct also supports a mapper-configured
.. _eagerloading_polymorphic_subtypes:
Eager Loading of Polymorphic Subtypes
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The use of :meth:`_orm.PropComparator.of_type` illustrated with the
:meth:`.Select.join` method in the previous section may also be applied
Optimizing Attribute Loads for Single Inheritance
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. Setup code, not for display
orig_query = context.query
- options = (enable_opt,) + orig_query._with_options + (disable_opt,)
+ if path.parent:
+ enable_opt_lcl = enable_opt._prepend_path(path)
+ disable_opt_lcl = disable_opt._prepend_path(path)
+ else:
+ enable_opt_lcl = enable_opt
+ disable_opt_lcl = disable_opt
+ options = (
+ (enable_opt_lcl,) + orig_query._with_options + (disable_opt_lcl,)
+ )
+
q2 = q.options(*options)
q2._compile_options = context.compile_state.default_compile_options
# chopped at odd intervals as well so this is less flexible
_PathRepresentation = Tuple[_PathElementType, ...]
+# NOTE: these names are weird since the array is 0-indexed,
+# the "_Odd" entries are at 0, 2, 4, etc
_OddPathRepresentation = Sequence["_InternalEntityType[Any]"]
_EvenPathRepresentation = Sequence[Union["MapperProperty[Any]", str]]
def _path_for_compare(self) -> Optional[_PathRepresentation]:
return self.path
+ def odd_element(self, index: int) -> _InternalEntityType[Any]:
+ return self.path[index] # type: ignore
+
def set(self, attributes: Dict[Any, Any], key: Any, value: Any) -> None:
log.debug("set '%s' on path '%s' to '%s'", key, self, value)
attributes[(key, self.natural_path)] = value
@property
def root_entity(self) -> _InternalEntityType[Any]:
- return cast("_InternalEntityType[Any]", self.path[0])
+ return self.odd_element(0)
@property
def entity_path(self) -> PathRegistry:
__slots__ = (
"path",
"context",
+ "additional_source_entities",
)
_traverse_internals = [
visitors.InternalTraversal.dp_has_cache_key_list,
),
("propagate_to_loaders", visitors.InternalTraversal.dp_boolean),
+ (
+ "additional_source_entities",
+ visitors.InternalTraversal.dp_has_cache_key_list,
+ ),
]
_cache_key_traversal = None
path: PathRegistry
context: Tuple[_LoadElement, ...]
+ additional_source_entities: Tuple[_InternalEntityType[Any], ...]
def __init__(self, entity: _EntityType[Any]):
insp = cast("Union[Mapper[Any], AliasedInsp[Any]]", inspect(entity))
self.path = insp._path_registry
self.context = ()
self.propagate_to_loaders = False
+ self.additional_source_entities = ()
def __str__(self) -> str:
return f"Load({self.path[0]})"
@classmethod
- def _construct_for_existing_path(cls, path: PathRegistry) -> Load:
+ def _construct_for_existing_path(
+ cls, path: AbstractEntityRegistry
+ ) -> Load:
load = cls.__new__(cls)
load.path = path
load.context = ()
load.propagate_to_loaders = False
+ load.additional_source_entities = ()
return load
def _adapt_cached_option_to_uncached_option(
) -> ORMOption:
return self._adjust_for_extra_criteria(context)
+ def _prepend_path(self, path: PathRegistry) -> Load:
+ cloned = self._clone()
+ cloned.context = tuple(
+ element._prepend_path(path) for element in self.context
+ )
+ return cloned
+
def _adjust_for_extra_criteria(self, context: QueryContext) -> Load:
"""Apply the current bound parameters in a QueryContext to all
occurrences "extra_criteria" stored within this ``Load`` object,
assert cloned.propagate_to_loaders == self.propagate_to_loaders
- if not orm_util._entity_corresponds_to_use_path_impl(
- cast("_InternalEntityType[Any]", parent.path[-1]),
- cast("_InternalEntityType[Any]", cloned.path[0]),
+ if not any(
+ orm_util._entity_corresponds_to_use_path_impl(
+ elem, cloned.path.odd_element(0)
+ )
+ for elem in (parent.path.odd_element(-1),)
+ + parent.additional_source_entities
):
if len(cloned.path) > 1:
attrname = cloned.path[1]
if cloned.context:
parent.context += cloned.context
+ parent.additional_source_entities += (
+ cloned.additional_source_entities
+ )
@_generative
def options(self, *opts: _AbstractLoad) -> Self:
)
if load_element:
self.context += (load_element,)
+ assert opts is not None
+ self.additional_source_entities += cast(
+ "Tuple[_InternalEntityType[Any]]", opts["entities"]
+ )
else:
for attr in attrs:
return cloned
- def _prepend_path_from(
- self, parent: Union[Load, _LoadElement]
- ) -> _LoadElement:
+ def _prepend_path_from(self, parent: Load) -> _LoadElement:
"""adjust the path of this :class:`._LoadElement` to be
a subpath of that of the given parent :class:`_orm.Load` object's
path.
which is in turn part of the :meth:`_orm.Load.options` method.
"""
- cloned = self._clone()
- assert cloned.strategy == self.strategy
- assert cloned.local_opts == self.local_opts
- assert cloned.is_class_strategy == self.is_class_strategy
-
- if not orm_util._entity_corresponds_to_use_path_impl(
- cast("_InternalEntityType[Any]", parent.path[-1]),
- cast("_InternalEntityType[Any]", cloned.path[0]),
+ if not any(
+ orm_util._entity_corresponds_to_use_path_impl(
+ elem,
+ self.path.odd_element(0),
+ )
+ for elem in (parent.path.odd_element(-1),)
+ + parent.additional_source_entities
):
raise sa_exc.ArgumentError(
- f'Attribute "{cloned.path[1]}" does not link '
+ f'Attribute "{self.path[1]}" does not link '
f'from element "{parent.path[-1]}".'
)
- cloned.path = PathRegistry.coerce(parent.path[0:-1] + cloned.path[:])
+ return self._prepend_path(parent.path)
+
+ def _prepend_path(self, path: PathRegistry) -> _LoadElement:
+ cloned = self._clone()
+
+ assert cloned.strategy == self.strategy
+ assert cloned.local_opts == self.local_opts
+ assert cloned.is_class_strategy == self.is_class_strategy
+
+ cloned.path = PathRegistry.coerce(path[0:-1] + cloned.path[:])
return cloned
(
" Did you mean to use "
f'"{path[-2]}'
- f'.of_type({parent_entity_str})"?'
+ f'.of_type({parent_entity_str})" or "loadopt.options('
+ f"selectin_polymorphic({path[-2].mapper.class_.__name__}, "
+ f'[{parent_entity_str}]), ...)" ?'
if not path_is_of_type
and not path[-1].is_aliased_class
and orm_util._entity_corresponds_to(
from .context import _MapperEntity
from .context import ORMCompileState
from .mapper import Mapper
+ from .path_registry import AbstractEntityRegistry
from .query import Query
from .relationships import RelationshipProperty
from ..engine import Row
return self.mapper.class_
@property
- def _path_registry(self) -> PathRegistry:
+ def _path_registry(self) -> AbstractEntityRegistry:
if self._use_mapper_path:
return self.mapper._path_registry
else:
CompiledSQL(
"SELECT managers.person_id AS managers_person_id, "
"people.person_id AS people_person_id, "
- "people.company_id AS people_company_id, "
- "people.name AS people_name, people.type AS people_type, "
+ "people.type AS people_type, "
"managers.status AS managers_status, "
"managers.manager_name AS managers_manager_name "
"FROM people JOIN managers "
CompiledSQL(
"SELECT engineers.person_id AS engineers_person_id, "
"people.person_id AS people_person_id, "
- "people.company_id AS people_company_id, "
- "people.name AS people_name, people.type AS people_type, "
+ "people.type AS people_type, "
"engineers.status AS engineers_status, "
"engineers.engineer_name AS engineers_engineer_name, "
"engineers.primary_language AS engineers_primary_language "
)
eq_(result, [self.c1, self.c2])
+ def test_load_company_plus_employees_w_paperwork(self):
+ s = fixture_session()
+ q = (
+ s.query(Company)
+ .options(
+ selectinload(Company.employees).options(
+ selectin_polymorphic(Person, [Engineer, Manager]),
+ selectinload(Engineer.machines),
+ # NOTE: if this is selectinload(Person.paperwork),
+ # we get duplicate loads from the subclasses which is
+ # not ideal
+ )
+ )
+ .order_by(Company.company_id)
+ )
+
+ result = self.assert_sql_execution(
+ testing.db,
+ q.all,
+ CompiledSQL(
+ "SELECT companies.company_id AS companies_company_id, "
+ "companies.name AS companies_name FROM companies "
+ "ORDER BY companies.company_id",
+ {},
+ ),
+ CompiledSQL(
+ "SELECT people.company_id AS people_company_id, "
+ "people.person_id AS people_person_id, "
+ "people.name AS people_name, people.type AS people_type "
+ "FROM people WHERE people.company_id "
+ "IN (__[POSTCOMPILE_primary_keys]) "
+ "ORDER BY people.person_id",
+ {"primary_keys": [1, 2]},
+ ),
+ AllOf(
+ CompiledSQL(
+ "SELECT managers.person_id AS managers_person_id, "
+ "people.person_id AS people_person_id, "
+ "people.type AS people_type, "
+ "managers.status AS managers_status, "
+ "managers.manager_name AS managers_manager_name "
+ "FROM people JOIN managers "
+ "ON people.person_id = managers.person_id "
+ "WHERE people.person_id IN (__[POSTCOMPILE_primary_keys]) "
+ "ORDER BY people.person_id",
+ {"primary_keys": [3, 4]},
+ ),
+ CompiledSQL(
+ "SELECT engineers.person_id AS engineers_person_id, "
+ "people.person_id AS people_person_id, "
+ "people.type AS people_type, "
+ "engineers.status AS engineers_status, "
+ "engineers.engineer_name AS engineers_engineer_name, "
+ "engineers.primary_language AS engineers_primary_language "
+ "FROM people JOIN engineers "
+ "ON people.person_id = engineers.person_id "
+ "WHERE people.person_id IN (__[POSTCOMPILE_primary_keys]) "
+ "ORDER BY people.person_id",
+ {"primary_keys": [1, 2, 5]},
+ ),
+ CompiledSQL(
+ "SELECT machines.engineer_id AS machines_engineer_id, "
+ "machines.machine_id AS machines_machine_id, "
+ "machines.name AS machines_name "
+ "FROM machines "
+ "WHERE machines.engineer_id "
+ "IN (__[POSTCOMPILE_primary_keys]) "
+ "ORDER BY machines.machine_id",
+ {"primary_keys": [1, 2, 5]},
+ ),
+ ),
+ )
+ eq_(result, [self.c1, self.c2])
+
class TestGeometries(GeometryFixtureBase):
def test_threelevel_selectin_to_inline_mapped(self):
CompiledSQL(
"SELECT child_subclass1.id AS child_subclass1_id, "
"child.id AS child_id, "
- "child.parent_id AS child_parent_id, "
"child.type AS child_type "
"FROM child JOIN child_subclass1 "
"ON child.id = child_subclass1.id "
),
CompiledSQL(
"SELECT child_subclass1.id AS child_subclass1_id, "
- "child.id AS child_id, child.parent_id AS child_parent_id, "
+ "child.id AS child_id, "
"child.type AS child_type, other_1.id AS other_1_id, "
"other_1.child_subclass_id AS other_1_child_subclass_id "
"FROM child JOIN child_subclass1 "
exc.ArgumentError,
r'ORM mapped entity or attribute "Sub1.links" does not '
r'link from relationship "Link.child". Did you mean to use '
- r'"Link.child.of_type\(Sub1\)"\?',
+ r'"Link.child.of_type\(Sub1\)"\ or '
+ r'"loadopt.options'
+ r'\(selectin_polymorphic\(Parent, \[Sub1\]\), ...\)" \?',
):
session.query(cls).options(
joinedload(cls.links)
r"ORM mapped entity or attribute "
r'(?:"Mapper\[Engineer\(engineers\)\]"|"Engineer.engineer_name") '
r'does not link from relationship "Company.employees". Did you '
- r'mean to use "Company.employees.of_type\(Engineer\)"\?',
+ r'mean to use "Company.employees.of_type\(Engineer\)" '
+ r'or "loadopt.options'
+ r'\(selectin_polymorphic\(Person, \[Engineer\]\), ...\)" \?',
):
if use_options:
s.query(Company).options(
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
