>>> x._email
'john@doe.com'
-#### Custom List Classes {@name=customlist}
+#### Entity Collections {@name=entitycollections}
-Feature Status: [Alpha API][alpha_api]
+Mapping a one-to-many or many-to-many relationship results in a collection of
+values accessible through an attribute on the parent instance. By default, this
+collection is a `list`:
-A one-to-many or many-to-many relationship results in a list-holding element being attached to all instances of a class. The actual list is an "instrumented" list, which transparently maintains a relationship to a plain Python list. The implementation of the underlying plain list can be changed to be any object that implements a `list`-style `append` and `__iter__` method. A common need is for a list-based relationship to actually be a dictionary. This can be achieved by subclassing `dict` to have `list`-like behavior.
+ {python}
+ mapper(Parent, properties={
+ children = relation(Child)
+ })
-In this example, a class `MyClass` is defined, which is associated with a parent object `MyParent`. The collection of `MyClass` objects on each `MyParent` object will be a dictionary, storing each `MyClass` instance keyed to its `name` attribute.
+ parent = Parent()
+ parent.children.append(Child())
+ print parent.children[0]
+
+Collections are not limited to lists. Sets, mutable sequences and almost any
+other Python object that can act as a container can be used in place of the
+default list.
{python}
- # a class to be stored in the list
- class MyClass(object):
- def __init__(self, name):
- self.name = name
-
- # create a dictionary that will act like a list, and store
- # instances of MyClass
- class MyDict(dict):
+ # use a set
+ mapper(Parent, properties={
+ children = relation(Child, collection_class=set)
+ })
+
+ parent = Parent()
+ child = Child()
+ parent.children.add(child)
+ assert child in parent.children
+
+##### Custom Entity Collections {@name=customcollections}
+
+You can use your own types for collections as well. For most cases, simply
+inherit from `list` or `set` and add the custom behavior.
+
+Collections in SQLAlchemy are transparently *instrumented*. Instrumentation
+means that normal operations on the collection are tracked and result in changes
+being written to the database at flush time. Additionally, collection
+operations can fire *events* which indicate some secondary operation must take
+place. Examples of a secondary operation include saving the child item in the
+parent's `Session` (i.e. the `save-update` cascade), as well as synchronizing
+the state of a bi-directional relationship (i.e. a `backref`).
+
+The collections package understands the basic interface of lists, sets and dicts and will automatically apply instrumentation to those built-in types and their subclasses. Object-derived types that implement a basic collection interface are detected and instrumented via duck-typing:
+
+ {python}
+ class ListLike(object):
+ def __init__(self):
+ self.data = []
def append(self, item):
- self[item.name] = item
+ self.data.append(item)
+ def remove(self, item):
+ self.data.remove(item)
+ def extend(self, items):
+ self.data.extend(items)
def __iter__(self):
- return self.values()
+ return iter(self.data)
+ def foo(self):
+ return 'foo'
- # parent class
- class MyParent(object):
- pass
-
- # mappers, constructed normally
- mapper(MyClass, myclass_table)
- mapper(MyParent, myparent_table, properties={
- 'myclasses' : relation(MyClass, collection_class=MyDict)
+`append`, `remove`, and `extend` are known list-like methods, and will be
+instrumented automatically. `__iter__` is not a mutator method and won't be
+instrumented, and `foo` won't be either.
+
+Duck-typing (i.e. guesswork) isn't rock-solid, of course, so you can be explicit
+about the interface you are implementing by providing an `__emulates__` class
+attribute:
+
+ {python}
+ class SetLike(object):
+ __emulates__ = set
+
+ def __init__(self):
+ self.data = set()
+ def append(self, item):
+ self.data.add(item)
+ def remove(self, item):
+ self.data.remove(item)
+ def __iter__(self):
+ return iter(self.data)
+
+This class looks list-like because of `append`, but `__emulates__` forces it to
+set-like. `remove` is known to be part of the set interface and will be
+instrumented.
+
+But this class won't work quite yet: a little glue is needed to adapt it for use
+by SQLAlchemy. The ORM needs to know which methods to use to append, remove and
+iterate over members of the collection. When using a type like `list` or `set`,
+the appropriate methods are well-known and used automatically when present.
+This set-like class does not provide the expected `add` method, so we must
+supply an explicit mapping for the ORM via a decorator.
+
+##### Collection Decorators {@name=collectiondecorators}
+
+Decorators can be used to tag the individual methods the ORM needs to manage
+collections. Use them when your class doesn't quite meet the regular interface
+for its container type, or you simply would like to use a different method to
+get the job done.
+
+ {python}
+ from sqlalchemy.orm.collections import collection
+
+ class SetLike(object):
+ __emulates__ = set
+
+ def __init__(self):
+ self.data = set()
+
+ @collection.appender
+ def append(self, item):
+ self.data.add(item)
+
+ def remove(self, item):
+ self.data.remove(item)
+
+ def __iter__(self):
+ return iter(self.data)
+
+And that's all that's needed to complete the example. SQLAlchemy will add
+instances via the `append` method. `remove` and `__iter__` are the default
+methods for sets and will be used for removing and iteration. Default methods
+can be changed as well:
+
+ {python}
+ from sqlalchemy.orm.collections import collection
+
+ class MyList(list):
+ @collection.remover
+ def zark(self, item):
+ # do something special...
+
+ @collection.iterator
+ def hey_use_this_instead_for_iteration(self):
+ # ...
+
+There is no requirement to be list-, or set-like at all. Collection classes can
+be any shape, so long as they have the append, remove and iterate interface
+marked for SQLAlchemy's use. Append and remove methods will be called with a
+mapped entity as the single argument, and iterator methods are called with no
+arguments and must return an iterator.
+
+##### Dictionary-Based Collections {@name=dictcollections}
+
+A `dict` can be used as a collection, but a keying strategy is needed to map
+entities loaded by the ORM to key, value pairs. The
+[collections](rel:docstrings_sqlalchemy.orm.collections) package provides
+several built-in types for dictionary-based collections:
+
+ {python}
+ from sqlalchemy.orm.collections import column_mapped_collection, attr_mapped_collection, mapped_collection
+
+ mapper(Item, items_table, properties={
+ # key by column
+ notes = relation(Note, collection_class=column_mapped_collection(notes_table.c.keyword))
+ # or named attribute
+ notes2 = relation(Note, collection_class=attr_mapped_collection('keyword'))
+ # or any callable
+ notes3 = relation(Note, collection_class=mapped_collection(lambda entity: entity.a + entity.b))
})
-
- # elements on 'myclasses' can be accessed via string keyname
- myparent = MyParent()
- myparent.myclasses.append(MyClass('this is myclass'))
- myclass = myparent.myclasses['this is myclass']
-Note: SQLAlchemy 0.4 has an overhauled and much improved implementation for custom list classes, with some slight API changes.
+ # ...
+ item = Item()
+ item.notes['color'] = Note('color', 'blue')
+ print item.notes['color']
+
+These functions each provide a `dict` subclass with decorated `set` and
+`remove` methods and the keying strategy of your choice.
+
+The
+[collections.MappedCollection](rel:docstrings_sqlalchemy.orm.collections.MappedCollection)
+class can be used as a base class for your custom types or as a mix-in to
+quickly add `dict` collection support to other classes. It uses a keying
+function to delegate to `__setitem__` and `__delitem__`:
+
+ {python}
+ from sqlalchemy.util import OrderedDict
+ from sqlalchemy.orm.collections import MappedCollection
+
+ class NodeMap(OrderedDict, MappedCollection):
+ """Holds 'Node' objects, keyed by the 'name' attribute with insert order maintained."""
+
+ def __init__(self, *args, **kw):
+ MappedCollection.__init__(self, keyfunc=lambda node: node.name)
+ OrderedDict.__init__(self, *args, **kw)
+
+The ORM understands the `dict` interface just like lists and sets, and will
+automatically instrument all dict-like methods if you choose to subclass `dict`
+or provide dict-like collection behavior in a duck-typed class. You must
+decorate appender and remover methods, however- there are no compatible methods
+in the basic dictionary interface for SQLAlchemy to use by default. Iteration
+will go through `itervalues()` unless otherwise decorated.
+
+##### Instrumentation and Custom Types {@name=adv_collections}
+
+Many custom types and existing library classes can be used as a entity
+collection type as-is without further ado. However, it is important to note that
+the instrumentation process _will_ modify the type, adding decorators around
+methods automatically.
+
+The decorations are lightweight and no-op outside of relations, but they do add
+unneeded overhead when triggered elsewhere. When using a library class as a
+collection, it can be good practice to use the "trivial subclass" trick to
+restrict the decorations to just your usage in relations. For example:
+
+ {python}
+ class MyAwesomeList(some.great.library.AwesomeList):
+ pass
+
+ # ... relation(..., collection_class=MyAwesomeList)
+
+The ORM uses this approach for built-ins, quietly substituting a trivial
+subclass when a `list`, `set` or `dict` is used directly.
+
+The collections package provides additional decorators and support for authoring
+custom types. See the [package
+documentation](rel:docstrings_sqlalchemy.orm.collections) for more information
+and discussion of advanced usage and Python 2.3-compatible decoration options.
#### Custom Join Conditions {@name=customjoin}
-"""Support for attributes that hold collections of objects."""
+"""Support for collections of mapped entities.
+
+The collections package supplies the machinery used to inform the ORM of
+collection membership changes. An instrumentation via decoration approach is
+used, allowing arbitrary types (including built-ins) to be used as entity
+collections without requiring inheritance from a base class.
+
+Instrumentation decoration relays membership change events to the
+``InstrumentedCollectionAttribute`` that is currently managing the collection.
+The decorators observe function call arguments and return values, tracking
+entities entering or leaving the collection. Two decorator approaches are
+provided. One is a bundle of generic decorators that map function arguments
+and return values to events::
+
+ from sqlalchemy.orm.collections import collection
+ class MyClass(object):
+ # ...
+
+ @collection.adds(1)
+ def store(self, item):
+ self.data.append(item)
+
+ @collection.removes_return()
+ def pop(self):
+ return self.data.pop()
+
+
+The second approach is a bundle of targeted decorators that wrap appropriate
+append and remove notifiers around the mutation methods present in the
+standard Python ``list``, ``set`` and ``dict`` interfaces. These could be
+specified in terms of generic decorator recipes, but are instead hand-tooled for
+increased efficiency. The targeted decorators occasionally implement
+adapter-like behavior, such as mapping bulk-set methods (``extend``, ``update``,
+``__setslice``, etc.) into the series of atomic mutation events that the ORM
+requires.
+
+The targeted decorators are used internally for automatic instrumentation of
+entity collection classes. Every collection class goes through a
+transformation process roughly like so:
+
+1. If the class is a built-in, substitute a trivial sub-class
+2. Is this class already instrumented?
+3. Add in generic decorators
+4. Sniff out the collection interface through duck-typing
+5. Add targeted decoration to any undecorated interface method
+
+This process modifies the class at runtime, decorating methods and adding some
+bookkeeping properties. This isn't possible (or desirable) for built-in
+classes like ``list``, so trivial sub-classes are substituted to hold
+decoration::
+
+ class InstrumentedList(list):
+ pass
+
+Collection classes can be specified in ``relation(collection_class=)`` as
+types or a function that returns an instance. Collection classes are
+inspected and instrumented during the mapper compilation phase. The
+collection_class callable will be executed once to produce a specimen
+instance, and the type of that specimen will be instrumented. Functions that
+return built-in types like ``lists`` will be adapted to produce instrumented
+instances.
+
+When extending a known type like ``list``, additional decorations are not
+generally not needed. Odds are, the extension method will delegate to a
+method that's already instrumented. For example::
+
+ class QueueIsh(list):
+ def push(self, item):
+ self.append(item)
+ def shift(self):
+ return self.pop(0)
+
+There's no need to decorate these methods. ``append`` and ``pop`` are already
+instrumented as part of the ``list`` interface. Decorating them would fire
+duplicate events, which should be avoided.
+
+The targeted decoration tries not to rely on other methods in the underlying
+collection class, but some are unavoidable. Many depend on 'read' methods
+being present to properly instrument a 'write', for example, ``__setitem__``
+needs ``__getitem__``. "Bulk" methods like ``update`` and ``extend`` may also
+reimplemented in terms of atomic appends and removes, so the ``extend``
+decoration will actually perform many ``append`` operations and not call the
+underlying method at all.
+
+Tight control over bulk operation and the firing of events is also possible by
+implementing the instrumentation internally in your methods. The basic
+instrumentation package works under the general assumption that collection
+mutation will not raise unusual exceptions. If you want to closely
+orchestrate append and remove events with exception management, internal
+instrumentation may be the answer. Within your method,
+``collection_adapter(self)`` will retrieve an object that you can use for
+explicit control over triggering append and remove events.
+
+The owning object and InstrumentedCollectionAttribute are also reachable
+through the adapter, allowing for some very sophisticated behavior.
+"""
from sqlalchemy import exceptions, schema, util as sautil
from sqlalchemy.orm import mapper
"""A dictionary-based collection type with attribute-based keying.
Returns a MappedCollection factory with a keying based on the
- 'attr_name' atribute of entities in the collection.
+ 'attr_name' attribute of entities in the collection.
The key value must be immutable for the lifetime of the object. You
can not, for example, map on foreign key values if those key values will
return lambda: MappedCollection(keyfunc)
class collection(object):
- """Decorators for custom collection classes.
+ """Decorators for entity collection classes.
The decorators fall into two groups: annotations and interception recipes.
The annotating decorators (appender, remover, iterator,
- internally_instrumented) indicate the method's purpose and take no
- arguments. They are not written with parens:
+ internally_instrumented, on_link) indicate the method's purpose and take no
+ arguments. They are not written with parens::
@collection.appender
def append(self, append): ...
The recipe decorators all require parens, even those that take no
- arguments:
+ arguments::
@collection.adds('entity'):
def insert(self, position, entity): ...
The appender method is called with one positional argument: the value
to append. The method will be automatically decorated with 'adds(1)'
- if not already decorated.
+ if not already decorated::
@collection.appender
def add(self, append): ...
The remover method is called with one positional argument: the value
to remove. The method will be automatically decorated with
- 'removes_return()' if not already decorated.
+ 'removes_return()' if not already decorated::
@collection.remover
def zap(self, entity): ...
"""Tag the method as the collection remover.
The iterator method is called with no arguments. It is expected to
- return an iterator over all collection members.
+ return an iterator over all collection members::
@collection.iterator
def __iter__(self): ...
This tag will prevent any decoration from being applied to the method.
Use this if you are orchestrating your own calls to collection_adapter
in one of the basic SQLAlchemy interface methods, or to prevent
- an automatic ABC method decoration from wrapping your implementation.
+ an automatic ABC method decoration from wrapping your implementation::
# normally an 'extend' method on a list-like class would be
# automatically intercepted and re-implemented in terms of
Adds "add to collection" handling to the method. The decorator argument
indicates which method argument holds the SQLAlchemy-relevant value.
- Arguments can be specified positionally (i.e. integer) or by name.
+ Arguments can be specified positionally (i.e. integer) or by name::
@collection.adds(1)
def push(self, item): ...
holds the SQLAlchemy-relevant value to be added, and return value, if
any will be considered the value to remove.
- Arguments can be specified positionally (i.e. integer) or by name.
+ Arguments can be specified positionally (i.e. integer) or by name::
@collection.replaces(2)
def __setitem__(self, index, item): ...
Adds "remove from collection" handling to the method. The decorator
argument indicates which method argument holds the SQLAlchemy-relevant
value to be removed. Arguments can be specified positionally (i.e.
- integer) or by name.
+ integer) or by name::
@collection.removes(1)
def zap(self, item): ...
Adds "remove from collection" handling to the method. The return value
of the method, if any, is considered the value to remove. The method
- arguments are not inspected.
+ arguments are not inspected::
@collection.removes_return()
def pop(self): ...
return getattr(collection, '_sa_adapter', None)
-class CollectionAdaptor(object):
- """Bridges between the orm and arbitrary Python collections.
+class CollectionAdapter(object):
+ """Bridges between the ORM and arbitrary Python collections.
Proxies base-level collection operations (append, remove, iterate)
to the underlying Python collection, and emits add/remove events for
entities entering or leaving the collection.
+
+ The ORM uses an CollectionAdapter exclusively for interaction with
+ entity collections.
"""
def __init__(self, attr, owner, data):
self._data = weakref.ref(data)
self.link_to_self(data)
- owner = property(lambda s: s._owner())
- data = property(lambda s: s._data())
+ owner = property(lambda s: s._owner(),
+ doc="The object that owns the entity collection.")
+ data = property(lambda s: s._data(),
+ doc="The entity collection being adapted.")
def link_to_self(self, data):
+ """Link a collection to this adapter, and fire a link event."""
+
setattr(data, '_sa_adapter', self)
if hasattr(data, '_sa_on_link'):
getattr(data, '_sa_on_link')(self)
def unlink(self, data):
+ """Unlink a collection from any adapter, and fire a link event."""
+
setattr(data, '_sa_adapter', None)
if hasattr(data, '_sa_on_link'):
getattr(data, '_sa_on_link')(None)
def append_with_event(self, item, initiator=None):
+ """Add an entity to the collection, firing mutation events."""
+
getattr(self._data(), '_sa_appender')(item, _sa_initiator=initiator)
def append_without_event(self, item):
+ """Add or restore an entity to the collection, firing no events."""
+
getattr(self._data(), '_sa_appender')(item, _sa_initiator=False)
def remove_with_event(self, item, initiator=None):
+ """Remove an entity from the collection, firing mutation events."""
+
getattr(self._data(), '_sa_remover')(item, _sa_initiator=initiator)
def remove_without_event(self, item):
+ """Remove an entity from the collection, firing no events."""
+
getattr(self._data(), '_sa_remover')(item, _sa_initiator=False)
def clear_with_event(self, initiator=None):
+ """Empty the collection, firing a mutation event for each entity."""
+
for item in list(self):
self.remove_with_event(item, initiator)
def clear_without_event(self):
+ """Empty the collection, firing no events."""
+
for item in list(self):
self.remove_without_event(item)
def __iter__(self):
+ """Iterate over entities in the collection."""
+
return getattr(self._data(), '_sa_iterator')()
def __len__(self):
+ """Count entities in the collection."""
+
return len(list(getattr(self._data(), '_sa_iterator')()))
def __nonzero__(self):
return True
def fire_append_event(self, item, initiator=None):
+ """Notify that a entity has entered the collection.
+
+ Initiator is the InstrumentedAttribute that initiated the membership
+ mutation, and should be left as None unless you are passing along
+ an initiator value from a chained operation.
+ """
+
if initiator is not False and item is not None:
self.attr.fire_append_event(self._owner(), item, initiator)
def fire_remove_event(self, item, initiator=None):
+ """Notify that a entity has entered the collection.
+
+ Initiator is the InstrumentedAttribute that initiated the membership
+ mutation, and should be left as None unless you are passing along
+ an initiator value from a chained operation.
+ """
+
if initiator is not False and item is not None:
self.attr.fire_remove_event(self._owner(), item, initiator)
return wrapper
def _instrument_class(cls):
+ """Modify methods in a class and install instrumentation."""
+
# FIXME: more formally document this as a decoratorless/Python 2.3
# option for specifying instrumentation. (likely doc'd here in code only,
# not in online docs.)
class."""
def _tidy(fn):
- try:
- setattr(fn, '_sa_instrumented', True)
- fn.__doc__ = getattr(getattr(list, fn.__name__), '__doc__')
- except:
- raise
+ setattr(fn, '_sa_instrumented', True)
+ fn.__doc__ = getattr(getattr(list, fn.__name__), '__doc__')
def append(fn):
def append(self, item, _sa_initiator=None):
mapping class."""
def _tidy(fn):
- try:
- setattr(fn, '_sa_instrumented', True)
- fn.__doc__ = getattr(getattr(dict, fn.__name__), '__doc__')
- except:
- raise
+ setattr(fn, '_sa_instrumented', True)
+ fn.__doc__ = getattr(getattr(dict, fn.__name__), '__doc__')
Unspecified=object()
sequence class."""
def _tidy(fn):
- try:
- setattr(fn, '_sa_instrumented', True)
- fn.__doc__ = getattr(getattr(set, fn.__name__), '__doc__')
- except:
- raise
+ setattr(fn, '_sa_instrumented', True)
+ fn.__doc__ = getattr(getattr(set, fn.__name__), '__doc__')
Unspecified=object()
'remover': 'remove',
'iterator': '__iter__',
'_decorators': _set_decorators(), },
- # < 0.4 compatible naming (almost), deprecated- use decorators instead.
- dict: { 'appender': 'set',
- 'remover': 'remove',
- 'iterator': 'itervalues',
+ # decorators are required for dicts and object collections.
+ dict: { 'iterator': 'itervalues',
'_decorators': _dict_decorators(), },
# < 0.4 compatible naming, deprecated- use decorators instead.
- None: { 'appender': 'append',
- 'remover': 'remove',
- 'iterator': 'values', }
+ None: { }
}
"""A basic dictionary-based collection class.
Extends dict with the minimal bag semantics that collection classes require.
- "set" and "remove" are implemented in terms of a keying function: any
+ ``set`` and ``remove`` are implemented in terms of a keying function: any
callable that takes an object and returns an object for use as a dictionary
key.
"""
def __init__(self, keyfunc):
+ """Create a new collection with keying provided by keyfunc.
+
+ keyfunc may be any callable any callable that takes an object and
+ returns an object for use as a dictionary key.
+
+ The keyfunc will be called every time the ORM needs to add a member by
+ value-only (such as when loading instances from the database) or remove
+ a member. The usual cautions about dictionary keying apply-
+ ``keyfunc(object)`` should return the same output for the life of the
+ collection. Keying based on mutable properties can result in
+ unreachable instances "lost" in the collection.
+ """
self.keyfunc = keyfunc
def set(self, value, _sa_initiator=None):
+ """Add an item to the collection, with a key provided by this instance's keyfunc."""
+
key = self.keyfunc(value)
self.__setitem__(key, value, _sa_initiator)
set = collection.internally_instrumented(set)
set = collection.appender(set)
def remove(self, value, _sa_initiator=None):
+ """Remove an item from the collection by value, consulting this instance's keyfunc for the key."""
+
key = self.keyfunc(value)
# Let self[key] raise if key is not in this collection
if self[key] != value:
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
