0.4.0
- orm
+
- new collection_class api and implementation [ticket:213]
collections are now instrumented via decorations rather than
proxying. you can now have collections that manage their own
membership, and your class instance will be directly exposed on the
relation property. the changes are transparent for most users.
- - InstrumentedList (as it was) is removed, and relation properties no
- longer have 'clear()', '.data', or any other added methods beyond those
- provided by the collection type. you are free, of course, to add them
- to a custom class.
- - __setitem__-like assignments now fire remove events for the existing
- value, if any.
- - dict-likes used as collection classes no longer need to change __iter__
- semantics- itervalues() is used by default instead. this is a backwards
- incompatible change.
- - subclassing dict for a mapped collection is no longer needed in most cases.
- orm.collections provides canned implementations that key objects by a
- specified column or a custom function of your choice.
- - collection assignment now requires a compatible type- assigning None
- to clear a collection or assigning a list to a dict collection will now
- raise an argument error.
- - AttributeExtension moved to interfaces, and .delete is now .remove
- The event method signature has also been swapped around.
+ - InstrumentedList (as it was) is removed, and relation properties
+ no longer have 'clear()', '.data', or any other added methods
+ beyond those provided by the collection type. you are free, of
+ course, to add them to a custom class.
+ - __setitem__-like assignments now fire remove events for the
+ existing value, if any.
+ - dict-likes used as collection classes no longer need to change
+ __iter__ semantics- itervalues() is used by default instead. this
+ is a backwards incompatible change.
+ - subclassing dict for a mapped collection is no longer needed in
+ most cases. orm.collections provides canned implementations that
+ key objects by a specified column or a custom function of your
+ choice.
+ - collection assignment now requires a compatible type- assigning
+ None to clear a collection or assigning a list to a dict
+ collection will now raise an argument error.
+ - AttributeExtension moved to interfaces, and .delete is now
+ .remove The event method signature has also been swapped around.
+
- major interface pare-down for Query: all selectXXX methods
are deprecated. generative methods are now the standard
way to do things, i.e. filter(), filter_by(), all(), one(),
etc. Deprecated methods are docstring'ed with their
new replacements.
- - removed ancient query.select_by_attributename() capability.
- - added "aliased joins" positional argument to the front of
- filter_by(). this allows auto-creation of joins that are aliased
- locally to the individual filter_by() call. This allows the
- auto-construction of joins which cross the same paths but
- are querying divergent criteria. ClauseElements at the front
- of filter_by() are removed (use filter()).
+ - removed ancient query.select_by_attributename() capability.
+ - added "aliased joins" positional argument to the front of
+ filter_by(). this allows auto-creation of joins that are aliased
+ locally to the individual filter_by() call. This allows the
+ auto-construction of joins which cross the same paths but are
+ querying divergent criteria. ClauseElements at the front of
+ filter_by() are removed (use filter()).
+
+ - added composite column properties. using the composite(cls, *columns)
+ function inside of the "properties" dict, instances of cls will be
+ created/mapped to a single attribute, comprised of the values
+ correponding to *columns [ticket:211]
+
- improved support for custom column_property() attributes which
feature correlated subqueries...work better with eager loading now.
+
- along with recent speedups to ResultProxy, total number of
function calls significantly reduced for large loads.
test/perf/masseagerload.py reports 0.4 as having the fewest number
of function calls across all SA versions (0.1, 0.2, and 0.3)
+
- primary key "collapse" behavior; the mapper will analyze all columns
- in its given selectable for primary key "equivalence", that is, columns
- which are equivalent via foreign key relationship or via an explicit
- inherit_condition. primarily for joined-table inheritance scenarios
- where different named PK columns in inheriting tables should "collapse"
- into a single-valued (or fewer-valued) primary key. fixes things
- like [ticket:611].
+ in its given selectable for primary key "equivalence", that is,
+ columns which are equivalent via foreign key relationship or via an
+ explicit inherit_condition. primarily for joined-table inheritance
+ scenarios where different named PK columns in inheriting tables
+ should "collapse" into a single-valued (or fewer-valued) primary key.
+ fixes things like [ticket:611].
+
- secondary inheritance loading: polymorphic mappers can be
constructed *without* a select_table argument. inheriting mappers
whose tables were not represented in the initial load will issue a
second SQL query immediately, once per instance (i.e. not very
efficient for large lists), in order to load the remaining
columns.
- - secondary inheritance loading can also move its second query into
- a column- level "deferred" load, via the "polymorphic_fetch"
- argument, which can be set to 'select' or 'deferred'
+ - secondary inheritance loading can also move its second query into
+ a column- level "deferred" load, via the "polymorphic_fetch"
+ argument, which can be set to 'select' or 'deferred'
+
- added undefer_group() MapperOption, sets a set of "deferred"
columns joined by a "group" to load as "undeferred".
+
- sql
- significant architectural overhaul to SQL elements (ClauseElement).
all elements share a common "mutability" framework which allows a
from sqlalchemy import util as sautil
from sqlalchemy.orm.mapper import Mapper, object_mapper, class_mapper, mapper_registry
from sqlalchemy.orm.interfaces import SynonymProperty, MapperExtension, EXT_PASS, ExtensionOption
-from sqlalchemy.orm.properties import PropertyLoader, ColumnProperty, BackRef
+from sqlalchemy.orm.properties import PropertyLoader, ColumnProperty, CompositeProperty, BackRef
from sqlalchemy.orm import mapper as mapperlib
from sqlalchemy.orm.query import Query
from sqlalchemy.orm.util import polymorphic_union
from sqlalchemy.orm.session import Session as create_session
from sqlalchemy.orm.session import object_session, attribute_manager
-__all__ = ['relation', 'column_property', 'backref', 'eagerload', 'lazyload', 'noload', 'deferred', 'defer', 'undefer', 'undefer_group', 'extension',
+__all__ = ['relation', 'column_property', 'composite', 'backref', 'eagerload', 'lazyload', 'noload', 'deferred', 'defer', 'undefer', 'undefer_group', 'extension',
'mapper', 'clear_mappers', 'compile_mappers', 'class_mapper', 'object_mapper', 'MapperExtension', 'Query',
'polymorphic_union', 'create_session', 'synonym', 'contains_alias', 'contains_eager', 'EXT_PASS', 'object_session'
]
def column_property(*args, **kwargs):
"""Provide a column-level property for use with a Mapper.
+
+ Column-based properties can normally be applied to the mapper's
+ ``properties`` dictionary using the ``schema.Column`` element directly.
+ Use this function when the given column is not directly present within
+ the mapper's selectable; examples include SQL expressions, functions,
+ and scalar SELECT queries.
+
+ Columns that arent present in the mapper's selectable won't be persisted
+ by the mapper and are effectively "read-only" attributes.
+ """
- Normally, custom column-level properties that represent columns
- directly or indirectly present within the mapped selectable
- can just be added to the ``properties`` dictionary directly,
- in which case this function's usage is not necessary.
-
- In the case of a ``ColumnElement`` directly present within the
- ``properties`` dictionary, the given column is converted to be the exact column
- located within the mapped selectable, in the case that the mapped selectable
- is not the exact parent selectable of the given column, but shares a common
- base table relationship with that column.
-
- Use this function when the column expression being added does not
- correspond to any single column within the mapped selectable,
- such as a labeled function or scalar-returning subquery, to force the element
- to become a mapped property regardless of it not being present within the
- mapped selectable.
+ return ColumnProperty(*args, **kwargs)
+
+def composite(class_, *cols, **kwargs):
+ """Return a composite column-based property for use with a Mapper.
- Note that persistence of instances is driven from the collection of columns
- within the mapped selectable, so column properties attached to a Mapper which have
- no direct correspondence to the mapped selectable will effectively be non-persisted
- attributes.
+ This is very much like a column-based property except the given class
+ is used to construct values composed of one or more columns. The class must
+ implement a constructor with positional arguments matching the order of
+ columns given, as well as a __colset__() method which returns its attributes
+ in column order.
"""
- return ColumnProperty(*args, **kwargs)
+
+ return CompositeProperty(class_, *cols, **kwargs)
def _relation_loader(mapper, secondary=None, primaryjoin=None, secondaryjoin=None, lazy=True, **kwargs):
return PropertyLoader(mapper, secondary, primaryjoin, secondaryjoin, lazy=lazy, **kwargs)
self.__log("adding ColumnProperty %s" % (column_key))
elif isinstance(prop, ColumnProperty):
if prop.parent is not self:
- prop = ColumnProperty(deferred=prop.deferred, group=prop.group, *prop.columns)
+ prop = prop.copy()
prop.set_parent(self)
self.__props[column_key] = prop
prop.columns.append(column)
if prop is None:
return NO_ATTRIBUTE
#print "get column attribute '%s' from instance %s" % (column.key, mapperutil.instance_str(obj))
- return prop.getattr(obj)
+ return prop.getattr(obj, column)
def set_attr_by_column(self, obj, column, value):
"""Set the value of an instance attribute using a Column as the key."""
- self.columntoproperty[column][0].setattr(obj, value)
+ self.columntoproperty[column][0].setattr(obj, value, column)
def save_obj(self, objects, uowtransaction, postupdate=False, post_update_cols=None, single=False):
"""Issue ``INSERT`` and/or ``UPDATE`` statements for a list of objects.
if history:
a = history.added_items()
if len(a):
- params[col.key] = a[0]
+ params[col.key] = prop.get_col_value(col, a[0])
hasdata = True
else:
# doing an INSERT, non primary key col ?
import sets, random
from sqlalchemy.orm.interfaces import *
-__all__ = ['ColumnProperty', 'PropertyLoader', 'BackRef']
+__all__ = ['ColumnProperty', 'CompositeProperty', 'PropertyLoader', 'BackRef']
class ColumnProperty(StrategizedProperty):
"""Describes an object attribute that corresponds to a table column."""
self.columns = list(columns)
self.group = kwargs.pop('group', None)
self.deferred = kwargs.pop('deferred', False)
-
+
def create_strategy(self):
if self.deferred:
return strategies.DeferredColumnLoader(self)
else:
return strategies.ColumnLoader(self)
-
- def getattr(self, object):
+
+ def copy(self):
+ return ColumnProperty(deferred=self.deferred, group=self.group, *self.columns)
+
+ def getattr(self, object, column):
return getattr(object, self.key)
- def setattr(self, object, value):
+ def setattr(self, object, value, column):
setattr(object, self.key, value)
def get_history(self, obj, passive=False):
def compare(self, value):
return self.columns[0] == value
+ def get_col_value(self, column, value):
+ return value
+
ColumnProperty.logger = logging.class_logger(ColumnProperty)
mapper.ColumnProperty = ColumnProperty
+
+class CompositeProperty(ColumnProperty):
+ """subclasses ColumnProperty to provide composite type support."""
+
+ def __init__(self, class_, *columns, **kwargs):
+ super(CompositeProperty, self).__init__(*columns, **kwargs)
+ self.composite_class = class_
+
+ def copy(self):
+ return CompositeProperty(deferred=self.deferred, group=self.group, composite_class=self.composite_class, *self.columns)
+
+ def getattr(self, object, column):
+ obj = getattr(object, self.key)
+ return self.get_col_value(column, obj)
+
+ def setattr(self, object, value, column):
+ obj = getattr(object, self.key, None)
+ if obj is None:
+ obj = self.composite_class(*[None for c in self.columns])
+ for a, b in zip(self.columns, value.__colset__()):
+ if a is column:
+ setattr(obj, b, value)
+
+ def compare(self, value):
+ return sql.and_([a==b for a, b in zip(self.columns, value.__colset__())])
+
+ def get_col_value(self, column, value):
+ for a, b in zip(self.columns, value.__colset__()):
+ if a is column:
+ return b
+
class PropertyLoader(StrategizedProperty):
"""Describes an object property that holds a single item or list
ret = self._extension.get(self, ident, **kwargs)
if ret is not mapper.EXT_PASS:
return ret
+
+ # convert composite types to individual args
+ # TODO: account for the order of columns in the
+ # ColumnProperty it corresponds to
+ if hasattr(ident, '__colset__'):
+ ident = ident.__colset__()
+
key = self.mapper.identity_key(ident)
return self._get(key, ident, **kwargs)
else:
ident = util.to_list(ident)
params = {}
+
for i, primary_key in enumerate(self.primary_key_columns):
params[primary_key._label] = ident[i]
try:
super(ColumnLoader, self).init()
self.columns = self.parent_property.columns
self._should_log_debug = logging.is_debug_enabled(self.logger)
+ self.is_composite = hasattr(self.parent_property, 'composite_class')
def setup_query(self, context, eagertable=None, parentclauses=None, **kwargs):
for c in self.columns:
context.statement.append_column(c)
def init_class_attribute(self):
+ if self.is_composite:
+ self._init_composite_attribute()
+ else:
+ self._init_scalar_attribute()
+
+ def _init_composite_attribute(self):
+ self.logger.info("register managed composite attribute %s on class %s" % (self.key, self.parent.class_.__name__))
+ def copy(obj):
+ return self.parent_property.composite_class(*obj.__colset__())
+ def compare(a, b):
+ for col, aprop, bprop in zip(self.columns, a.__colset__(), b.__colset__()):
+ if not col.type.compare_values(aprop, bprop):
+ return False
+ else:
+ return True
+ sessionlib.attribute_manager.register_attribute(self.parent.class_, self.key, uselist=False, copy_function=copy, compare_function=compare, mutable_scalars=True)
+
+ def _init_scalar_attribute(self):
self.logger.info("register managed attribute %s on class %s" % (self.key, self.parent.class_.__name__))
coltype = self.columns[0].type
sessionlib.attribute_manager.register_attribute(self.parent.class_, self.key, uselist=False, copy_function=coltype.copy_value, compare_function=coltype.compare_values, mutable_scalars=self.columns[0].type.is_mutable())
-
+
def create_row_processor(self, selectcontext, mapper, row):
- if self.columns[0] in row:
+ if self.is_composite:
+ for c in self.columns:
+ if c not in row:
+ break
+ else:
+ def execute(instance, row, isnew, ispostselect=None, **flags):
+ if isnew or ispostselect:
+ if self._should_log_debug:
+ self.logger.debug("populating %s with %s/%s..." % (mapperutil.attribute_str(instance, self.key), row.__class__.__name__, self.columns[0].key))
+ instance.__dict__[self.key] = self.parent_property.composite_class(*[row[c] for c in self.columns])
+ self.logger.debug("Returning active composite column fetcher for %s %s" % (mapper, self.key))
+ return (execute, None)
+
+ elif self.columns[0] in row:
def execute(instance, row, isnew, ispostselect=None, **flags):
if isnew or ispostselect:
if self._should_log_debug:
instance.__dict__[self.key] = row[self.columns[0]]
self.logger.debug("Returning active column fetcher for %s %s" % (mapper, self.key))
return (execute, None)
- else:
- (hosted_mapper, needs_tables) = selectcontext.attributes.get(('polymorphic_fetch', mapper), (None, None))
- if hosted_mapper is None:
- return (None, None)
-
- if hosted_mapper.polymorphic_fetch == 'deferred':
- def execute(instance, row, isnew, **flags):
- if isnew:
- sessionlib.attribute_manager.init_instance_attribute(instance, self.key, callable_=self._get_deferred_loader(instance, mapper, needs_tables))
- self.logger.debug("Returning deferred column fetcher for %s %s" % (mapper, self.key))
- return (execute, None)
- else:
- self.logger.debug("Returning no column fetcher for %s %s" % (mapper, self.key))
- return (None, None)
+
+ (hosted_mapper, needs_tables) = selectcontext.attributes.get(('polymorphic_fetch', mapper), (None, None))
+ if hosted_mapper is None:
+ return (None, None)
+
+ if hosted_mapper.polymorphic_fetch == 'deferred':
+ def execute(instance, row, isnew, **flags):
+ if isnew:
+ sessionlib.attribute_manager.init_instance_attribute(instance, self.key, callable_=self._get_deferred_loader(instance, mapper, needs_tables))
+ self.logger.debug("Returning deferred column fetcher for %s %s" % (mapper, self.key))
+ return (execute, None)
+ else:
+ self.logger.debug("Returning no column fetcher for %s %s" % (mapper, self.key))
+ return (None, None)
def _get_deferred_loader(self, instance, mapper, needs_tables):
def load():
def init(self):
super(DeferredColumnLoader, self).init()
+ if hasattr(self.parent_property, 'composite_class'):
+ raise NotImplementedError("Deferred loading for composite types not implemented yet")
self.columns = self.parent_property.columns
self.group = self.parent_property.group
self._should_log_debug = logging.is_debug_enabled(self.logger)
def init_class_attribute(self):
self.logger.info("register managed attribute %s on class %s" % (self.key, self.parent.class_.__name__))
- sessionlib.attribute_manager.register_attribute(self.parent.class_, self.key, uselist=False, callable_=lambda i:self.setup_loader(i), copy_function=lambda x: self.columns[0].type.copy_value(x), compare_function=lambda x,y:self.columns[0].type.compare_values(x,y), mutable_scalars=self.columns[0].type.is_mutable())
+ sessionlib.attribute_manager.register_attribute(self.parent.class_, self.key, uselist=False, callable_=self.setup_loader, copy_function=self.columns[0].type.copy_value, compare_function=self.columns[0].type.compare_values, mutable_scalars=self.columns[0].type.is_mutable())
def setup_query(self, context, **kwargs):
if self.group is not None and context.attributes.get(('undefer', self.group), False):
"""tests general mapper operations with an emphasis on selecting/loading"""
-from testbase import PersistTest, AssertMixin
+from testbase import PersistTest, AssertMixin, ORMTest
import testbase
import unittest, sys, os
from sqlalchemy import *
print item.item_name
self.assert_sql_count(db, go, 0)
self.assert_(item.item_name == 'item 4')
-
+class CompositeTypesTest(ORMTest):
+ def define_tables(self, metadata):
+ global graphs, edges
+ graphs = Table('graphs', metadata,
+ Column('id', Integer, primary_key=True),
+ Column('version_id', Integer, primary_key=True),
+ Column('name', String(30)))
+
+ edges = Table('edges', metadata,
+ Column('id', Integer, primary_key=True),
+ Column('graph_id', Integer, ForeignKey('graphs.id'), nullable=False),
+ Column('x1', Integer),
+ Column('y1', Integer),
+ Column('x2', Integer),
+ Column('y2', Integer))
+
+ def test_basic(self):
+ class Point(object):
+ def __init__(self, x, y):
+ self.x = x
+ self.y = y
+ def __colset__(self):
+ return [self.x, self.y]
+ def __eq__(self, other):
+ return other.x == self.x and other.y == self.y
+ def __ne__(self, other):
+ return not self.__eq__(other)
+
+ class Graph(object):
+ pass
+ class Edge(object):
+ def __init__(self, start, end):
+ self.start = start
+ self.end = end
+
+ mapper(Graph, graphs, properties={
+ 'edges':relation(Edge)
+ })
+ mapper(Edge, edges, properties={
+ 'start':composite(Point, edges.c.x1, edges.c.y1),
+ 'end':composite(Point, edges.c.x2, edges.c.y2)
+ })
+
+ sess = create_session()
+ g = Graph()
+ g.id = 1
+ g.version_id=1
+ g.edges.append(Edge(Point(3, 4), Point(5, 6)))
+ g.edges.append(Edge(Point(14, 5), Point(2, 7)))
+ sess.save(g)
+ sess.flush()
+
+ sess.clear()
+ g2 = sess.query(Graph).get([g.id, g.version_id])
+ for e1, e2 in zip(g.edges, g2.edges):
+ assert e1.start == e2.start
+ assert e1.end == e2.end
+
+ g2.edges[1].end = Point(18, 4)
+ sess.flush()
+ sess.clear()
+ e = sess.query(Edge).get(g2.edges[1].id)
+ assert e.end == Point(18, 4)
+
+ e.end.x = 19
+ e.end.y = 5
+ sess.flush()
+ sess.clear()
+ assert sess.query(Edge).get(g2.edges[1].id).end == Point(19, 5)
+
+ g.edges[1].end = Point(19, 5)
+
+ sess.clear()
+ def go():
+ g2 = sess.query(Graph).options(eagerload('edges')).get([g.id, g.version_id])
+ for e1, e2 in zip(g.edges, g2.edges):
+ assert e1.start == e2.start
+ assert e1.end == e2.end
+ self.assert_sql_count(testbase.db, go, 1)
+
+ def test_pk(self):
+ """test using a composite type as a primary key"""
+
+ class Version(object):
+ def __init__(self, id, version):
+ self.id = id
+ self.version = version
+ def __colset__(self):
+ return [self.id, self.version]
+ def __eq__(self, other):
+ return other.id == self.id and other.version == self.version
+ def __ne__(self, other):
+ return not self.__eq__(other)
+
+ class Graph(object):
+ def __init__(self, version):
+ self.version = version
+
+ mapper(Graph, graphs, properties={
+ 'version':composite(Version, graphs.c.id, graphs.c.version_id)
+ })
+
+ sess = create_session()
+ g = Graph(Version(1, 1))
+ sess.save(g)
+ sess.flush()
+
+ sess.clear()
+ g2 = sess.query(Graph).get([1, 1])
+ assert g.version == g2.version
+ sess.clear()
+
+ g2 = sess.query(Graph).get(Version(1, 1))
+ assert g.version == g2.version
+
+
+
class NoLoadTest(MapperSuperTest):
def testbasic(self):
"""tests a basic one-to-many lazy load"""
keep_mappers = False
keep_data = False
def setUpAll(self):
- global metadata
- metadata = BoundMetaData(db)
- self.define_tables(metadata)
- metadata.create_all()
- def define_tables(self, metadata):
+ global _otest_metadata
+ _otest_metadata = BoundMetaData(db)
+ self.define_tables(_otest_metadata)
+ _otest_metadata.create_all()
+ def define_tables(self, _otest_metadata):
raise NotImplementedError()
def get_metadata(self):
- return metadata
+ return _otest_metadata
def tearDownAll(self):
- metadata.drop_all()
+ _otest_metadata.drop_all()
def tearDown(self):
if not self.keep_mappers:
clear_mappers()
if not self.keep_data:
- for t in metadata.table_iterator(reverse=True):
+ for t in _otest_metadata.table_iterator(reverse=True):
t.delete().execute().close()
class TestData(object):
projects / thirdparty / sqlalchemy / sqlalchemy.git / commitdiff
